← Writing
22 Sep 2026 · Field Notes

The Agreeable Machine

A flocked nodding-dog ornament on a lamplit side table, looking into a small brass mirror where its reflection nods back, with a laptop glowing at the edge of the frame

First, a position. Psychologists matter. When something is genuinely wrong, careful professional management is what helps, and nothing here argues otherwise.

But most of the time there’s no crisis. There’s a question. You’ve had medical news and want to understand it. You’ve had an argument and want to work out why it landed so hard. You can’t seem to connect with a colleague, or you keep losing the room when you make your case in meetings. None of that is a request for treatment. It’s a wish to understand, and to get a little better at your own life. So what do you do then?

More and more people take those questions to an AI, often at eleven at night. It works, a bit like a CBT diary that answers back. And almost everyone doing it has, at some point, been told they shouldn’t: by a headline, by a professional body’s press release, or by the chatbot itself, which will politely suggest they “consult a qualified professional”.

So is there a rule? And if not, what should you actually watch for?

Nobody has banned you from thinking

There is no rule against educating yourself. Read carefully, the professional bodies’ objections are narrower than the headlines. The American Psychological Association tells people not to rely on chatbots or wellness apps for psychotherapy or treatment, and wants chatbots stopped from posing as licensed therapists. It doesn’t tell anyone not to use them to think. The UK’s counselling body, the BACP, reports therapists seeing children given harmful advice, and says in the same breath that young people turn to AI because they can’t get support. The laws follow the same line. Illinois and Nevada restrict AI providing therapy. Utah regulates mental-health chatbots instead of banning them: suppliers must disclose that it’s AI and can’t sell what you tell it. Australia’s TGA regulates software by what its supplier claims it does. In every case the rules bind the supplier, not the user’s private reflection.

The feeling of not being allowed comes from two things that aren’t law. One is liability-driven product caution. The other is professional rhetoric, some of it fair and some of it plainly about protecting the profession. When a body agrees that people turn to AI because human care is unavailable or unaffordable, and its remedy includes more public money for the services its members provide, that’s a remedy that happens to benefit the people proposing it. (The Illinois regulator said outright that its law also protects providers’ jobs.) In Australia, Medicare’s Better Access subsidises up to ten individual sessions a year, usually with a gap to pay, and when the Australian Psychological Society last surveyed its members, in 2022, a third had closed their books and waits averaged about eight weeks. For many people the realistic alternative to talking it through with an AI is not a psychologist. It’s nothing.

None of that makes professionals unnecessary. It means self-education and professional care aren’t competitors.

What actually helps

The evidence is more specific than “AI is good for you”. The psychology underneath is decades old. Writing about a difficult experience helps a little. Structured self-help appears to help somewhat more. Guided self-help, where a therapist or coach gives brief support while you work through a set programme, has matched face-to-face therapy in comparative trials, mostly for anxiety and some for depression. Structure plus a little guidance is what does the work.

My argument is that AI can supply the structure and some of the prompting. That is an inference, not a finding: no trial has tested it against a human guide. The nearest evidence is a field study of more than 15,000 people on a US mental-health site, in which an AI walked users through identifying a negative thought, naming the thinking trap and reframing it. Of those asked, two-thirds reported the emotion eased straight afterwards. There was no comparison group without the AI and no follow-up, but it’s the closest test yet of the thought-diary use. The largest meta-analysis of purpose-built chatbots finds real but small short-term effects on depression and anxiety, concentrated in people who already have symptoms. Where AI looks most useful is explaining something you’ve already been told: in a 2026 online experiment, people given an AI-simplified MRI report understood it slightly better than those given the original.

Two techniques from emotion research make it work better. One is distance: describe the event in the third person, using your own name, or as an outside observer would. On average that takes some of the heat out. The other is reappraisal: deliberately look for other explanations. Both translate directly into prompts.

What goes wrong: it agrees with you

The biggest risk isn’t bad information. It’s agreeable information.

Language models are trained partly on what people prefer, and people prefer being agreed with. A 2026 study in Science tested eleven models on people’s requests for personal advice. The AI endorsed what the user had done about 50% more often than people did. On Reddit posts where the crowd had judged the writer to be in the wrong, the models still often sided with the writer. When participants discussed a conflict from their own lives, the agreeable responses made them more certain they were right and less willing to repair the relationship. They also rated those responses as better.

That’s the trap in “why did that hit me so hard?”. You bring a one-sided account, and the default failure isn’t bad advice. It’s warm validation that feels like insight. You leave the conversation more convinced and less likely to fix anything, and it felt great.

Two other risks are worth knowing, in proportion:

  • Working out what’s wrong with you is different from understanding what you’ve been told. In an Oxford trial, people using AI to work through medical scenarios identified the relevant conditions less often than people using their usual methods. The problem was the conversation: people left things out, the answers mixed good suggestions with bad, and people couldn’t tell which was which. Use AI to understand a diagnosis and to write questions for your GP, not to diagnose yourself.
  • More daily use tracks with more loneliness and dependence. The link is correlational and the effects are small, but the pattern is consistent. If the AI is replacing people rather than preparing you for them, that’s the signal.

Crisis handling has improved, by the vendors’ own tests, but it is still less reliable in long conversations and on questions that sit between clearly safe and clearly dangerous. If things are dark, talk to a person: 000 in an emergency, or Lifeline on 13 11 14.

Using a personality framework on someone else

A common variation is feeding the AI your Myers-Briggs type and a guess at the other person’s, and asking how to handle them. There’s something real underneath. The MBTI’s scales track real differences, such as thinking aloud versus reflecting first. But the sixteen types don’t hold up: scores cluster around the middle of each scale instead of splitting into two camps, and a sizeable minority get a different type on retest, about four in ten within a few months on the publisher’s own figures. Coworkers are also poorer judges of personality than friends or family, especially of the inner traits that difficult interactions usually turn on.

Supply a guessed type and the AI isn’t assessing anyone. It’s elaborating your hypothesis, and given everything above, it will tend to agree with it. The most useful finding here comes from a series of 25 experiments: imagining someone else’s perspective didn’t make people more accurate about them, and if anything made them slightly less accurate. In the final experiment, asking them did.

So use the framework to generate questions, not verdicts. “What if they need time to think before committing, rather than stonewalling?” is a good question to go and test. Describe the behaviour, not the person. That’s also the privacy answer: in Australia, an opinion about an identifiable person is personal information whether or not it’s accurate. The Privacy Act binds organisations, not you at your kitchen table, but at work it’s your employer’s problem, and the principle is a good one anywhere.

Before you do this at work

One fact surprises most people. In Microsoft 365 Copilot, your prompts and the responses are stored in a hidden folder of your work mailbox. They can be searched in eDiscovery, held and exported like email, and they fall under whatever retention policies the organisation sets. Drafting a difficult email or preparing for a hard conversation in work Copilot is a sensible use, but it creates a record. Working through a diagnosis or a relationship problem there is a different matter.

Organisations should tell staff this before it matters, and give them a private alternative, usually the EAP. They should also remember what a wellbeing chatbot can’t do. Australia’s model work health and safety rules require employers to eliminate or minimise psychosocial hazards such as excessive job demands, unclear roles and conflict, and the national code says to start with how the work is designed. A chatbot doesn’t fix a toxic workload.

A version that never leaves your phone

If the sharing risk is what stops you, the technology now allows a reflection tool that runs entirely on your own device. Small language models now run inside Safari on an iPhone or Mac running iOS 26 or macOS 26 or later, and an app built that way can keep entries encrypted on the device and send nothing anywhere once the model is downloaded. The catch is that small models are worse at exactly this kind of conversation. In simulated tests, small open models largely stopped intervening once distress was tangled up with delusional thinking, while the large commercial models mostly held. A responsible local tool has to do the safety work itself. The app, not the model, runs the thought-record steps. Crisis language is detected by plain code before the model sees it, and routed to hard-coded helpline numbers. The model is never trusted to judge risk.

Four questions, four ways in

“I’ve had medical news and want to understand it.” Paste in what you were actually told, the letter or the report, and ask for it in plain words, with the original alongside. Ask what it is, what it isn’t, what usually happens next, and what the options are, with the odds given as numbers out of 100 or 1,000, not “increased risk”. Then have it help you write three questions for your next appointment. Finish by explaining it back in your own words and asking what you’ve got wrong. Don’t ask it what else it might be; that’s the doctor’s job, and it’s where AI does worst.

“I’ve had an argument and want to unpack why.” Retell it in the third person, as someone watching would. Name the thought and how strongly you felt it. Then ask for three generous readings of what the other person said or did, and the strongest case against your own view. The point is not to decide who was right. It’s to find the question you could actually ask them.

“I can’t connect with a colleague.” Describe the behaviour, not the person: “pushes back in groups, agrees one-to-one” rather than a name and a personality type. Ask for explanations you could test, then go and test one by asking. Asking beats imagining.

“I keep losing the room in meetings.” This is where an agreeable machine is most dangerous, because it will tell you your argument is excellent. Ask it to be the sceptic in the room instead: the three objections the most doubtful person will raise, what they care about that you haven’t addressed, and the weakest link in your case. Rehearse your answers. Then find out what the doubters actually think before the meeting, not during it.

The discipline is yours

The case for AI as a thinking partner is strongest where the user brings the discipline, and weakest where the tool sets the tone. In practice:

  • Retell it in the third person.
  • Use a structure: what happened, what you thought, how strongly you felt it, the evidence for and against, a more balanced view, and how strongly you feel it now.
  • Ask for the three most charitable readings of the other person, and the strongest case against yourself.
  • Turn what you learn into questions to ask the person, not conclusions about them.
  • Understand diagnoses; don’t make them.
  • Turn off training on sensitive chats, and leave names out.
  • Take it back to a human (a friend, a GP, a psychologist) when it matters.

You don’t need anyone’s permission to understand your own life better. You do need a thinking partner that will tell you when you’re wrong, and by default this one won’t.

Sources