AI Can Write a Therapy Note, But Can It Think Like a Therapist?

As a therapist, I have increasingly mixed feelings about artificial intelligence.

On one hand, AI can generate treatment plans, summarize sessions, draft worksheets, provide psychoeducation, and even sound remarkably empathic. It can be an incredible tool. On the other hand, a recent study reminded me why I'm not worried about AI replacing therapists anytime soon.

Researchers recently gave several leading AI models a classic psychological assessment called the Stroop Test, a task psychologists have used for decades to measure attention, executive functioning, and inhibitory control.

The task sounds simple.

Participants are shown words representing colors, but the ink color doesn't always match the word. For example, the word RED might be printed in blue ink. The goal isn't to read the word. It's to name the color of the ink.

Humans naturally want to read the word, so completing the task requires us to suppress our automatic response and stay focused on the goal.

What surprised researchers wasn't that AI struggled a little. It was how dramatically performance declined as the task became longer and more cognitively demanding.

GPT-4o reportedly scored around 91% accuracy on short versions of the test but dropped to approximately 15% accuracy on the longest versions. Other leading AI models showed similar declines. The researchers concluded that while AI could often identify the correct strategy, it struggled to maintain that strategy when cognitive demands increased.

As I read the study, I found myself thinking less about artificial intelligence and more about psychotherapy.

Because good therapy isn't primarily about intelligence. It's about attention.

When I sit with a client, I'm not simply listening to their words. I'm tracking emotional themes, attachment patterns, trauma responses, inconsistencies, strengths, defenses, treatment goals, body language, and what we've discussed over weeks, or sometimes years.

A client may be talking about an argument with their spouse, but I'm also asking myself:

• Is this really about the dishes, or is it about feeling unseen?
• Is this anger, or is there fear underneath it?
• Does this reaction fit the current situation, or is an old wound being activated?
• What changed since our last session?
• What isn't being said?

Therapy requires us to hold multiple possibilities in mind while deciding which one deserves our attention.

That takes executive functioning.

It requires knowing when to stay with an emotion, when to challenge a belief, when to remain silent, and when to gently redirect the conversation. It requires resisting the urge to jump to conclusions simply because one explanation seems obvious.

That's very different from generating the next likely sentence.

This study also highlights something important for people who use AI for mental health support.

I believe AI has tremendous value.

It can explain psychological concepts, generate journaling prompts, teach coping skills, summarize research, help organize thoughts, and even provide a starting point for self-reflection. I use AI myself as a professional tool.

But a tool isn't the same thing as a therapist.

A therapist doesn't just provide information.

A therapist decides which information matters most in that particular moment.

Sometimes a client says, "I'm fine," while their voice cracks.

Sometimes a couple spends twenty minutes arguing about chores when the real conversation is about loneliness.

Sometimes the smallest detail in an entire session turns out to be the key that unlocks everything else.

Those moments aren't found by simply recognizing patterns.

They're found through sustained attention, clinical judgment, and the ability to tolerate uncertainty.

What struck me most about this research is that AI's greatest limitation may not be a lack of empathy.

It may be a lack of executive functioning.

Therapists constantly filter distractions, prioritize what's clinically significant, inhibit less helpful directions, and stay oriented toward the client's goals, even when sessions become emotionally messy or cognitively complex.

That's something humans do remarkably well.

As AI continues to improve, I think it will become an even more valuable companion to mental health care. It will likely continue to enhance education, accessibility, and support.

But therapy has never been about producing the right answer the fastest.

It's about knowing which question needs to be asked next.

And sometimes, those are two very different things.

Natalie 🦋

Source: The conversation in this blog was inspired by the article "AI models utterly fail classic psychology test as cognitive demands increase" and the underlying research examining AI performance on the Stroop Test.