Your Model is Wrong

26 Feb, 2026

Have you ever seen a movie where someone time travels and has to ask what year it is? They always do it carefully, or awkwardly, as if it were a stupid question or somehow suspicious. It is apparently inconceivable that a sane human being would not know what year they are living in.

Now, how many times have you had to tell ChatGPT what year it is?

AI is confidently wrong a lot. It insists it is 2024 when it is not, misrepresents company policies, and cites sources that do not exist. While these failures are obviously annoying, I don't think they necessarily warrant quite the level of frustration that people often express with failing AI systems. We are used to search results not yielding what we're looking for and apps not working as expected. We complain, perhaps curse a little, and move on. When ChatGPT misunderstands us, the reaction can feel almost personal.

Part of the problem is that the AI's model of the world is wrong. The other part is that our model of the AI is wrong.

Personally, I think this has less to do with the fact that the AI is getting things wrong and more to do with Joseph Weizenbaum’s 1966 chatbot, ELIZA.

The ELIZA Effect

ELIZA was the first NLP chatbot application, developed by Joseph Weizenbaum at MIT from 1964 to 1966. When he showed ELIZA to his secretary, she reportedly asked him to leave so she could have a "real conversation" with it. Weizenbaum was so struck by this kind of reaction that he later wrote: "Extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people."¹

ELIZA was but a simple pattern-matching program with certainly far less “understanding” of what it was saying compared to modern AI systems, and yet it was enough to make someone treat it as something that could understand them, and was worth having a private conversation with. Now consider what happens when the system on the other end isn't matching patterns from a script, but generating fluid, context-aware prose that adapts to your tone.

When we use a modern LLM chatbot, it appears to speak like a human. It echoes our tone, mirrors our mannerisms, and responds in kind. Without realising it, we begin to apply Theory of Mind (ToM)—our ability to model what others believe, know, and intend²—modelling how this system will interpret our words just as we would for another person. And then the AI confidently tells us something wrong, or ignores what we thought was a clear instruction, or does something we never asked for. The result is the same frustration we feel when a person misunderstands us.

Human Communication

Human communication is much richer than it appears at face value. We lean on tone, gesture, expression, and shared physical context to fill in what words leave out. But even more than all of this, our ability to communicate effectively relies on understanding not at the linguistic level, but at the experiential level.

We use ToM constantly without ever knowing it. We predict how our words will land and adjust them for our audience. When I say “the meeting was a disaster,” I trust you’ll infer frustration rather than literal catastrophe. ToM allows us to deliver complex ideas and concepts to others through simple words, because we understand how they will understand those words. Take this away, and we would be left with just the words themselves, without the context. What does “blue” mean to a mosquito anyway?

From one perspective, language can be seen as a compression algorithm, and an extremely lossy one at that. Take the example of the word “blue” above. What does that mean? What is “blue” to you? What is it to me? What about your mum? Are they the same? Where’s the line between “blue” and “not blue” on the colour wheel, for you, or me, or your mum?

When we use the word “blue” we’re compressing a rich set of sensory experiences into four letters on a page. Those letters don’t mean anything at all without the reader’s ability to decode them through the lens of their lived experience. If you go swimming in the ocean every day, your “blue” is going to be different from the “blue” of someone living next to an alpine lake, or the staff member at Bunnings selling you paint from the Dulux colour chart. The word “blue” doesn’t itself contain the colour; it's merely a trigger for the reader’s pre-installed “blue” decoder.

Our communication with others therefore succeeds in proportion to the similarity of our decoders—decoders derived from our lifetimes of lived experience. Humans all share a large part of this experiential decoder: our physiology. We have the same sense organs, pain receptors and body morphology; we're all affected by gravity, and heat, and the progress of time. These things give us a huge foundation of shared experience, and on top of that we layer the cultural context of our daily lives: our shared history, food, slang, and upbringings. ToM is how we reason about all of this. It's how we predict the decoder someone else will use to interpret our words, and how we adjust the way we encode our intent into language.

Calibrating Theory of Mind

ToM isn’t a fixed capacity; it’s calibrated through experience and we learn it like any other skill: through practice. Research has found that people’s ability to predict others’ mental states is greatly impaired across cultures compared to within their own culture.³⁴⁵ I bet you have a much better idea of how your best friend thinks than that coworker you speak to occasionally in the lunch room. When you move to a new culture, or begin to interact with a new group or individual, you have to recalibrate your ToM.

Just as we build ToM for biological forms of intelligence, we must build it for the artificial variety. However, this poses an immediate problem: AI doesn’t share any of our biological decoder—it doesn’t even feel the progress of time—and its cultural context is so diluted that it gives us little to reason on. Fundamentally, AI doesn’t produce intelligent behaviour in the same way as the biological entities we are used to interacting with. Its knowledge and behaviour is grounded in linguistic patterns, rather than sensory experience.

This symbol grounding problem, as it is often referred to, is a long-standing discussion in the AI literature.⁶⁷⁸ A human’s understanding of “blue” may differ between individuals, but it is always grounded in experience: the blue of the ocean, the alpine lake or the Dulux colour chart. For LLMs, on the other hand, “blue” is grounded in relationships between symbols: symbols point to other symbols rather than to the world. This gap between linguistic competence and experiential understanding is invisible most of the time, which is precisely what makes it so disorienting when it surfaces.

Even when we consciously understand what we're talking to, our subconscious assumptions about the intelligence we're interacting with are often wrong. This is why confident wrongness can feel less like a tool malfunction than a social betrayal, and why correcting the mismatch means recalibrating our model of the AI.

Fixing Your Model

When we move overseas, we adjust our ToM to work for the new people in our new situation, and I believe this is precisely how people learn to interact with AI as well. Through trial and error and exposure, people who are highly effective at using AI have built ToM for this new form of intelligence. They have a clear model of how their words will be interpreted and acted upon. They know not to assume the AI will remember their past conversations, and have learned to provide all the context upfront every time, without even thinking about it. They know that just because the AI expresses something confidently, doesn’t mean it is right, and ask it to “double check that with a web search” when it's making factual claims.

At the beginning, remembering to do these things takes conscious effort, just as it does for me to remember to ask for directions to a “gas station” and not a “servo” when I visit the USA. Over time, however, our brains naturally adapt. Given enough exposure, the subconscious model updates, and communication gets easier. The skill is in being patient with yourself, and the AI, while it happens.

So, whether you’re new to AI, or just trying the latest model, expect your intuitions to be wrong, pay attention to where your predictions fail, and expose yourself to lots of different situations. Play around. Try weird things. Build your ToM.

References

#ai