What a language model is actually doing

Generating one token at a time

Predictive text is a useful starting analogy: the system uses context to estimate what could come next. A language model learns richer patterns during training, but a plausible continuation is not necessarily a true statement.

That is a large language model. It looks at everything written so far and produces a ranked list of what could plausibly come next. Then it picks one, sticks it on the end, and does the whole thing again.

Step through it. Two things should feel strange, and both are true:

1. It picks one word at a time. When you asked for a poem, there was no moment where it "had the poem" and then typed it out. The poem happened as it was typed.
2. It is not choosing the top answer every time. There is a dice roll. That is why you get a different answer when you ask twice, and it is deliberate - always taking the top word produces flat, repetitive, weirdly robotic text.

Turn on X-ray in that scene. temperature is how loaded the dice are. Low temperature = boring and predictable. High = creative and unhinged. There is no "accurate" setting. Accuracy is not what the knob controls.

Tokens: the model does not see words, and definitely not letters

The text gets chopped into chunks called tokens before the model sees anything. Roughly:

"unbelievable"     →  ["un", "bel", "iev", "able"]      4 tokens
"cat"              →  ["cat"]                           1 token
"strawberry"       →  ["str", "aw", "berry"]            3 tokens
100 words of English ≈ 130 tokens

This is not trivia. It explains a famous embarrassment: ask a model how many r's are in "strawberry" and it may well say two. It never saw the letters. It saw three chunks. Asking it to count letters is like asking you to count the brush strokes in a word you heard out loud.

Rule of thumb you can actually use: if a task depends on individual characters, exact arithmetic, or counting, do not ask the model - make it use a tool. Level 3 shows you how that works.

Where the "knowledge" comes from, and why it is blurry

During training the model read a very large slice of the written internet, books, code and documentation, and repeatedly played a fill-in-the-blank game with itself. Getting the blank right nudged its internal numbers. Billions of nudges later, the numbers encode patterns:

• grammar and style (very strongly)
• widely repeated facts, like capitals of countries (strongly)
• things mentioned twice in an obscure forum in 2019 (faintly, unreliably)
• what happened last Tuesday (not at all - training stopped at some date)

Nothing is stored as a lookup table. There is no row in a database saying "Paris is the capital of France". There is a pattern that makes "Paris" overwhelmingly likely in that sentence. That is why the model can be confidently wrong: a wrong answer and a right answer feel identical from the inside. It is producing the most plausible-sounding continuation either way.

This is the definition of a hallucination, and it is not a bug that will be patched. It is the same mechanism that makes the model useful, running on a question where plausible and true came apart. Grounding it in real data (Level 3) is the fix, not scolding it.

No memory. Seriously, none.

This is where nearly every consumer misunderstanding lives. The model does not remember your last message. It is handed the entire conversation again, from scratch, every single time you press enter. The illusion of memory is a piece of very ordinary software re-sending the transcript.

Watch that fill up. The context window is the size of the desk. Everything the model can consider must be physically on the desk right now - your instructions, the document you pasted, the whole chat so far. When the desk overflows, the oldest paper goes in the bin.

Three consumer-grade consequences:

• "It forgot what I told it" almost always means: it fell off the desk. Restate it.
• Long chats get worse, not better. More paper, more distraction, more money per message.
• Starting a fresh chat is a real technique, not giving up. A clean desk with a good summary beats a cluttered one every time.

What it is good at, honestly

Read the left column again. Almost everything there is transforming text you already gave it. That is the sweet spot, and it is the reason the next lesson exists: if you can feed it the right facts and let it act, the risky column shrinks dramatically.

Where things stand (as of 2026)

Fluent, confident text generation is now completely routine - nobody is impressed by grammatical, well-structured output anymore. What is still unreliable: factual precision on anything obscure, exact counting, and knowing the actual current date without being told. Context windows have gotten much larger than they used to be, but "larger window" has not solved uneven attention or context rot - a bigger desk is still a desk, and things still get lost in the middle of it. Hallucination has not been "solved" and, based on how the mechanism works, is unlikely to ever fully disappear - grounding with real documents remains the practical fix. Exact numbers for context size, pricing, and capability change every few months across providers, so treat any specific figure you read as needing a date check.

How to read the docs and look things up

The primary sources that matter here are each provider's own model card (a short technical description of what a specific model is and isn't good at) and their API documentation, not third-party "best AI in 2026" roundup articles. A model card usually states training cutoff date, context window size, and known limitations plainly - that's the tell for a real spec versus marketing copy, which instead uses words like "revolutionary" and shows no dates or numbers. To test a claim yourself: pick something the marketing page brags about (like "great at counting" or "up to date knowledge") and try the strawberry test, or ask it a question about very recent news. Ten minutes tells you more than the page did. Keep a dated note of what you tested and what happened, since model behaviour genuinely changes between versions.

Check yourself

• Why can a model write a beautiful paragraph about a book that does not exist?
• What is a token, and why does it break letter-counting?
• Your assistant "forgot" an instruction from twenty messages ago. What actually happened?

Common mistake: arguing with it. Telling a model "no, that's wrong, think harder" sometimes works by accident, because your correction becomes new text on the desk. Giving it the right document works on purpose, every time.

Recap: what changed in your head

• You know a model is predicting the next token, one at a time, not "thinking" in one shot.
• You know why it can't count letters reliably: it never sees letters, only tokens.
• You know hallucination isn't a bug to scold - it's the same mechanism that makes it useful.
• You know it has no memory between messages; the whole transcript is re-sent every time.
• You know which tasks are its sweet spot (transforming text you gave it) versus its risky zone.