How large language models work, explained without the jargon

The AI systems behind chat assistants are large language models, and at their core they do something surprisingly simple to describe: given some text, they predict what comes next.

Tokens, not words

A model reads and writes in “tokens” — chunks of text that are often a word or part of a word. Given a sequence of tokens, the model estimates the probability of every possible next token and picks one, then repeats. Stringing those predictions together is how it produces fluent sentences.

Training versus inference

There are two distinct phases. Training happens once and is expensive: the model is shown vast amounts of text and its internal settings — called parameters, often numbering in the billions — are adjusted so its next-token predictions match real text more closely. This is machine learning at scale.

Inference is what happens when you use the model. The parameters are now fixed; the model simply runs your input through them to generate a response. Training builds the system; inference uses it.

Why they still make mistakes

Because a model generates text by predicting plausible continuations, it can produce statements that sound confident but are wrong — often called “hallucinations.” It has no built-in database of facts to check against; it has patterns learned from data. Techniques like retrieving real documents at inference time, or having the model show its reasoning, reduce errors but do not eliminate them.

Why it matters

Understanding that these systems are pattern-based predictors, not knowledge engines, is the key to using them well. They are powerful tools for drafting, summarizing, and transforming text, and they work best when a human verifies anything that has to be correct. The technology is advancing quickly, but that basic shape — predict the next token, at scale — has stayed the same.