ChatGPT works by predicting the next small piece of text — called a token — over and over again. It was trained on a very large amount of human writing, where it learned which words and phrases tend to follow which others, and it was then fine-tuned with human feedback so its predictions come out as helpful answers instead of random continuations. When you send a message, it does not look an answer up: it scores every plausible next token, picks one, adds it to the text, and repeats the whole process until the reply is finished. That single loop explains almost everything about it — why it sounds fluent, why it can answer questions it was never explicitly taught, and why it can state something false in a perfectly confident tone.
The one idea everything else hangs on: prediction, not lookup
A search engine stores pages and retrieves them. ChatGPT stores no answers at all. OpenAI’s own explanation is plain about this: during training the model “analyzes relationships within this data — such as how words typically appear together in context — and uses that understanding to predict the next most likely word when generating a response, one word at a time” (OpenAI Help Center).
What the model actually keeps from training is a very large set of numbers, called weights or parameters. As it trains, those numbers are adjusted slightly, again and again, until the predictions get good. OpenAI stresses that the model “does not store or retain copies of the data they are trained on” — the training text is gone; only the adjusted numbers remain, like a teacher who can explain a subject from understanding rather than by reading the textbook aloud. ChatGPT is one product built on this kind of system, which is called a large language model — the chat window is the interface, and the model underneath does the predicting.
A worked example: one sentence, built token by token
Most explanations say “it predicts the next word” and stop there. Here is the loop actually running, on the smallest prompt that shows it clearly. The percentages below are illustrative — invented for teaching, in the shape of what the model really computes — because OpenAI does not publish the live probabilities inside ChatGPT. The arithmetic, however, is exactly the arithmetic the model does: at every step, the scores for all possible next tokens add up to 100%.
Step 1 — your text is chopped into tokens. Suppose the conversation so far is just:
The cat sat on the ___
The model never sees letters or whole sentences as units. It sees tokens — pieces of text that, in OpenAI’s words, “may represent whole words, parts of words, or punctuation.” This prompt becomes roughly five tokens: The · cat · sat · on · the. Long or unusual words are split into several tokens; short common words are usually one. Everything the model knows about your message, it knows through this token sequence plus the conversation before it.
Step 2 — every possible next token gets a score. Given those five tokens, the model produces a probability for each candidate continuation. The top of that list might look like this (illustrative):
| Next-token candidate | Probability |
|---|---|
| mat | 42% |
| floor | 24% |
| sofa | 15% |
| roof | 6% |
| grass | 4% |
| All other tokens combined | 9% |
| Total | 100% |
Notice what is not in this table: any check of what is true. There is no fact column. “Mat” leads because, in the text the model trained on, “the cat sat on the mat” is an extremely common pattern — not because the model has seen your cat or knows where it is sitting.
Step 3 — one token is picked, and the loop restarts. Say “mat” is picked. It is glued onto the sequence — the input is now The · cat · sat · on · the · mat — and the model scores the next position from scratch. A plausible second table (illustrative again):
| Next-token candidate | Probability |
|---|---|
| . (full stop) | 55% |
| and | 18% |
| purring | 9% |
| All other tokens combined | 18% |
| Total | 100% |
Pick the full stop, and the sentence is complete: “The cat sat on the mat.” A real answer is just this loop running hundreds of times — score, pick, append, re-score — fast enough that the reply appears to stream onto your screen word by word. Every essay, poem, and block of code ChatGPT has ever produced was assembled exactly this way, one token at a time, with no plan for the finished text written down anywhere in advance.
Where the patterns came from: training
The probabilities in those tables are not programmed by hand. They are the residue of training, which OpenAI describes as happening in stages: preparing training data, pre-training, and post-training, with evaluation continuing after deployment.
Pre-training is the long first stage. OpenAI says its foundation models are developed from three sources of information: (1) information publicly available on the internet, (2) information accessed through partnerships with third parties, and (3) information that users, human trainers, and researchers provide or generate. For the public internet portion, OpenAI says it uses only freely and openly accessible material, does not intentionally gather from paywalled sources or the dark web, and applies filters to remove material such as hate speech, adult content, sites that aggregate personal information, and spam.
The training task itself is the same guessing game as the worked example, played at enormous scale. OpenAI’s own illustration: given “Instead of turning left, she turned ___,” an untrained model guesses almost at random; after processing a large volume of text and being corrected across millions of sentences, it reliably puts “right,” “around,” or “back” at the top of the list. Each correction nudges the model’s weights a little. Played often enough, those nudges add up to a system that has absorbed grammar, facts as they are commonly stated, writing styles, and the shapes of arguments and code — all as patterns of what tends to follow what. This is the broader family of techniques that generative AI is built on, and it sits inside machine learning rather than beside it — the distinction is drawn out in AI vs machine learning.
Post-training (fine-tuning and human feedback) is what turns a raw text-predictor into an assistant. Left alone, a pre-trained model just continues text: give it a question and it might continue with more questions, because that is what questions are often followed by in books and forums. So the model is further trained to behave like a helper. A core technique is reinforcement learning from human feedback (RLHF): human trainers first demonstrate good responses, then rank several of the model’s draft answers from best to worst, and those rankings train a scoring system that the model is then optimized to please. The result is a predictor that has been steered — toward answering the question asked, in a sensible order, in a tone people rated as helpful, and away from outputs people rated as harmful or useless. The predicting never stops; it is simply predicting text that now tends to look like a good assistant’s reply.
The transformer and “attention,” without the maths
One honest obstacle remains: how does the model decide that “mat” fits better than “grass” after “The cat sat on the”? The scoring is done by a type of neural network called a transformer — the T in GPT, which stands for Generative Pre-trained Transformer. The transformer’s key mechanism is called attention, and the plain-English version is this: before scoring the next token, the model weighs how much every earlier token should influence the decision, all at once.
A small example shows why that matters. In “The animal didn’t cross the street because it was too tired,” the word “it” points to “animal,” not “street.” In “…because it was too wide,” the same “it” points to “street.” The words are identical; only the surrounding context flips the reference. Attention is the machinery that lets the model route influence from the right earlier words — “tired” pulling toward the animal, “wide” toward the street — when it interprets “it” and when it predicts what comes next. The University of Central Florida’s explainer of how generative AI tools work describes the same generation process from the academic side: tokenization, a probability distribution over the next token, and an iterative loop that appends each chosen token and calculates again. Because the model considers the whole visible conversation at every step — your messages and its own replies so far, up to a size limit called the context window — it can answer follow-ups like “when was he born?” correctly, as long as “he” was established earlier in that same conversation.
Why the same prompt can get a different answer twice
If the model always picked the single highest-scoring token, the same prompt would produce the same reply every time. It doesn’t, and OpenAI explains why directly: because there are multiple plausible ways to complete a text, “there is an inherent element of randomness in how the model responds. As a result, the same question may yield different answers across different queries.”
In the worked example, “mat” led at 42% — but “floor” at 24% and “sofa” at 15% were live possibilities, and a picker that sometimes samples from the leaders rather than always taking the top one will occasionally sit the cat on the floor instead. One different early token changes everything downstream, because every later score is computed from the text actually produced. This is also why regenerating a disappointing answer is a rational thing to do: you are not asking the same machine to repeat itself, you are rolling the same weighted dice again. (Developers using the underlying models directly can turn this randomness up or down with a setting called temperature; the chat product makes that choice for you.)
Why it can sound completely sure and be completely wrong
This is the limit that follows directly from the mechanism, and it is the one worth remembering. The model’s job, at every step, is to produce the most plausible next token — and plausible and true are different properties. Nothing in the loop described above contains a step that checks a claim against the world. Facts are not stored as records the model can consult; they survive only as patterns in the weights, which is why a rarely mentioned date, name, or citation can come out smoothly assembled and entirely invented. A confident tone is itself just another learned pattern: text written by sure-sounding people was in the training data, so sure-sounding text is what the predictor produces, whether or not the content underneath it holds up. When a language model generates a fluent statement that is false, that failure has a name — AI hallucination — and it is not a glitch in the prediction process. It is the prediction process, working exactly as built, on a question where the most likely-sounding continuation happens not to be the true one.
The practical habits follow from this: treat ChatGPT’s answers as a strong first draft rather than a verified source, check names, numbers, quotes, and citations independently before relying on them, and be most careful exactly where the answer sounds smoothest — fluency is the one thing the system is guaranteed to produce.
Its real limits, in one place
- No lookup by default. The base mechanism generates from learned patterns; it is not searching a database of facts as it writes. (Some ChatGPT setups can call tools such as web search — OpenAI’s current product documentation, including its guide to ChatGPT Work, describes tool-using modes — but tool results are then fed back in as more text for the same prediction loop.)
- A fixed training past. What the model learned comes from its training data; events after that training are not in the patterns unless a tool supplies them.
- A finite window. It can only weigh the conversation that fits in its context window; scroll far enough back and early details fall out of what it can attend to.
- No understanding to appeal to. It models relationships between pieces of text. It does not know when it is wrong, and asking it whether it is sure just runs the prediction loop again — the “yes, I’m sure” it returns is more predicted text, not an inspection of its own answer.
- It does not keep learning from your chat. Your conversation steers the replies inside that conversation, because it is part of the input the model reads. It does not rewrite the model’s weights as you talk.
Frequently asked questions
Does ChatGPT think or understand? Not in the human sense. It computes which token is most likely to come next, given the text so far and the patterns stored in its weights from training. That process can produce answers that look like reasoning — reasoning-shaped text was in the training data — but there is no separate step where the model checks its beliefs or consults what it knows.
Is ChatGPT just copying from websites it read? No. OpenAI states that the models do not retain copies of their training data and do not “copy and paste” from it; training adjusts the values of the model’s parameters to reflect patterns in the data. What comes out is newly generated text, assembled token by token at the moment you ask.
Why does ChatGPT sometimes give different answers to the same question? Because generation involves randomness by design. Where several continuations are plausible, the pick among them is not fixed, and one early difference cascades into a different answer. OpenAI gives exactly this explanation in its own documentation.
What is a token, in one sentence? A token is the small piece of text — a whole word, part of a word, or a punctuation mark — that the model actually reads and produces, one at a time.
How is ChatGPT different from Google search? Search retrieves existing pages that match your query and shows them to you. ChatGPT generates a new response, token by token, from patterns learned in training. Search results can be opened and checked at the source; a generated answer has no single source — which is why verifying it matters more, not less.
This article is general information about how ChatGPT works, based on OpenAI’s published documentation and the university source linked above, checked in November 2026. ChatGPT’s models, features, and tools change frequently; the prediction mechanism described here is the stable part, and OpenAI’s help pages linked above carry the current product details.