How ChatGPT Understands Your Questions?

Most explanations of LLMs stay vague — "it predicts the next word" — and stop there. This post goes deeper into each stage of the pipeline: what actually happens to your text from the moment you hit send to the moment a response appears on screen.

1. What Is an LLM?

A Large Language Model is a neural network trained on enormous amounts of text — books, websites, code, articles — to predict the statistical likelihood of what text should come next, given some input. The "large" refers to two things at once: the size of the training data (often trillions of tokens) and the size of the model itself (billions to hundreds of billions of parameters — the adjustable internal values the model learns during training).

It's not a database and it's not a search engine. It has no lookup table of facts. Everything it "knows" is encoded as weighted connections between artificial neurons, learned by repeatedly adjusting those weights to get better at predicting text during training.

What LLMs are used for

• Question answering

• Long-form writing and editing

• Translation

• Code generation and debugging

• Summarization and information extraction

• Reasoning through multi-step problems

Major models in production today

LLM	Company
ChatGPT (GPT series)	OpenAI
Gemini	Google
Claude	Anthropic
Llama	Meta

2. From Prompt to Response: The Full Pipeline

When you send a message, it passes through several distinct stages before you see a single word of output:

Your prompt (raw text)
    │
    ▼
Tokenization (text → tokens)
    │
    ▼
Embedding (tokens → vectors of numbers)
    │
    ▼
Transformer layers (context + attention processing)
    │
    ▼
Probability distribution over the vocabulary
    │
    ▼
Sampling (pick the next token)
    │
    ▼
Repeat until the response is complete

Each of these steps deserves a closer look.

3. Tokenization: Breaking Text Into Units the Model Can Use

Computers don't process "words" — they process discrete units called tokens, produced by a tokenizer (most modern models use a method called Byte Pair Encoding, or BPE, or a variant of it).

A token might be:

• A whole common word ("cat", "the", "love")

• A sub-word piece of a longer or rarer word ("un", "believ", "able" from "unbelievable")

• A punctuation mark

• Even a single character, for unusual strings

Why split words into pieces instead of using whole words as the unit? Two reasons:

1. Vocabulary size control. If every possible word were its own token, the vocabulary would be unmanageably large and full of rare entries the model rarely sees. Sub-word tokenization keeps the vocabulary to a fixed, manageable size (commonly 30,000–100,000+ tokens) while still being able to represent any word, even ones never seen during training, by combining smaller pieces.

2. Generalization across languages and rare words. Sub-word units let the model handle misspellings, made-up words, and multiple languages without needing a separate entry for every variant.

After tokenization, "I love ice cream." might become five or six tokens depending on the specific tokenizer: I, love, ice, cream, ..

4. Embeddings: Turning Tokens Into Numbers

Once text is tokenized, each token is mapped to a vector — a long list of numbers, typically hundreds to thousands of dimensions — through an embedding layer. This vector is the model's numerical representation of that token's meaning, learned during training.

Critically, embeddings capture relationships: tokens with similar meanings or usage patterns end up with vectors that are mathematically close to each other in this high-dimensional space. This is what lets the model recognize that "king" and "queen" are related in similar ways to "man" and "woman," purely from the geometry of the numbers, without anyone explicitly programming that relationship.

The model also adds positional information to each token's vector, because raw embeddings alone don't encode word order — and word order obviously matters ("dog bites man" vs. "man bites dog").

5. The Transformer: Context Through Attention

This is the architecture, introduced in the 2017 paper "Attention Is All You Need," that made modern LLMs possible.

The core mechanism is called self-attention. For every token in the input, the model calculates how relevant every other token is to understanding that one — and weighs them accordingly.

Take the sentence: "The dog chased the ball because it was red."

To resolve what "it" refers to, the model computes attention scores between "it" and every preceding word. Through training, it has learned patterns that make "ball" the stronger candidate than "dog" in this context (since dogs aren't typically described as red), and it weighs the ball's vector more heavily when building its understanding of "it."

Self-attention is computed in parallel across many "heads," each potentially focusing on different types of relationships (grammatical structure, factual association, long-range dependencies, etc.). These are called multi-head attention layers, and a model stacks dozens of them on top of each other, each layer refining the representation further.

This is also why Transformers handle long, complex context so much better than older architectures (like RNNs, which processed text strictly word-by-word and struggled to "remember" earlier parts of a long passage): every token can directly attend to every other token, regardless of distance, in a single computation step.

6. From Representation to Prediction

After passing through all the Transformer layers, the model has a rich numerical representation of the input. The final layer converts this into a probability distribution across the entire vocabulary — essentially a score for every possible next token, indicating how likely each one is given everything that came before.

The model then samples from this distribution to choose the next token. This isn't always the single highest-probability token; depending on settings (often called temperature, top-p, or top-k sampling), there's controlled randomness involved, which is part of why the same prompt can produce slightly different responses each time.

7. Autoregressive Generation: One Token at a Time

Once a token is chosen, it gets appended to the input, and the entire process — tokenize, embed, attend, predict — runs again to generate the next token. This is called autoregressive generation: each new token depends on everything generated before it.

This is also why responses appear to "stream" in token by token rather than all at once — that's literally how they're being computed, repeatedly, until the model produces a special end-of-sequence token or hits a length limit.

8. Why the Model Isn't "Looking Things Up"

A common misconception is that the model searches the internet or a database for each answer. It doesn't (unless explicitly given a separate search tool). Everything it generates comes from patterns encoded in its weights during training — compressed statistical regularities about how language and concepts relate to each other, not stored facts in retrievable form. This is also why models can be confidently wrong (a phenomenon often called hallucination): they're generating the statistically plausible continuation of the text, not retrieving a verified fact.

9. Putting the Whole Pipeline Together

1. You write a prompt.

2. The tokenizer breaks it into tokens.

3. Each token is converted into a numerical embedding, plus positional information.

4. The embeddings pass through stacked Transformer layers, where self-attention builds contextual understanding across the entire input.

5. The final layer produces a probability distribution over the vocabulary for the next token.

6. A token is sampled from that distribution.

7. The new token is appended to the sequence, and the cycle repeats.

8. Generation stops at an end token or a length limit, producing the final response.

Key Takeaways

• An LLM is a neural network that predicts the next token based on patterns learned from massive amounts of training text.

• Text is broken into tokens via sub-word tokenization, not whole words.

• Tokens become numerical embeddings that capture meaning and relationships.

• Transformers use self-attention to let every token consider every other token's relevance, which is what gives LLMs strong contextual understanding.

• Generation is autoregressive: one token is predicted at a time, each depending on everything before it.

• The model isn't retrieving facts from a database — it's generating statistically likely continuations based on learned patterns, which is both its strength and the source of its failure modes like hallucination.