What is a large language model, really?
Time: 9:30 AM to 10:05 AM
Tokens: how an LLM reads text
An LLM does not read words the way you do. It splits text into tokens — fragments that can be whole words, parts of words, or single characters. Every piece of text goes through this process before the model touches it. Here is how the sentence “The robot drives forward” gets tokenized:
That was a simple sentence. Longer or unusual words get split into pieces:
Why does this matter? Because everything the model does is measured in tokens — how much text it can read, how much it can generate, and how much it costs. A typical English word is about 1.3 tokens.
How training works
A model learns by reading billions of tokens and adjusting billions of numbers (called weights) to get better at one task: predicting the next token. That is the entire training loop. No one programs the answers. No one writes rules. The model discovers patterns on its own by seeing enormous amounts of text. This loop runs trillions of times across weeks of training on thousands of GPUs. The result is a model that has learned grammar, facts, reasoning patterns, code syntax, and even humor — all from predicting the next token.GPT-4 has roughly 1.8 trillion parameters (weights). Training it cost over $100 million in compute. You are about to use it for free from a Raspberry Pi.
How inference works
Inference is what happens when you actually use the model. You send a message, and the model generates a response one token at a time. It does not “know” the answer. It does not look anything up. It generates the most likely next token, appends it, and repeats — like an extremely sophisticated autocomplete. Each token takes a fraction of a second. The model generates dozens of tokens per second, which is why responses feel nearly instant.Temperature: controlling randomness
When the model picks the next token, it does not always pick the single most likely one. The temperature setting controls how much randomness is allowed.System prompts: shaping the model’s personality
A system prompt is a hidden instruction prepended before your message. It defines the model’s role, personality, and constraints. The user never sees it, but it shapes every response. Here is the same question — “What is the speed of light?” — answered with three different system prompts:
The system prompt is the single most powerful control you have over the model’s behavior. Later today, you will write system prompts for your robot.
Context window: the model’s working memory
The context window is the maximum amount of text the model can consider at once — your messages, its responses, and the system prompt all count. Think of it like a whiteboard that can only hold a fixed number of sticky notes. When the whiteboard fills up, the oldest notes fall off. The model cannot remember anything outside its context window.
Your robot conversations will use a few hundred tokens at most, so you will not hit this limit. But it explains why the model “forgets” things from earlier in very long conversations.