What an LLM Actually Does
At its core, a large language model predicts the next token in a sequence of text, given everything that came before it. That simple objective, trained across enormous amounts of text with billions of parameters, turns out to produce systems capable of translation, summarization, reasoning-like behavior, and code generation — none of which were explicitly programmed in.
Tokens, Not Words
Models don’t process text word by word — they process tokens, which are often sub-word chunks. This matters practically: pricing, context limits, and even some model quirks (like struggling with letter-counting tasks) trace back to tokenization. Understanding that “strawberry” might be split into multiple tokens explains a lot of otherwise-confusing model behavior.
Context Windows
A model’s context window is the amount of text it can consider at once — both your input and its output. Larger context windows let you pass in entire documents or codebases, but longer context isn’t free: cost scales with tokens processed, and very long contexts can dilute a model’s attention to the most relevant parts.
Training vs Inference
Training is the expensive, one-time process of teaching a model from data. Inference is running the trained model to generate responses — what happens every time you send a prompt. As a developer integrating LLMs, you’re almost always working purely at the inference stage, unless you’re fine-tuning a model on your own data.
Why Models “Hallucinate”
Because a model is fundamentally predicting plausible text rather than looking up facts in a database, it can generate confident, fluent, and completely incorrect statements. This isn’t a bug to be patched away entirely — it’s a consequence of how the technology works, which is why techniques like retrieval-augmented generation and careful prompt design matter so much for production use cases.
What This Means for Building Products
Treat an LLM as a powerful but unreliable component, not an oracle. Validate outputs where correctness matters, keep humans in the loop for high-stakes decisions, and design your product to fail gracefully when the model gets something wrong — because it will, and understanding why makes that far easier to plan for.