How AI models work, without the maths
Understand what's really happening when an AI answers you, explained without any maths or code.
An AI assistant works a bit like an incredibly well-practised "guess the next word" player.
Step 1: Reading. Before you ever use it, the AI is shown a gigantic amount of writing: books, websites, articles and more. This is called .
Step 2: Practising. Over and over, it tries to guess the next word in a sentence, then checks if it was right and adjusts itself. After billions of goes, it gets extremely good at it, so good that it picks up grammar, facts, and how to explain and reason.
Step 3: Learning manners. People then teach it to be helpful, polite and safe, by showing examples of good answers and rating its replies.
Step 4: Answering you. When you ask something, it writes the answer one small piece at a time, each time picking what's most likely to come next, given your question and everything said so far.
That's why it's so fluent, and also why it can sometimes make things up: it's producing what sounds right, not looking it up in a book.
The big idea: prediction
A large language model (LLM) is trained to do one thing: predict what comes next in a piece of text. That sounds simple, but to predict well across every topic, it has to absorb grammar, facts, styles of writing and patterns of reasoning. Those abilities emerge from the prediction task.
Stage 1: pre-training
The model starts as a huge network of adjustable numbers (called parameters, often hundreds of billions of them) set to random values. It's shown vast amounts of text, broken into small chunks called tokens (roughly three-quarters of a word each). For each chunk it predicts the next one, sees the right answer, and adjusts its numbers slightly to do better next time. Repeat this trillions of times on enormous computers and you get a model that's remarkably good at continuing any text.
Stage 2: post-training
A raw pre-trained model just continues text; it isn't a helpful assistant. So labs train it further:
- Examples of good answers: showing it questions paired with ideal responses.
- Human and AI feedback: rating its answers so it learns which are more helpful, honest and safe.
- Practice on problems with checkable answers (like maths and coding), rewarding correct solutions. This has been key to recent "reasoning" improvements.
Stage 3: answering you
When you send a message, the model reads it along with the conversation so far (its context) and generates a response one at a time. There's a little controlled randomness, which is why asking twice can give different wording.
Reasoning models first produce a hidden or visible "thinking" section where they work through the problem step by step, then write the answer. This takes longer but helps with complex questions.
Add-ons: tools
Modern assistants can also use tools: web search for current information, code for calculations, and connectors to your files and apps. The model decides when to use them, then reads the results before answering.
What this explains
- Why it's fluent: it learned from more writing than any person could read.
- Why it can be wrong: it predicts plausible text; it doesn't check facts unless using a tool.
- Why context helps: everything in your conversation shapes the prediction.
Neural networks in one paragraph
A neural network is a large mathematical function made of layers of simple units, connected by weighted links. The weights (parameters) determine how inputs are transformed into outputs. is the process of nudging billions of weights so that, across huge numbers of examples, the network's outputs get closer to the desired ones. No one writes the knowledge in; it's distributed across the weights in ways that are hard to read directly.
The transformer
Today's language models are built on the transformer design, introduced by Google researchers in 2017. Its key idea is attention: when processing each , the model weighs how relevant every other token in the context is. In "The trophy didn't fit in the suitcase because it was too big", attention helps the model link "it" to "trophy". Stacking many attention layers lets models track meaning across long passages, which is why they can follow long documents and conversations.
Tokens and the context window
Text is split into tokens: common words are often one token, rarer words several. Everything the model considers at once (your instructions, the conversation, uploaded documents, tool results) must fit in its , measured in tokens. Windows have grown from a few pages in early models to hundreds or even thousands of pages in leading models, though very long contexts still cost more and can reduce accuracy.
Scaling
A central finding of the last decade is that performance improves predictably with more data, more parameters and more compute (computing power). This drove the race to build ever-larger models and data centres. More recently, labs have also found big gains from post-training with reinforcement learning, and from letting models spend more compute at answer time by reasoning longer. These are why reasoning models made rapid progress in maths, science and coding.
Post-training in more detail
- Supervised : training on curated examples of ideal responses.
- Reinforcement learning from human feedback (RLHF): people compare answers; a reward model learns their preferences; the language model is trained to score highly on it.
- AI feedback and written principles: some labs use AI to critique outputs against a set of written principles, reducing reliance on human labelling.
- Reinforcement learning on verifiable tasks: rewarding correct answers on problems that can be checked automatically, which strengthens step-by-step reasoning.
Post-training also adds safety behaviour: refusing clearly harmful requests, handling sensitive topics carefully, and being honest about uncertainty.
Multimodal models
Many leading models are trained on images, audio and sometimes video as well as text, converting each into tokens the same network can process. That's why one assistant can read a photo, listen to your voice and generate an image.
Why it's hard to fully understand
Because knowledge is spread across billions of weights, it's hard to say exactly why a model gave a particular answer. Interpretability research tries to map what's inside, and has found internal "features" corresponding to concepts, but full understanding remains a work in progress. This is part of why testing (benchmarks, red-teaming, real-world evaluation) matters so much, and why haystak combines several independent sources when ranking models.
Open and closed models
Some labs keep their model weights private and offer access through apps and paid connections for developers (an API). Others publish the weights so anyone can run and modify the model (open-weight models). Open models enable research and local, private use; they also raise questions about misuse, since safeguards can be removed.
Try this prompt
Explain how you work to me as if I'm a curious 12-year-old, using an analogy about [something I like, such as cooking or football]. Then tell me one common misunderstanding about AI.
Replace the [brackets] with your own details
On haystak
Quick check · 1 of 3
What is an AI language model trained to do at its core?
Next step
AI and fairness: bias, copyright, privacy and the environment
Get a balanced, plain-English view of the biggest debates about AI, so you can form your own opinion.