What AI gets wrong, and how to spot it
Know the mistakes AI makes most often, so you can catch them before they cause problems.
AI is impressive, but it makes mistakes. The biggest one has a name: . That's when it makes something up, like a fake fact, quote, book or statistic, and says it confidently.
It happens because AI predicts what a good answer looks like. Usually that's right. Sometimes it just sounds right.
Watch out for:
- Specific facts and numbers: dates, prices, laws, statistics.
- Quotes and sources: it can invent articles or references that don't exist.
- Recent events: it may not know what happened recently.
- Maths: it can slip up on sums, especially long ones.
- Agreeing too easily: if you say "I'm right, aren't I?", it may just agree.
How to protect yourself:
- Ask "Where does this come from?" and check the source exists.
- Double-check anything important on an official website.
- Ask it "What might be wrong with this answer?"
- For health, money or legal questions, use AI to understand and prepare, then check with a professional.
The main mistakes
1. Hallucinations (making things up). The best-known problem. The assistant states something false with total confidence: a law that doesn't exist, a made-up statistic, a fake book title, an invented quote. Lawyers have been caught out submitting court documents citing cases that an AI invented.
2. Out-of-date information. Models learn from information up to a date. Unless the assistant searches the web, it may not know about recent prices, rules or events.
3. Maths and counting slips. Assistants are much better at maths than they used to be, but long calculations, percentages and counting can still go wrong.
4. Telling you what you want to hear. If you push ("Surely this is fine?"), assistants can be too agreeable. This is sometimes called sycophancy.
5. Missing context. It doesn't know your situation unless you say. Advice about rent, tax or employment depends heavily on where you live.
6. Bias. AI learns from human writing, which includes stereotypes. It can sometimes reflect them in subtle ways.
Why it happens
AI writes the most likely next words based on patterns it learned. It doesn't look facts up in a database unless it's using a search tool. So a very likely-sounding answer can be wrong, and the assistant itself often can't tell.
How to spot problems
- Be suspicious of precise details you didn't give it: exact figures, dates, names, references.
- Ask for sources, then open them. If a link doesn't work or the source doesn't say what was claimed, don't trust the claim.
- Use web search where the assistant offers it. Answers with linked sources are easier to check.
- Ask the same question another way, or in a new chat. If the answers disagree, check.
- Ask it to critique itself: "What are the weak points in this answer?" or "How confident are you?"
A sensible rule of thumb
| Type of task | How much to check |
|---|---|
| Rewording, brainstorming, drafting | Light check: does it say what you mean? |
| Explanations of general topics | Medium: does it match what you know? |
| Specific facts, numbers, deadlines | Always verify with a trusted source |
| Health, legal, money decisions | Use AI to prepare; decide with a professional |
Lesson 10 shows you a fuller method for checking answers.
Why language models hallucinate
A large language model is trained to continue text in the most plausible way. During training it sees enormous amounts of writing and adjusts itself to get better at predicting what comes next. It doesn't store facts in neat rows like a database; knowledge is spread across billions of numbers inside the model.
Most of the time, the most plausible continuation is also the true one, because true things are written about often. But for rare facts, specific numbers or anything the model saw little of, "plausible" and "true" can come apart. The model then produces something that has the shape of a correct answer: a reference in the right format, a statistic of the right size, a quote in the right voice. Because it was trained to sound fluent and helpful, it rarely signals doubt unless prompted.
Labs have reduced considerably through better training, by teaching models to say "I don't know", and by connecting them to search and documents so they can quote real sources. It hasn't been eliminated, and independent tests still find errors, especially on obscure topics.
Training cut-offs and search
Every model has a : the point after which it saw no new information. Assistants can bridge the gap with web search, but search brings its own risk: the assistant may pick a poor source or summarise it wrongly. Always look at which sources it used.
Sycophancy
Models are partly trained using human ratings, and people tend to rate agreeable answers highly. This can produce assistants that agree too readily or soften bad news. Labs actively work against this, but you can help by asking neutral questions ("What are the arguments for and against?") rather than leading ones ("This is a good idea, isn't it?").
Reasoning and maths
Modern "reasoning" models work through problems step by step before answering, and many can run calculations with a built-in tool. That has made them far more reliable at maths, but errors in setting up the problem (misreading a number or a question) still happen. Check the inputs as well as the answer.
Bias
Models learn from human-written text and can absorb its patterns, including stereotypes about gender, ethnicity, age or nationality. Labs test for and reduce this, but it can surface in subtle ways: assumptions in example names, uneven tone, or advice that fits one country's rules. Be especially careful when AI is used in decisions about people, such as hiring.
A practical verification toolkit
- Ask for sources and open them. Check the source exists and actually supports the claim.
- Prefer primary sources: the official government page, the company's own terms, the original study.
- Triangulate. Ask a second assistant, or the same one in a fresh chat with different wording.
- Ask for uncertainty. "Rate your confidence in each claim from 1 to 5 and say why."
- Separate drafting from deciding. AI is excellent for preparing (questions to ask a GP, a summary of your rights to take to Citizens Advice). The decision should rest on verified information.
Where the stakes are highest
In health, legal and financial matters, a confident wrong answer can cause real harm. The NHS website, GOV.UK, Citizens Advice and regulated professionals remain the places to confirm anything you'll act on. AI is a superb way to understand the jargon first.
Try this prompt
Answer this question, then tell me how confident you are and which parts I should double-check: [your question]
Replace the [brackets] with your own details
On haystak
Quick check · 1 of 3
What is a "hallucination" in AI?
Next step
Staying safe: what never to share, plus AI scams and fakes
Know what to keep out of chatbots and how to spot AI-powered scams, fake voices and fake images.