45 mins lesson duration•9 mins read
Lesson 8: Why LLMs Hallucinate
Hallucinations are a feature of how LLMs work, not a bug you can switch off. Understand the root causes.
What Is a Hallucination?
A hallucination is output that is fluent, confident, and wrong — a fact, citation, or code snippet that looks real but was invented.
Why It's Inevitable
Remember the core loop from Lesson 1: the model predicts the most plausible next token — not the true next token. "Truth" was never part of the training objective. The model optimizes for text that looks like the text it was trained on.
| Root Cause | Explanation |
|---|---|
| Next-token objective | The model picks likely text, not verified text |
| Knowledge cutoff | Anything after training time is unknown — the model fills the gap with plausible guesses |
| Rare facts | The model never saw the fact, so it improvises from similar patterns |
| Prompt ambiguity | Vague questions leave room for the model to invent specifics |
| Compression | Trillions of tokens compress into billions of weights — details are lost |
| Fine-tuning pressure | RLHF rewards fluent, helpful answers — even when wrong |
Types of Hallucinations
| Type | Example |
|---|---|
| Factual | "The capital of Australia is Sydney." (it's Canberra) |
| Fabricated sources | A citation to a real-looking paper that doesn't exist |
| Logical | A confident proof with a wrong step |
| Code | An API function client.doThing() that was never in the library |
| Instructional | Following a false premise you gave it ("as you said, X, so…") |
The "Plausibility Trap"
The danger is that hallucinations are indistinguishable in style from correct answers. A model won't hedge when it's guessing — confidence is not calibrated to truth. You can ask "are you sure?" and it will often double down.
When Hallucinations Are Most Likely
- Open-ended questions about niche or recent topics.
- Requests for specific numbers, dates, and citations.
- Tasks outside the training data (new APIs, your internal docs).
- Long outputs where the model drifts from the original context.
Key Takeaways
- Hallucinations come from the next-token objective — they cannot be "fixed" by prompting alone.
- The model optimizes plausibility, not truth; confidence ≠ correctness.
- New, niche, numeric, and cited content is where hallucinations live.
- Your job: build verification into the workflow (next lesson).
Next up: Detecting and reducing hallucinations — grounding, RAG, and verification.
Interactive Lesson Code Snippet
# Simulating the root cause: an LLM fills gaps with *plausible* text
knowledge = {
"Capital of France": "Paris",
"Capital of India": "New Delhi",
"Speed of light": "299,792 km/s",
}
def tiny_llm(question):
if question in knowledge:
return f"Known fact: {knowledge[question]}"
# Unknown question -> the model "invents" a confident answer
return "Confident but unverified: the answer is almost certainly true."
for q in ["Capital of France", "Capital of Atlantis", "Speed of light"]:
print(f"Q: {q}")
print(f"A: {tiny_llm(q)}\n")Language: python
Lesson Code (Python)
# Simulating the root cause: an LLM fills gaps with *plausible* text
knowledge = {
"Capital of France": "Paris",
"Capital of India": "New Delhi",
"Speed of light": "299,792 km/s",
}
def tiny_llm(question):
if question in knowledge:
return f"Known fact: {knowledge[question]}"
# Unknown question -> the model "invents" a confident answer
return "Confident but unverified: the answer is almost certainly true."
for q in ["Capital of France", "Capital of Atlantis", "Speed of light"]:
print(f"Q: {q}")
print(f"A: {tiny_llm(q)}\n")Console Output
Q: Capital of France
A: Known fact: Paris
Q: Capital of Atlantis
A: Confident but unverified: the answer is almost certainly true.
Q: Speed of light
A: Known fact: 299,792 km/sCode Visualization Tips
- Draw the knowledge boundary: training data inside the circle, everything after cutoff outside — note the model invents to fill the outside.
- Make a 'confidence vs. truth' scatter: correct answers and hallucinations both sit at high confidence.
- Keep a running list of real hallucinations you catch — you'll spot the pattern faster.
Professional Tips & Tricks
- For anything with dates, numbers, or citations, assume hallucination until verified.
- Notice the model's failure mode: it invents specifics to sound complete — ask for 'unknown' as an allowed answer.
- New versions of the same model still hallucinate — upgrading is not a fix; verification is.
Python Code Judge & Practice Arena
LeetCode StyleRun real Python 3.12 WebAssembly code directly in your browser against automated test suites.
Solved:0 / 1
0 / 10 XP
Problem 1 of 1
Classify the Hallucination
Easy+10 XP
Classify each as factual, fabricated-source, logical, or code hallucination: (a) 'According to Smith et al. 2019...' (paper doesn't exist), (b) 'The function pandas.read_xlsx() handles Excel files' (doesn't exist), (c) 'Mount Everest is 4,000 m tall'.
main.pyPython 3.12 (WASM)
1
2
3
4
5
6
7
8
9
10
11
12
Press Run Code to test or Submit to verify test cases
Test Your Knowledge
Instant feedbackQuick Check: Why LLMs Hallucinate
Which is NOT a root cause of hallucinations?
Up next · Continue learning
Detecting & Reducing Hallucinations
Grounding, RAG, citations, self-checking prompts, and workflow design — the practical toolkit for trustworthy LLM output.
10 mins read55 mins
Start next lesson