Lesson 4: What Is a Context Window?
The context window is everything the model can 'see' at once — input plus output. Understand limits, truncation, and memory.
What Is a Context Window?
The context window is the maximum number of tokens a model can process in a single request — input + output combined. It is the model's working memory: everything it "knows" in a conversation lives inside this window.
Why It Matters
| Model (approx.) | Context Window |
|---|---|
| Early GPT-3 | 2,048 tokens (~1,500 words) |
| GPT-3.5 / GPT-4 | 8K–32K tokens |
| GPT-4o / Claude 3.5 | 128K–200K tokens |
| Gemini 1.5 / 2.x | 1M–2M tokens |
A bigger window lets you paste whole documents, codebases, or long conversations. But bigger is not always better:
- Cost: you pay per token — every input token is billed.
- Latency: more tokens = slower responses.
- Attention dilution: models can "lose focus" in the middle of extremely long contexts ("lost in the middle" effect).
The Window Is Shared
Your prompt and the model's reply share the same window:
┌────────────────────────────────────────────────┐
│ CONTEXT WINDOW (e.g. 8,000 tokens) │
│ │
│ SYSTEM PROMPT .... 500 tokens │
│ CONVERSATION HISTORY .... 4,000 tokens │
│ YOUR NEW PROMPT .... 500 tokens │
│ ────────────────────────────────────────────── │
│ MODEL OUTPUT (max ~3,000 tokens left) │
└──────────────────────────────────────────────────┘
If your input exceeds the window, the model truncates — usually dropping the oldest messages. That silently erases earlier context.
What "The Model Has No Memory" Means
Outside a single request, an LLM remembers nothing. Chat UIs fake memory by resending the whole conversation inside the context window every time. That's why long chats get expensive and why old details disappear when the window fills up.
Practical Rules of Thumb
- Keep ~1,000–2,000 tokens free for the model's reply.
- Put the most important instructions at the start and end of your prompt (the middle gets the least attention).
- Summarize old conversation turns instead of replaying them in full.
- For big documents, use chunking + RAG (next lesson) instead of pasting everything.
Key Takeaways
- The context window = input + output tokens the model can process at once.
- Token limits cause silent truncation of the oldest content.
- LLMs have no memory between requests — "memory" is just re-sent context.
- Important instructions belong at the start and end of the window.
Next up: Working with text longer than the window — chunking, summarization, and RAG.
# Estimating how many tokens your prompt uses (heuristic: ~4 chars per token)
prompt = """You are a helpful assistant. Please summarize the annual report for Q3 2026 focusing on revenue growth and risks."""
estimated_tokens = len(prompt) // 4 + 1
context_limit = 8000
output_reserve = 2000 # space we keep for the model's reply
print(f"Prompt characters: {len(prompt)}")
print(f"Estimated tokens (len // 4 + 1): {estimated_tokens}")
print(f"Context window size: {context_limit} tokens")
print(f"Context used: {estimated_tokens / context_limit * 100:.1f}%")
print(f"Tokens remaining for output: {context_limit - estimated_tokens}")
# Will this fit with a 2,000-token output reserve?
fits = (estimated_tokens + output_reserve) <= context_limit
print(f"Fits with {output_reserve}-token output reserve: {fits}")Lesson Code (Python)
# Estimating how many tokens your prompt uses (heuristic: ~4 chars per token)
prompt = """You are a helpful assistant. Please summarize the annual report for Q3 2026 focusing on revenue growth and risks."""
estimated_tokens = len(prompt) // 4 + 1
context_limit = 8000
output_reserve = 2000 # space we keep for the model's reply
print(f"Prompt characters: {len(prompt)}")
print(f"Estimated tokens (len // 4 + 1): {estimated_tokens}")
print(f"Context window size: {context_limit} tokens")
print(f"Context used: {estimated_tokens / context_limit * 100:.1f}%")
print(f"Tokens remaining for output: {context_limit - estimated_tokens}")
# Will this fit with a 2,000-token output reserve?
fits = (estimated_tokens + output_reserve) <= context_limit
print(f"Fits with {output_reserve}-token output reserve: {fits}")Console Output
Prompt characters: 113
Estimated tokens (len // 4 + 1): 29
Context window size: 8000 tokens
Context used: 0.4%
Tokens remaining for output: 7971
Fits with 2000-token output reserve: TrueCode Visualization Tips
- Draw the context window as a bar and color it in as you add system prompt, history, prompt, and output.
- Simulate a long chat and watch the oldest messages fall off the left edge as the window fills.
- Compare context sizes of different models on a number line to internalize the scale jump.
Professional Tips & Tricks
- Move critical instructions to the system prompt AND repeat them at the end of your user prompt.
- Use 'summarize the conversation so far' every few turns to compress history before the window fills.
- When using long documents, tell the model which sections to prioritize instead of dumping everything.
Python Code Judge & Practice Arena
LeetCode StyleRun real Python 3.12 WebAssembly code directly in your browser against automated test suites.
Context Budget Check
Test Your Knowledge
Instant feedbackQuick Check: The Context Window
Up next · Continue learning
Managing Long Contexts
Chunking, sliding windows, summarization, and retrieval — the techniques that let you work with text far larger than any window.