60 mins lesson duration•11 mins read

Lesson 13: RAG — Retrieval-Augmented Generation

RAG grounds LLM answers in your own data: index, retrieve, augment, generate. The professional standard for trustworthy AI.

What Is RAG?

Retrieval-Augmented Generation = retrieve relevant documents first, then let the LLM answer using only those documents. It solves the two biggest LLM problems at once: out-of-date knowledge and hallucinations over your data.

The RAG Pipeline

  DOCUMENTS                    QUERY
     │                           │
     ▼                           ▼
  CHUNK & EMBED              EMBED THE QUESTION
     │                           │
     ▼                           ▼
  VECTOR DATABASE ──retrieve top-k similar──▶ CONTEXT
                                                │
                                                ▼
                              PROMPT = QUESTION + CONTEXT
                                                │
                                                ▼
                                          LLM ANSWER

Stage	What Happens
1. Index	Split documents into chunks, embed each into a vector, store in a vector DB
2. Retrieve	Embed the user's question, find the most similar chunks (top-k)
3. Augment	Stuff the retrieved chunks into the prompt as context
4. Generate	The LLM answers using ONLY that context

Why It Works

The model never has to recall your facts — they're in the prompt.
Answers come with traceable sources (the retrieved chunks).
Your data can change daily without retraining anything.

Chunking Quality = Retrieval Quality

Retrieval is the weak link. Bad chunks → irrelevant context → bad answers. Improvements, in order of impact:

Chunk on semantic boundaries (headings, paragraphs, functions).
Add metadata (source, date, section) to chunks — enables filtering.
Hybrid search — combine vector similarity + keyword (BM25) hits.
Rerank — a second pass reorders retrieved chunks by true relevance.
Tune top-k — too few = missing context; too many = noise + cost.

The RAG Prompt (Augment)

Answer the QUESTION using ONLY the CONTEXT below.
If CONTEXT lacks the answer, say "Not found in the knowledge base."
Quote the source (doc_id) of each claim.

CONTEXT:
[doc 1] (id: 101) ...
[doc 2] (id: 205) ...

QUESTION: ...

RAG vs. Fine-Tuning

	RAG	Fine-tuning
What it does	Injects facts at query time	Bakes behavior into weights
Data updates	Instant (re-index)	Retrain
Best for	Facts, docs, retrieval	Style, format, domain behavior
Cost	Cheap to update	Expensive to retrain
Hallucination fix	Strong	Weak

Rule of thumb: facts → RAG; behavior → fine-tuning; and usually prompts first.

Evaluating RAG

Retrieval quality: did the right chunk get found? (hit rate, MRR)
Answer quality: grounded in retrieved chunks? (citation adherence, faithfulness)
End-to-end: does the user get the right answer? (human or LLM-judge eval — Lesson 15)

Key Takeaways

RAG = index → retrieve → augment → generate.
It grounds answers in your data and eliminates the knowledge-cutoff problem.
Retrieval quality is everything — chunk well, add metadata, consider reranking.
Facts → RAG; behavior → fine-tuning.

Next up: Embeddings & vector databases — how retrieval actually finds the right chunks.

Interactive Lesson Code Snippet

# Mini RAG: retrieve relevant context, then answer only from it
documents = {
    "python": "Python is a high-level programming language created in 1991.",
    "llm": "An LLM predicts the next token in a sequence.",
    "docker": "Docker packages applications into portable containers.",
}

def retrieve(query):
    # Keyword retrieval: score = number of query words found in the doc
    best, best_score = None, 0
    for key, text in documents.items():
        words = set(text.lower().split())
        score = sum(1 for word in query.lower().split() if word.strip("?.") in words)
        if score > best_score:
            best, best_score = text, score
    return best

def answer(question):
    context = retrieve(question)
    return f"Based on retrieved docs: {context}"

print(answer("What is an LLM and how does it predict?"))
print(answer("How does Python work?"))
print(answer("What does Docker do?"))

Language: python

Lesson Code (Python)

# Mini RAG: retrieve relevant context, then answer only from it
documents = {
    "python": "Python is a high-level programming language created in 1991.",
    "llm": "An LLM predicts the next token in a sequence.",
    "docker": "Docker packages applications into portable containers.",
}

def retrieve(query):
    # Keyword retrieval: score = number of query words found in the doc
    best, best_score = None, 0
    for key, text in documents.items():
        words = set(text.lower().split())
        score = sum(1 for word in query.lower().split() if word.strip("?.") in words)
        if score > best_score:
            best, best_score = text, score
    return best

def answer(question):
    context = retrieve(question)
    return f"Based on retrieved docs: {context}"

print(answer("What is an LLM and how does it predict?"))
print(answer("How does Python work?"))
print(answer("What does Docker do?"))

Console Output

Based on retrieved docs: An LLM predicts the next token in a sequence.
Based on retrieved docs: Python is a high-level programming language created in 1991.
Based on retrieved docs: Docker packages applications into portable containers.

Code Visualization Tips

Draw the 4-stage RAG pipeline as a conveyor belt with the query entering at the right moment.
Color-code a RAG answer: green = grounded in context, red = not in retrieved chunks.
Sketch the 'retrieval funnel': all chunks → top-k → reranked top-k → prompt.

Professional Tips & Tricks

Add source IDs to every chunk — citation tracking becomes trivial.
Log what was retrieved for each question; most RAG bugs are retrieval bugs.
Start with top-k=4–6; too much context dilutes the answer and inflates cost.

Python Code Judge & Practice Arena

LeetCode Style

Run real Python 3.12 WebAssembly code directly in your browser against automated test suites.

Solved:0 / 1

0 / 20 XP

Challenges:

Problem 1 of 1

Design a RAG System

Medium+20 XP

Outline the RAG system for a university's course-catalog chatbot (hundreds of PDFs). List chunking strategy, metadata, retrieval, and the prompt rules.

main.pyPython 3.12 (WASM)

# Design a RAG System
# Write your solution below

def solve():
    pass

Press Run Code to test or Submit to verify test cases

Test Your Knowledge

Instant feedback

Quick Check: RAG — Retrieval-Augmented Generation

1 / 3

What are the four stages of the RAG pipeline?

Up next · Continue learning

Embeddings & Vector Databases

Embeddings turn text into coordinates where meaning = proximity. Learn similarity search and choosing a vector database.

10 mins read55 mins

Start next lesson

Previous: Building a Simple Agent Next: Embeddings & Vector Databases

Made withbyAmol Shukla·amolshukla.online