Lesson 5: Self-Reflection & Self-Critique

The Power of Self-Reflection

One of the most powerful techniques in prompt engineering is getting the model to evaluate and improve its own outputs. This mirrors how human experts work — they don't just produce work, they review and refine it.

Why Self-Reflection Works

1. Separation of Concerns: Generating and evaluating are different cognitive tasks.
2. Error Detection: The model can catch mistakes it made in the first pass.
3. Quality Improvement: Iterative refinement produces better outputs.
4. Reduced Hallucination: Self-checking reduces made-up information.

Mental model: Think of self-reflection as built-in quality control — the model becomes its own editor, catching errors and suggesting improvements.

Technique 1: Generate → Evaluate → Refine

The fundamental self-reflection pattern:

Step 1: Generate Initial Output

Write a marketing email for our new product launch.

Step 2: Evaluate the Output

Now evaluate this email critically:
- Is the subject line compelling?
- Does the opening hook the reader?
- Is the call-to-action clear and urgent?
- Are there any clichés or weak phrases?
- Score it 1-10 and explain your reasoning.

Step 3: Refine Based on Evaluation

Now rewrite the email addressing all the issues you identified.

Technique 2: Chain of Verification (CoVe)

A structured approach to fact-checking the model's own outputs:

Step 1: Generate initial response Step 2: List all factual claims made Step 3: Verify each claim independently Step 4: Produce final, verified response

Example:

1. Generate: "Write a summary of Python's history"

2. Extract claims:
   - Python was created by Guido van Rossum
   - First released in 1991
   - Named after Monty Python

3. Verify each claim:
   - Guido van Rossum ✓ (confirmed)
   - 1991 ✓ (confirmed)
   - Monty Python connection ✓ (confirmed)

4. Final response: [Verified summary]

Technique 3: Constitutional AI Prompting

Inspired by Anthropic's Constitutional AI, this technique gives the model principles to evaluate its own outputs.

Example:

Write a customer service response to an angry customer.

Then evaluate your response against these principles:
1. Is it empathetic and acknowledge the customer's frustration?
2. Does it avoid blaming the customer?
3. Does it offer a clear resolution?
4. Is the tone professional but warm?
5. Does it maintain brand voice?

If any principle is violated, revise the response.

Common Constitutional Principles:

• Accuracy: Is the information correct?
• Helpfulness: Does it actually solve the user's problem?
• Safety: Is it free from harmful content?
• Clarity: Is it easy to understand?
• Completeness: Does it address all aspects of the question?

Technique 4: Refinment Prompts

Structured prompts that guide the model through systematic improvement:

Review the following [content type] and improve it:

[Original content]

Improvement checklist:
1. Clarity: Are any sentences confusing or ambiguous?
2. Conciseness: Can any words be removed without losing meaning?
3. Accuracy: Are all facts correct?
4. Structure: Is the organization logical?
5. Engagement: Is it interesting to read?
6. Grammar: Are there any errors?

For each issue found:
- Quote the problematic text
- Explain why it's an issue
- Provide the improved version

Then provide the complete revised version.

Technique 5: Perspective Shifting

Ask the model to evaluate from different viewpoints:

You're writing a blog post about remote work.

First, write the initial draft.

Then evaluate it from these perspectives:

1. As a CEO: Is this practical and business-focused?
2. As an employee: Does this resonate with daily experiences?
3. As a skeptic: What counterarguments are missing?
4. As an editor: Is the writing quality high?

Synthesize feedback and create an improved version.

Building Self-Reflection Into Your Prompts

Template: Reflective Prompt

[TASK: Your original request]

After generating your response:
1. Identify 3 potential weaknesses or errors
2. Suggest specific improvements for each
3. Rate your confidence in the final answer (1-10)
4. If confidence < 7, explain what additional information would help

Provide your final, improved response.

Common Mistakes to Avoid

• Mistake: Skipping the evaluation step — Fix: Always include explicit evaluation criteria.
• Mistake: Accepting the first improvement — Fix: Run 2-3 reflection cycles for critical content.
• Mistake: Vague evaluation criteria — Fix: Be specific about what "good" looks like.
• Mistake: Not validating facts — Fix: Use Chain of Verification for factual content.

Professional Tips & Tricks

• For high-stakes content, always run at least one reflection cycle.
• Use checklists to make evaluation systematic and repeatable.
• Document common issues you find — they become prevention items for future prompts.
• Combine self-reflection with few-shot examples of good evaluations.

Key Takeaways

• Self-reflection separates generation from evaluation, improving quality.
• The Generate → Evaluate → Refine pattern is foundational.
• Chain of Verification fact-checks the model's own claims.
• Constitutional AI gives the model principles to evaluate against.
• Perspective shifting catches blind spots by evaluating from multiple viewpoints.

Next up: Handling edge cases and error recovery in prompts.