Lesson 4: Chain-of-Thought Variants

Beyond Basic Chain-of-Thought

Chain-of-thought (CoT) prompting revolutionized how we get LLMs to reason. But there are several variants, each optimized for different scenarios. Understanding when to use each variant is key to unlocking maximum performance.

The CoT Family

Mental model: Think of CoT variants as different levels of a reasoning engine — zero-shot CoT is like turning on cruise control, while self-consistency CoT is like having multiple expert advisors vote on the best approach.

Zero-Shot Chain-of-Thought

The simplest CoT variant — just append a magic phrase to your prompt.

The Magic Phrases:

• "Let's think step by step"
• "Let's work this out in a step by step way"
• "First, let's understand the problem"

Example:

Q: A store has 5 apples. They buy 3 more boxes with 4 apples each. 
Then they sell 6 apples. How many apples do they have?

Let's think step by step.

Output:

1. Start with 5 apples
2. Buy 3 boxes × 4 apples = 12 apples
3. Total after purchase: 5 + 12 = 17 apples
4. Sell 6 apples: 17 - 6 = 11 apples
5. Answer: 11 apples

Why it works: The phrase triggers the model to generate intermediate reasoning steps rather than jumping to a final answer. This dramatically improves accuracy on math, logic, and multi-step problems.

Few-Shot Chain-of-Thought

Provide examples that demonstrate explicit reasoning before asking for a new answer.

When to use:

• Complex reasoning patterns
• Domain-specific logic
• When zero-shot CoT isn't accurate enough

Example:

Solve these word problems step by step:

Q: Sarah has 3 times as many books as Tom. Together they have 24 books. How many does Tom have?
A: Let's define variables. Let Tom = x books. Sarah = 3x books. Together: x + 3x = 24. So 4x = 24. x = 6. Tom has 6 books.

Q: A train travels 60 mph for 2.5 hours, then 80 mph for 1.5 hours. What's the total distance?
A: First segment: 60 × 2.5 = 150 miles. Second segment: 80 × 1.5 = 120 miles. Total: 150 + 120 = 270 miles.

Q: Mike earns $25/hour and works 40 hours/week. He gets a 10% raise. What's his new weekly salary?
A: [Your reasoning here]

Key principles for few-shot CoT:

1. Show diverse reasoning patterns (not just the same type repeated)
2. Include the reasoning steps explicitly (not just the answer)
3. Match the complexity of your examples to the target problem
4. Use 3-5 examples for most tasks

Auto Chain-of-Thought

When you don't have good examples, let the model generate its own reasoning demonstrations.

Technique: Use clustering to generate diverse examples, then use those as few-shot demonstrations.

Simplified version:

Q: [Your problem here]

Let's solve this step by step. First, I'll identify what we know and what we need to find. Then I'll break this into smaller sub-problems. Finally, I'll combine the results.

[Model generates its own reasoning]

When to use:

• You don't have good examples available
• The problem domain is unfamiliar
• You want to explore different reasoning approaches

Complexity-Based CoT

Generate multiple reasoning paths and select the most coherent one.

Implementation:

1. Ask the model to solve the problem 3-5 times
2. Evaluate the complexity/quality of each reasoning chain
3. Select the answer from the most detailed reasoning

Example prompt:

Solve this problem 3 different ways, using different approaches each time:

[Problem statement]

For each approach:
1. Clearly state your method
2. Show all reasoning steps
3. State your final answer

Then, evaluate which approach seems most reliable and explain why.

Why it works: Different reasoning paths may catch different errors. By comparing multiple approaches, you increase the likelihood of finding the correct answer.

Self-Consistency CoT

The most robust CoT variant — generate multiple independent solutions and take the majority vote.

Process:

1. Generate N independent CoT solutions (typically 5-10)
2. Extract the final answer from each
3. Select the most common answer

Example:

Solve this problem independently 5 times:

Q: If a shirt costs $25 and there's a 20% discount on purchases of 3 or more items, how much do 5 shirts cost?

[Run 1]: ... Answer: $100
[Run 2]: ... Answer: $100
[Run 3]: ... Answer: $100
[Run 4]: ... Answer: $125 (different reasoning path)
[Run 5]: ... Answer: $100

Majority vote: $100 (4 out of 5 agree)

When to use:

• Math problems where accuracy is critical
• Logic puzzles
• Medical/legal analysis where errors are costly

Trade-off: 5x the tokens and API calls, but significantly higher accuracy.

Choosing the Right CoT Variant

Common Mistakes to Avoid

• Mistake: Using CoT for simple tasks — Fix: CoT adds overhead; use it only when reasoning steps are genuinely needed.
• Mistake: Too few examples in few-shot CoT — Fix: 3-5 diverse examples establish a clear pattern.
• Mistake: Not verifying the reasoning — Fix: The model can make logical errors even with CoT; validate critical steps.
• Mistake: Assuming more steps = better answer — Fix: Focus on quality of reasoning, not quantity.

Professional Tips & Tricks

• Start with zero-shot CoT ("Let's think step by step") — it's often sufficient.
• For few-shot CoT, include examples that cover edge cases.
• Self-consistency is worth the extra cost for high-stakes decisions.
• Document which CoT variant works best for your common use cases.

Key Takeaways

• Zero-shot CoT is the simplest: just add "Let's think step by step."
• Few-shot CoT provides examples with explicit reasoning chains.
• Auto-CoT generates its own examples when you don't have good ones.
• Self-consistency takes multiple solutions and votes on the best answer.
• Choose the variant based on task complexity and accuracy requirements.

Next up: Self-reflection and self-critique techniques for improving output quality.