ASAmol Shukla
Projects
Learning Hub
Prompts
Skills
Contact
Resume
Course Outline
Syllabus Overview

AI Tools: LLM & Prompt Engineering Mastery

Courses/AI Tools: LLM & Prompt Engineering Mastery/Lesson 20: Self-Consistency — Sample & Vote

AI Tools

Lesson 20: Self-Consistency — Sample & Vote

Run the same prompt several times and take the majority answer to lift accuracy on hard reasoning tasks, with examples.

45 mins9 mins read

The Problem with One Answer

Chain-of-thought helps — but the model can still pick a wrong reasoning path. Self-consistency runs the same reasoning prompt multiple times and takes the majority answer. Wrong paths are diverse; the right path repeats.

The Recipe

  1. Ask the question with chain-of-thought.
  2. Generate 3–5 independent answers (use a higher temperature like 0.5–0.7 so paths vary).
  3. Cluster the answers and pick the most common one (majority vote).

Worked Example

Question: A store sells shirts for $25. A 20% discount applies when
you buy 3 or more. How much do 5 shirts cost? Think step by step.

Run it 5 times (temperature 0.6):

Run Reasoning path Answer
1 5 shirts → discount applies → 20% of 25 = 5 → 20 × 5 100
2 Discount applies → 25 × 5 = 125 → 125 − 25 100
3 Forgot discount → 25 × 5 125
4 20% off 25 = 5 → 20 each → 5 × 20 100
5 Discount only on 3 → 3 × 20 + 2 × 25 110

Majority = 100 — the correct answer wins even though runs 3 and 5 went wrong.

When to Use It

Situation Use Self-Consistency?
Math, logic, multi-step reasoning ✅ Yes — biggest gains here
Factual recall (names, dates) ⚠️ Rarely helps — all runs share the same wrong memory
Creative writing ❌ No — you want variety, not a vote
Production APIs with budget ⚠️ 3–5x cost and latency; use only for critical answers

The Cost Trade-Off

Runs Accuracy gain Cost
1 baseline 1x
3 good 3x
5 diminishing returns 5x

Rule of thumb: start with 3 runs for high-stakes reasoning, and only escalate if the vote is split.

Common Mistakes

  • Voting on creative outputs — there is no "correct" creative answer.
  • Using temperature 0 — all runs give the same answer, so the vote is pointless.
  • Taking the "most common" of free-form essays — self-consistency works best with short, extractable answers (a number, a label, a one-line conclusion).

Key Takeaways

  • Self-consistency = multiple CoT runs + majority vote.
  • It fixes reasoning errors, not memory errors.
  • Works best with short, comparable answers; costs 3–5x.
  • Use 3 runs first; escalate only when the vote is split.

Next up: Tree of thought and persona prompting — exploring branches and assigning expertise.

python · worked example
# Self-consistency: sample several answers, take the majority
import random
from collections import Counter

answers = []

def sample_answer(seed):
    rng = random.Random(seed)
    # Simulates 5 CoT runs: correct answer 70% of the time
    return "21" if rng.random() < 0.7 else "20"

for i in range(5):
    ans = sample_answer(i)
    answers.append(ans)
    print(f"Sample {i+1}: {ans}")

vote = Counter(answers).most_common(1)[0]
print(f"\nMajority vote: {vote[0]} (appeared {vote[1]} times)")
Read-only sample

Lesson Code (python)

# Self-consistency: sample several answers, take the majority
import random
from collections import Counter

answers = []

def sample_answer(seed):
    rng = random.Random(seed)
    # Simulates 5 CoT runs: correct answer 70% of the time
    return "21" if rng.random() < 0.7 else "20"

for i in range(5):
    ans = sample_answer(i)
    answers.append(ans)
    print(f"Sample {i+1}: {ans}")

vote = Counter(answers).most_common(1)[0]
print(f"\nMajority vote: {vote[0]} (appeared {vote[1]} times)")

Console Output

Sample 1: 20
Sample 2: 21
Sample 3: 20
Sample 4: 21
Sample 5: 21

Majority vote: 21 (appeared 3 times)

Code Visualization Tips

  • Draw 5 reasoning paths as arrows from the question; the majority arrow wins the vote.
  • Make a tally chart of answers across runs — the 'cluster' around the correct one is the signal.
  • Sketch the cost ladder: 1 run = 1x, 3 runs = 3x, 5 runs = 5x with accuracy plateauing.

Professional Tips & Tricks

  • Extract the final answer as a single line ('Answer: ...') so votes are easy to compare.
  • Use temperature 0.5–0.7 for the samples — too low gives clones, too high gives noise.
  • Skip self-consistency for factual questions; ground those with RAG instead.

Python Code Judge & Practice Arena

LeetCode Style

Run real Python 3.12 WebAssembly code directly in your browser against automated test suites.

Solved:0 / 1
0 / 20 XP
Challenges:
Problem 1 of 1

Vote on Reasoning Paths

Medium+20 XP
Below are 4 CoT answers to 'A pizza costs $12. A 10% tip is added. What is the total?' — answer with the majority: (a) 13.2, (b) 13.2, (c) 12.1, (d) 13.2.
main.pyPython 3.12 (WASM)
1
2
3
4
5
6
7
8
9
10
11
12
Press Run Code to test or Submit to verify test cases

Test Your Knowledge

Instant feedback

Quick Check: Self-Consistency

1 / 3
What does self-consistency do?

Up next · Continue learning

Tree of Thought & Persona Prompting

Explore several reasoning branches before deciding, and assign the model a persona to unlock specialized perspectives — with examples.

10 mins read50 mins
Start next lesson
Previous: Prompting for Agentic WorkflowsNext: Tree of Thought & Persona Prompting
Made withbyAmol Shukla·amolshukla.online
ASAmol Shukla

AI Developer, Trainer & Agentic AI Expert building practical learning systems and real-world AI applications.

Explore

  • Projects
  • Learning Hub
  • Prompts
  • Skills
  • Contact
  • Experience
  • Blogs

Connect

  • Resume
  • Contact
© 2026 Amol Shukla·Created withbyamolshukla.online
Back to top