45 mins lesson duration•9 mins read

Lesson 6: Temperature Explained

What temperature actually does to the probability distribution, and when to use low, medium, or high values.

Temperature: The Creativity Dial

Temperature controls how random the model's token choices are. It reshapes the probability distribution before sampling:

Low temperature (0–0.3): the model almost always picks the most likely token → predictable, precise, factual.
Medium (0.5–1.0): a balance — the default for most assistants.
High (1.0–2.0): the model happily picks unlikely tokens → creative, varied, sometimes incoherent.

The Math (Intuition Only)

The model produces logits (raw scores) for every possible next token. Temperature divides the logits before applying softmax:

P(token) = \frac{e^{logit / T}}{\sum_{j} e^{logit_j / T}}

T → 0: the distribution collapses onto the single most likely token (greedy).
T = 1: original distribution.
T > 1: the distribution flattens — unlikely tokens become much more likely.

Choosing Temperature by Task

Task	Temperature	Why
Math, code, data extraction, JSON	0–0.3	One right answer — you want determinism
Email drafts, summaries	0.3–0.7	Professional but not robotic
General chat, Q&A	0.7–1.0	Default range
Brainstorming, marketing copy, story ideas	0.9–1.5	You want variety and surprise
Poetry, wild creative writing	1.2–2.0	Maximum novelty (accept some nonsense)

Practical Notes

Temperature ≠ intelligence. Lowering temperature makes a model more consistent, not smarter. A wrong answer at T=0.2 is confidently wrong.
Different APIs, different scales: some tools call it temperature, some creativity, some use 0–1 only. Check the docs.
For deterministic workflows (agents, pipelines, tests), set temperature to 0 or 0.1 — otherwise the same prompt can return different results every run.
Higher temperature + self-consistency: for creative tasks where quality matters, run several samples and pick the best (see Lesson 15).

Key Takeaways

Temperature reshapes probabilities: low = predictable, high = creative.
Use ~0 for code/math/structured output; 0.7–1.0 for chat; higher for ideation.
Temperature controls consistency, not capability.
Set T≈0 in agents and pipelines to avoid flaky behavior.

Next up: Top-p and top-k sampling — the other knobs that shape output.

Interactive Lesson Code Snippet

# How temperature reshapes the probability distribution
import math

def softmax_with_temperature(logits, temperature):
    scaled = [x / temperature for x in logits]
    exps = [math.exp(x) for x in scaled]
    total = sum(exps)
    return [e / total for e in exps]

logits = [5.2, 2.1, 1.4, 0.3]
tokens = ["Paris", "London", "Berlin", "Madrid"]

for temp in [0.2, 1.0, 1.5]:
    probs = softmax_with_temperature(logits, temp)
    print(f"Temperature {temp}:")
    for token, p in zip(tokens, probs):
        print(f"  {token:8s} {p*100:5.1f}%")
    print()

Language: python

Lesson Code (Python)

# How temperature reshapes the probability distribution
import math

def softmax_with_temperature(logits, temperature):
    scaled = [x / temperature for x in logits]
    exps = [math.exp(x) for x in scaled]
    total = sum(exps)
    return [e / total for e in exps]

logits = [5.2, 2.1, 1.4, 0.3]
tokens = ["Paris", "London", "Berlin", "Madrid"]

for temp in [0.2, 1.0, 1.5]:
    probs = softmax_with_temperature(logits, temp)
    print(f"Temperature {temp}:")
    for token, p in zip(tokens, probs):
        print(f"  {token:8s} {p*100:5.1f}%")
    print()

Console Output

Temperature 0.2:
  Paris    100.0%
  London     0.0%
  Berlin     0.0%
  Madrid     0.0%

Temperature 1.0:
  Paris     93.0%
  London     4.2%
  Berlin     2.1%
  Madrid     0.7%

Temperature 1.5:
  Paris     80.4%
  London    10.2%
  Berlin     6.4%
  Madrid     3.1%

Code Visualization Tips

Plot the distribution as bars at T = 0.2, 1.0, and 1.5 — watch the tall bar shrink as others grow.
Draw a slider labeled 'predictable → creative' and mark where each task type sits.
Run the same creative prompt at T=0.3 and T=1.4 and compare outputs side by side.

Professional Tips & Tricks

Before blaming a 'bad model', check the temperature — a coding tool set to 1.5 will hallucinate APIs.
For production pipelines, pin temperature to 0 and add a retry on parse errors.
Creative copy? Generate 5 samples at T=1.2 and pick the best instead of fighting one sample.

Python Code Judge & Practice Arena

LeetCode Style

Run real Python 3.12 WebAssembly code directly in your browser against automated test suites.

Solved:0 / 1

0 / 10 XP

Challenges:

Problem 1 of 1

Pick the Right Temperature

Easy+10 XP

Match each task to a temperature: (a) extracting a date from an email, (b) writing 10 ad headlines for a shoe brand, (c) drafting a polite rejection email.

main.pyPython 3.12 (WASM)

# Pick the Right Temperature
# Write your solution below

def solve():
    pass

Press Run Code to test or Submit to verify test cases

Test Your Knowledge

Instant feedback

Quick Check: Temperature Explained

1 / 3

What does a low temperature (0–0.3) do?

Up next · Continue learning

Top-K & Top-P (Nucleus) Sampling

The other sampling dials: top-k trims the candidate list, top-p trims by probability mass. Learn to combine them.

9 mins read45 mins

Start next lesson

Previous: Managing Long Contexts Next: Top-K & Top-P (Nucleus) Sampling

Made withbyAmol Shukla·amolshukla.online