← Retour aux guides

Temperature vs. top_p: A Practical Sampling Guide

What sampling means in an AI model

When an AI model writes a response, it does not always select the single most likely next token. Instead, it calculates probabilities for possible tokens and samples from that distribution. A token may be a word, part of a word, punctuation, or whitespace.

Sampling controls how much choice the model has. Low randomness tends to produce predictable, focused answers. Higher randomness allows unusual wording, alternative ideas, and more creative combinations. Two settings are commonly used to control this behavior: temperature and top_p.

Temperature explained

Temperature changes the shape of the model's probability distribution. A low value makes high-probability tokens much more dominant. A high value flattens the distribution, giving less likely tokens a better chance of being selected.

• 0 to 0.2: focused and consistent output, useful for extraction, classification, and code.
• 0.3 to 0.6: a practical range for explanations, support replies, and general business writing.
• 0.7 to 1.0: more varied language and brainstorming.
• Above 1.0: increasingly unpredictable output, which may reduce accuracy or coherence.

Temperature does not add knowledge or improve reasoning by itself. It only changes which already-available token is selected. A temperature of 0 is also not a universal guarantee of identical responses: server-side behavior, parallel execution, model updates, and other settings can still introduce variation.

What top_p does

Top_p is also called nucleus sampling. The model sorts candidate tokens from most to least probable, then keeps the smallest group whose combined probability reaches the top_p value. It samples only from that group.

For example, top_p set to 0.9 allows tokens that together represent 90% of the probability mass. Very unlikely candidates are removed. A value near 1.0 keeps more possibilities, while a lower value makes selection more conservative.

• top_p 0.1 to 0.3: narrow, predictable choices.
• top_p 0.5 to 0.8: controlled variation for normal generation.
• top_p 0.9 to 1.0: broad sampling with more diversity.

Unlike temperature, top_p changes the candidate pool rather than continuously reshaping every probability. The effective number of candidates can vary from token to token because the model's confidence changes throughout a response.

Temperature and top_p together

Both parameters affect randomness, so changing both at once makes results harder to diagnose. A sensible starting practice is to adjust temperature while leaving top_p at 1.0, or adjust top_p while leaving temperature at its default. Many providers recommend tuning one primary control rather than combining aggressive settings.

For a reliable JSON extractor, start with temperature 0 to 0.2 and top_p 1. For a customer-support assistant, try temperature 0.3 to 0.6. For brainstorming names or campaign concepts, try temperature 0.7 to 0.9, then test whether the output remains useful. Always validate the result rather than assuming a setting is correct.

A practical testing method

• Keep the model, prompt, maximum output length, and input fixed.
• Run at least 10 to 20 requests for each setting.
• Measure task accuracy, formatting errors, repetition, refusal rate, and latency.
• Choose the lowest randomness that still produces the variety your task needs.
• Record the model version and parameter values so you can reproduce comparisons.

For structured output, also validate JSON in your application and retry malformed responses carefully. Sampling settings cannot replace schemas, type checks, or application-level error handling.

Using these settings with 59API

59API provides low-cost, pay-as-you-go access to native Claude models, including Opus, Sonnet, Haiku, and Fable, as well as GPT models. Its API base URL is https://api.59api.com, and it is compatible with Claude Code, Codex, and OpenAI SDK integrations.

When sending a request, pass temperature and, where the selected model supports it, top_p in the normal request parameters. Check the model's documentation because parameter support and default values can differ between providers. 59API is useful for testing several sampling configurations without committing to a large subscription, while still using official-quality models rather than downgraded substitutes.

Start with a small test budget, compare outputs against your actual task requirements, and then scale the configuration that performs best. Developers who want a low-cost relay can sign up for 59API and may also benefit from its referral rebate.

Tarifs · Docs