← Volver a las guías

max_tokens and Temperature: A Coding Task Quick Start

Start with these coding defaults

For coding tasks, start with temperature 0–0.2 and an output budget matched to the deliverable. Try 2,000 tokens for a small patch, 4,000 for a focused test suite, or 8,000 for a complete module. These are starting points, not model-independent guarantees. Check the selected model’s supported parameters and output ceiling before sending the request.

The shortcut: use temperature to adjust variation, and use the output-token limit to prevent runaway answers. Neither setting replaces a clear specification, relevant source files, or executable tests.

1. Understand what each setting controls

max_tokens usually caps generated output, not the combined prompt and response. It does not mean “write exactly this much.” A model can finish earlier. Your prompt, conversation history, tool results, and requested output must also fit the model’s context constraints.

Temperature adjusts how broadly the model samples possible next tokens. Lower values generally produce more consistent answers; higher values encourage variation. Temperature zero does not guarantee identical responses across runs, nor does it guarantee correct code.

Parameter names depend on the endpoint. Anthropic Messages uses max_tokens. OpenAI Chat Completions uses max_completion_tokens for many newer models; older integrations may use max_tokens. OpenAI Responses uses max_output_tokens. Some reasoning models reject or restrict temperature. Follow the endpoint and model documentation rather than copying one request shape everywhere.

2. Match the budget to the requested artifact

• Small bug fix: Start at 2,000 output tokens and temperature 0–0.2. Request a unified diff instead of an entire rewritten file.
• Tests for one function: Start at 4,000 tokens and temperature 0–0.2. Specify the framework, fixtures, edge cases, and whether mocks are allowed.
• New module: Start at 8,000 tokens and temperature 0–0.2. State interfaces, dependencies, error handling, and expected exports.
• Alternative designs: Try temperature 0.4–0.7 if supported. Request three short approaches, choose one, then generate the implementation at a lower temperature.

Tokenization varies by model and language. Code, identifiers, and punctuation do not follow a reliable words-to-tokens ratio. Measure actual usage instead of budgeting from line count alone. On reasoning models, the output allowance may include reasoning tokens, leaving less room for visible code.

3. Make the prompt do the precision work

A useful request is: “Fix the null-handling bug in the supplied function. Preserve the public signature. Return only a unified diff and two regression tests. Do not add dependencies.” This narrows the artifact more effectively than simply lowering temperature.

Include relevant types, package versions, and the failing test output. Exclude unrelated files. If the requested implementation cannot reasonably fit, split it into interface design, implementation, and tests rather than forcing everything into a tiny budget.

4. Detect truncation before accepting code

Inspect the API’s completion status. Anthropic Messages reports stop_reason; OpenAI Chat Completions reports finish_reason; Responses exposes its own completion and incomplete-status fields. A token-limit stop is not a successful finished artifact.

If output is truncated, increase the budget or reduce the scope, then regenerate the affected artifact. Blindly appending “continue” can create duplicate functions or malformed JSON. Always parse structured output, apply patches in a disposable workspace, and run formatting, type checks, and tests.

5. Tune cost using completed tasks

Run ten representative tasks with the same model, prompts, and settings. Record input/output usage, truncation frequency, test pass rate, retries, and total cost per accepted change. Change one setting at a time. A smaller token ceiling can cost more overall if it causes repeated failed attempts.

59API is a low-cost, pay-as-you-go relay for Claude and GPT models, offering native, official-quality models without downgrading. It works with Claude Code, Codex, and OpenAI SDKs. Configure the service base at https://api.59api.com using the endpoint path required by your client; check model IDs and parameter support before switching.

Its inexpensive access makes repeated tuning experiments practical, and a referral rebate is available. If you want to benchmark these settings on your own repository, consider signing up for 59API and starting with a small, measured workload.

Precios · Docs