← Voltar aos guias

Stream Claude and GPT Responses for Less

Streaming improves responsiveness, not the price per token

Streaming lets your application display an AI response as it is generated instead of waiting for the complete answer. That can make a support chat or coding assistant feel much faster, especially when a response takes several seconds. But streaming alone does not make the model cheaper: if it generates 800 output tokens, you generally pay for those tokens whether they arrive all at once or in chunks.

The savings come from what streaming enables. Users can see progress sooner, stop an answer they no longer need, and avoid asking the model to repeat work. Combine those behaviors with sensible output limits and model selection, and you can reduce both wasted generation and perceived wait time.

Set up a streaming request

Claude and GPT-compatible APIs typically stream events over Server-Sent Events (SSE). Send a request with streaming enabled, then read events until the server signals that generation is complete. For a GPT-compatible request, set stream to true. Claude’s Messages API also accepts a streaming option. Handle partial text as it arrives rather than trying to parse each chunk as a complete response.

With 59API, use https://api.59api.com as the API base URL. The relay is designed for compatibility with Claude Code, Codex, and OpenAI SDKs, so you can often keep your existing client and change the base URL and credentials. Check the model and request format you select against the current API documentation before deploying.

In your event handler, append each text delta to the current response, update the interface, and stop when the stream ends. Also handle disconnects, errors, and cancellation. If a user presses Stop, cancel the underlying request where supported; merely hiding the output may leave generation running and incur charges.

Use concrete controls to cut waste

• Set a maximum output length. If a routine status summary usually needs about 150 tokens, do not allow an unrestricted multi-page response. Start with a limit around 250 tokens, test answer quality, and adjust. The limit is a ceiling, not a target.
• Offer a stop button. If a user has enough information after 100 tokens, cancellation can prevent the remaining response from being generated. Measure actual usage, because already-generated tokens are still billable.
• Choose the model by task. Use a lower-cost model such as Haiku for straightforward classification or short summaries, and reserve stronger options such as Sonnet or Opus for tasks that need deeper reasoning. Test a representative set of prompts before switching production traffic.
• Keep prompts focused. Remove duplicated instructions and send only relevant conversation history. For example, trimming 2,000 unnecessary input tokens from 1,000 daily requests avoids sending two million input tokens over that period.
• Track tokens and completion rates. Log model, input and output usage, cancellations, and whether users accepted the answer. Compare cost per successfully completed task, not just cost per request.

Make streaming reliable in production

Do not assume that every event contains a full sentence, or that a network chunk matches a model token. Buffers can split text in the middle of a word. Accumulate deltas safely, and use the provider’s completion event or stream termination to determine when the answer is finished. If you parse structured output, wait until enough content has arrived before attempting to validate it.

Set timeouts and provide a useful fallback message if the connection drops. For long-running generations, show that the assistant is still working, but do not automatically retry a request after partial output unless you have a plan to prevent duplicate work and charges.

Keep relay costs predictable

For teams comparing API routes, 59API offers pay-as-you-go access to Claude Opus, Sonnet, Haiku, Fable, and GPT models. It positions itself as a low-cost relay using native, official-quality models rather than downgraded substitutes, and also offers a referral rebate. Confirm current model rates and billing details before estimating a production budget; prices can vary by model and usage.

Start with a small streaming test, record token usage, and compare cost per useful result across models. If you want to try a compatible relay without changing your whole application, you can sign up for 59API and test it with a limited workload first.

Preços · Docs