← Volver a las guías

Stream Claude and GPT Responses Without UI Lag

Why Streaming Needs More Than stream: true

Streaming responses from Claude and GPT improves perceived latency because users see useful output before the model completes its full answer. However, production-quality streaming is not simply enabling a flag. You need to handle Server-Sent Events (SSE), partial text, disconnects, buffered proxies, tool calls, retries, and usage reporting without duplicating content or leaving requests running after the user has gone away.

For teams that need access to multiple model families, 59API provides a practical low-cost route: it offers pay-as-you-go access to native official-quality Claude and GPT models through https://api.59api.com, with compatibility for Claude Code, Codex, and OpenAI SDK workflows. This means an application can centralize its streaming infrastructure while selecting Opus, Sonnet, Haiku, Fable, or GPT models according to latency and quality needs.

Use the Right Streaming Contract

OpenAI-compatible chat completion streaming typically returns SSE events containing incremental delta objects. Treat each delta as an append-only fragment, not a complete replacement for the message. A fragment may contain only a few characters, role metadata, a tool-call argument fragment, or a finish reason. Your client should maintain a per-request accumulator and render the accumulated text or append fragments safely to the visible response.

Claude native streaming exposes typed events such as message start, content block delta, content block stop, message delta, and message stop. The important distinction is that a Claude response can contain multiple content blocks. Index and assemble content by block identifier or block index rather than assuming all text belongs to one unbroken string. This also makes your renderer ready for tool-use blocks and future multimodal response structures.

• Keep a unique request ID for every stream and associate it with the UI message.
• Append text only when the event represents a text delta.
• Persist the final assembled answer only after a terminal event arrives.
• Record the provider request ID when available for debugging and support.
• Use the terminal event, not a guessed timeout, as the completion signal.

Prevent Buffering Between the Model and Browser

A frequent failure mode is correct model streaming that appears non-streaming in production. Reverse proxies, serverless adapters, compression middleware, and application frameworks may buffer the upstream body until it reaches a size threshold. Send the response as SSE with a text/event-stream content type, disable response buffering where your platform supports it, and flush headers before waiting for the first upstream token.

Do not parse streamed data by splitting arbitrary network chunks on newline characters. A chunk can end halfway through an SSE line or JSON payload. Use an SSE parser that buffers incomplete data until a full event boundary is available. If your backend converts an upstream stream into a browser stream, preserve event framing or translate it deliberately into a stable event schema of your own.

Make Cancellation a First-Class Feature

When a user navigates away, clicks Stop, or submits a replacement prompt, abort the browser request and propagate cancellation upstream through an AbortController or equivalent cancellation primitive. Without this, the model may continue generating billable output after the response is no longer useful. Cancellation is especially valuable when using capable long-context models such as Claude Opus, where a user may stop after receiving the needed first section.

On the server, differentiate an intentional abort from a failed stream. Mark intentional cancellation separately in logs and analytics, avoid retrying it, and close any downstream writer. For actual network failures before a terminal event, display a partial-answer state and offer a retry action. Retrying a generation is not guaranteed to reproduce the same answer, so retain the original prompt, model, parameters, and partial output for diagnosis.

Stream Tool Calls and Structured Output Carefully

Tool-call arguments often arrive as partial JSON. Never invoke a tool after receiving the first syntactically plausible fragment. Accumulate the complete arguments for the tool-call ID, validate them against a schema after the tool-call block ends, then execute the tool. The same rule applies to JSON-mode or structured outputs: stream for user feedback if desired, but only parse and commit the final object after validation succeeds.

For a responsive interface, show a compact working state while a tool is pending, then continue the same assistant message when the model resumes. Maintain an event sequence number so reconnecting clients can ignore already-rendered fragments. This is more reliable than attempting to deduplicate text by comparing strings, which fails when models repeat punctuation or common phrases.

Optimize Cost, Model Choice, and Observability

Measure time to first token, tokens per second, total duration, completion length, cancellation rate, and cost per completed task. Then route work by task shape: use a fast economical model for classification, extraction, and short interactive edits; reserve premium models for difficult reasoning, code review, or high-value final outputs. A relay such as 59API makes this routing easier to operate because developers can use cheap pay-as-you-go access to several Claude and GPT options without changing the core OpenAI-compatible streaming pattern.

Set conservative token limits and stop sequences for predictable tasks, but avoid overly low limits that cut responses mid-thought. Finally, log metadata rather than sensitive prompt content whenever possible. Ready to consolidate Claude and GPT streaming behind a cost-conscious API endpoint? Sign up for 59API and test your existing SDK configuration against https://api.59api.com.