← सभी गाइड पर लौटें

Streaming LLM Responses in React: Production Tactics

Why Token Streaming Needs More Than a Loading Spinner

Streaming an LLM response makes a React chat interface feel dramatically faster because users see the first useful token before the full answer is generated. However, a production implementation must handle partial UTF-8 characters, interrupted connections, rapid re-renders, cancellation, malformed stream events, and duplicate submissions. Treat streaming as an I/O pipeline rather than simply appending text inside a loop.

A cost-effective model relay also matters when every prompt creates a long-lived request. 59API provides pay-as-you-go access to native official-quality Claude and GPT models through https://api.59api.com, while remaining compatible with OpenAI SDK conventions. That makes it practical to use faster, lower-cost models for routine chat turns and reserve premium models for complex work without redesigning the frontend integration.

Keep API Credentials Out of the Browser

Do not call an LLM provider directly from a React component when the request requires a secret API key. Route the browser request through a server-side endpoint, such as a Next.js route handler, Express endpoint, or serverless function. The frontend sends conversation messages to your application endpoint; that endpoint validates the user, applies rate limits, adds the 59API key, and proxies the streamed response back to the browser.

On the server, configure an OpenAI-compatible client with the 59API base URL and select the required model. Keep model selection server-controlled, even when the UI offers a quality or speed preference. Map those preferences to an allowlist so a client cannot silently choose an unexpectedly expensive model.

Read the Response as a Byte Stream

In React, use fetch with an AbortController and consume response.body through getReader(). Decode chunks with TextDecoder using stream mode enabled. This detail is important: network chunk boundaries do not necessarily match character boundaries. Decoding each chunk independently can corrupt multibyte text, especially for non-English output or emoji.

Many OpenAI-compatible streaming endpoints return server-sent event frames. Buffer decoded text until a newline boundary, then parse complete data lines. Ignore keep-alive comments, stop on the provider's completion sentinel, and parse JSON only after a complete event has arrived. Never assume one read equals one JSON object. Preserve any incomplete trailing data for the next chunk.

Batch UI Updates Instead of Rendering Every Token

Calling setState for every tiny token can cause excessive rendering, particularly on slower devices or long answers. Store incoming text in a ref, then flush it into state on requestAnimationFrame or at a short interval. This preserves the perceived streaming effect while limiting React reconciliation work. At completion, perform one final flush so no trailing text is lost.

Create the assistant message before opening the request and update that message by a stable ID. Avoid rebuilding the whole messages array from a stale closure. A functional state update is safer because a user may edit, retry, or start another action while a stream is active. For larger chat histories, keep transcript state separate from transient stream buffers.

Make Cancellation and Retries First-Class Features

Attach an AbortController to each generation and expose a Stop control while the request is active. Aborting should immediately prevent further UI writes, mark the message as stopped, and release the active-generation state. Abort the request during component cleanup as well; otherwise a response can attempt to update an unmounted chat view.

Use a request ID or generation counter to prevent an older stream from overwriting a newer answer. When a retry starts, invalidate the previous request before creating the next one. Retry only failures that are plausibly temporary, such as network errors, 429 responses, or selected 5xx responses. Do not automatically retry after partial output unless your server can safely identify the request and the user explicitly accepts a regenerated answer.

Stream Structured Data Carefully

For tool calls, citations, or JSON output, do not render each partial fragment as final structured data. Accumulate deltas in a buffer and parse only when the model signals completion. If the UI needs progressive status, display a neutral processing state rather than exposing invalid JSON. On the server, validate completed tool arguments against a schema before executing anything.

Log timing data for request start, first token, completion, cancellation, model, and token usage where available. These measurements reveal whether delays come from browser networking, your proxy, or model generation. With 59API's low-cost pay-as-you-go routing, teams can compare compatible Claude and GPT options using real latency and quality data rather than relying on assumptions. Sign up for 59API when you are ready to test production streaming economics with your existing OpenAI-compatible workflow.