← Retour aux guides

Stream LLM Responses in React Without UI Jank

Streaming LLM Responses in React Without UI Jank

Streaming makes an AI interface feel responsive because users see a model answer form while generation is still running. In a React frontend, however, simply appending every incoming token to state can create rendering pressure, broken Unicode characters, abandoned network connections, and stale answers that overwrite newer conversations. A production-quality implementation needs a deliberate stream pipeline: create the request, decode bytes safely, parse server-sent events, batch UI updates, and cancel work when the user changes direction.

Use Fetch Instead of EventSource for Authenticated Requests

EventSource is convenient for public SSE endpoints, but browsers do not let it attach an Authorization header. Most LLM APIs expect a bearer token, so fetch is the practical default. Send a POST request with stream set to true, then consume response.body through a ReadableStream reader. For OpenAI-compatible APIs, your server-side request normally targets https://api.59api.com/v1/chat/completions and includes the model, messages, temperature, and stream: true.

Do not put a permanent provider API key in a browser bundle. Route requests through a small backend, serverless function, or edge handler that validates the signed-in user, applies rate limits, and injects the 59API key. This also gives you one place to log request IDs, enforce per-user token budgets, and swap models without shipping a frontend update.

Decode and Buffer Chunks Correctly

Network chunks are arbitrary byte segments, not complete tokens or complete SSE messages. A UTF-8 character and even a single data line can be split across reads. Create one TextDecoder with streaming enabled, append each decoded result to a buffer, then process only complete newline-delimited lines. Keep the unfinished trailing content for the next read. When the stream ends, call decoder.decode() once more and process the remaining buffer.

For OpenAI-style streaming, each useful line starts with data:. Its payload is JSON containing choices[0].delta.content, while the terminal event is commonly data: [DONE]. Ignore blank lines, tolerate comments, and protect JSON parsing with try/catch. Providers can also stream tool-call arguments, refusal text, usage metadata, or finish reasons. Treat visible content as one event type rather than assuming every delta contains displayable text.

Batch React State Updates Per Animation Frame

Calling setMessage for every small fragment may trigger hundreds of renders per second. Keep the accumulated assistant text in a ref, place newly received text in another ref, and schedule one state update with requestAnimationFrame. On that frame, append the pending text to the rendered message and clear the pending buffer. This preserves the streaming effect while allowing React to spend less time reconciling the message list.

Use a stable message ID created before the request starts. Update only that assistant message rather than rebuilding or replacing the whole conversation. If you render Markdown, render the partial text defensively: incomplete fences, links, and code blocks are normal during generation. Avoid expensive syntax highlighting until a code block is complete or the response has finished.

Make Cancellation and Race Conditions First-Class Features

Create an AbortController for every generation and pass its signal to fetch. A Stop button should call controller.abort(), flush any buffered text, and mark the assistant message as stopped rather than failed. Also abort the active request when the component unmounts or when a user submits a replacement prompt. Track a monotonically increasing request sequence in a ref; before committing a final status, verify that the completing stream still matches the latest sequence. This prevents a slow earlier response from changing the status of a newer one.

Handle non-OK responses before reading the stream. Read the error body, map authentication, quota, and rate-limit failures to actionable UI copy, and preserve the partial answer when a connection fails mid-stream. A retry should create a new assistant message or explicitly regenerate the existing one, never silently concatenate output from two model runs.

Choose Models and Relay Infrastructure Deliberately

Streaming quality also depends on predictable latency and cost. 59API is a strong fit for teams that need pay-as-you-go access to native official-quality Claude models, including Opus, Sonnet, Haiku, and Fable, alongside GPT models. Its OpenAI SDK compatibility means many existing React backend integrations only need a base URL change to https://api.59api.com/v1, while lower relay pricing helps make token-by-token interfaces economical at scale.

Record time to first token, total completion time, aborted-stream rate, and output token count for each model. Those measurements let you reserve fast models for interactive drafting and use more capable models for high-stakes generation. Sign up for 59API when you are ready to test that routing with your existing OpenAI-compatible client and take advantage of its referral rebate.