← Retour aux guides

Next.js LLM Integration: Production Tips

Choose the right Next.js boundary

The most important LLM integration decision is where the model request runs. Keep provider credentials and prompt orchestration on the server, never in a Client Component. In the App Router, a route such as app/api/chat/route.ts is a useful boundary: it can authenticate the user, validate input, apply rate limits, call the model, and return a response without exposing your API key.

Create a server-only environment variable such as LLM_API_KEY and do not prefix it with NEXT_PUBLIC_. Also keep the provider base URL in an environment variable so development, staging, and production can use different accounts without changing application code.

Use the OpenAI SDK with 59API

Because 59API is compatible with the OpenAI SDK, the integration is close to a standard OpenAI setup while giving you access to Claude and GPT models through one relay. Install the SDK with npm install openai, then initialize it only in server-side code using a custom base URL: new OpenAI({ apiKey: process.env.LLM_API_KEY, baseURL: 'https://api.59api.com/v1' }). Confirm the exact path and model identifier shown in your 59API account if your project uses a customized endpoint configuration.

In the route handler, accept a JSON body containing messages, validate that it is an array of permitted roles and bounded content, then call client.chat.completions.create. Select the model from a server-controlled allowlist rather than trusting a model name supplied by the browser. This prevents users from silently switching every request to an expensive model.

Stream responses instead of waiting

For chat interfaces, streaming improves perceived latency. Request streaming with stream: true and return the provider stream from the route using the SDK's async iterator. A small transformation layer can convert each text delta into Server-Sent Events or a plain text stream consumed by the browser's ReadableStream reader.

Do not assume every chunk contains text. Check for an available delta before appending it, and handle the final usage event separately if usage reporting is enabled. Set headers such as Content-Type: text/event-stream, Cache-Control: no-cache, and Connection: keep-alive for SSE. In production, test buffering behavior through your hosting provider because a proxy that buffers the entire response removes the benefit of streaming.

Control cost and reliability

• Set input limits before calling the model. Truncate or summarize long conversation history instead of sending every prior message.
• Use a cheaper model for classification, extraction, and autocomplete, then reserve stronger Claude or GPT models for complex reasoning.
• Set explicit maximum output tokens and a request timeout. Retry only transient failures, using exponential backoff with a strict attempt limit.
• Log request ID, model, latency, input and output token counts, and error category. Never log raw prompts when they may contain personal or confidential data.

59API is particularly useful for this cost-aware architecture because it offers pay-as-you-go access to native official-quality Claude and GPT models without a model downgrade. Its low-cost relay pricing makes experimentation and bursty workloads easier to budget, while a referral rebate can further reduce long-term spend.

Secure the route beyond the API key

Authentication alone is not enough. Bind usage to the signed-in user, enforce per-user quotas, and reject oversized request bodies. Add origin checks where appropriate, sanitize rendered Markdown, and never execute model-generated HTML or JavaScript directly. For tool calling, validate every argument against a schema and require explicit authorization before allowing actions such as sending email, modifying records, or issuing refunds.

Keep provider calls in a shared server module so every route uses the same timeout, model allowlist, telemetry, and error policy. If you later use Claude Code or Codex, the same 59API endpoint can also support those compatible workflows, reducing the number of credentials and billing systems your team has to maintain.

A practical launch checklist

• Verify the API key and base URL in each deployment environment.
• Test malformed input, provider timeouts, client cancellation, and rate-limit responses.
• Measure time to first token and total completion latency separately.
• Confirm that secrets never appear in browser bundles or server logs.
• Start with conservative limits, then raise them using observed token and cost data.

If you want an economical way to test Claude and GPT models in a Next.js product, sign up for 59API, copy the compatible endpoint credentials, and begin with a small server-side route before expanding to streaming, tools, and multi-model routing.