← सभी गाइड पर लौटें

Next.js LLM Integration: Fix API, Streaming & Cost Issues

Integrating LLMs into Next.js: Start Server-Side

The safest architecture is straightforward: your browser calls a Next.js Route Handler, and that handler calls the LLM provider. Never send the provider API key to the browser, even for a prototype. This guide uses the App Router and the OpenAI JavaScript SDK.

59API is a useful low-cost option because it offers pay-as-you-go access to Claude and GPT models through an OpenAI-compatible interface. Its native, official-quality models avoid downgraded substitutes, while SDK compatibility reduces integration work. Still, compare your actual workload’s input, output, and latency requirements before choosing a model.

1. Configure Credentials and a Minimal Route

• Install the SDK with npm install openai.
• Add LLM_API_KEY, LLM_BASE_URL, and LLM_MODEL to .env.local. Do not prefix secrets with NEXT_PUBLIC_.
• For 59API, the API host is https://api.59api.com. Set LLM_BASE_URL to the documented OpenAI-compatible base endpoint, including any required path prefix. Copy an available model ID exactly; display names are not necessarily API identifiers.
• Restart the development server after changing environment variables. Add the same variables to your deployment environment separately.

Create app/api/chat/route.ts with this minimal non-streaming handler:

import OpenAI from 'openai'; export const runtime = 'nodejs'; export async function POST(request: Request) { const { prompt } = await request.json(); if (typeof prompt !== 'string' || !prompt.trim() || prompt.length > 8000) return Response.json({ error: 'Invalid prompt' }, { status: 400 }); const client = new OpenAI({ apiKey: process.env.LLM_API_KEY!, baseURL: process.env.LLM_BASE_URL! }); const result = await client.chat.completions.create({ model: process.env.LLM_MODEL!, messages: [{ role: 'user', content: prompt }] }); return Response.json({ text: result.choices[0]?.message?.content ?? '' }); }

From a Client Component, POST JSON shaped as { "prompt": "Explain this error" } to /api/chat with Content-Type set to application/json. Check response.ok before displaying the returned text. This starter needs authentication, rate limiting, malformed-JSON handling, and sanitized upstream-error handling before public deployment.

2. Why Am I Getting 401, 404, or Model Errors?

• 401 or 403: Check the relay-issued key, account permissions, and deployment variables. An original provider key is not automatically valid at a relay.
• 404: Inspect the requested URL. A missing or duplicated endpoint prefix can send requests to the wrong route. Do not append /chat/completions to baseURL; the SDK adds the resource path.
• Unknown model: Confirm the exact model ID and account availability. Switching between Claude and GPT may also require changing unsupported request parameters.
• 429: Check both request limits and available balance. Retry transient rate limits with bounded exponential backoff and jitter, not an unlimited loop.

Log the upstream status and request identifier when available. Avoid logging authorization headers, complete prompts, or sensitive model output.

3. Why Does Streaming Arrive All at Once?

First confirm that non-streaming requests work. Then enable stream: true and forward chunks from the server as they arrive. Calling response.json() or response.text() waits for the complete response, so neither provides incremental rendering.

Use an SSE-aware parser or a compatible streaming library in the client. Network chunks can split JSON records and UTF-8 characters; one chunk is not necessarily one event. Check whether your hosting platform or proxy buffers responses, and cancel upstream generation when the user disconnects where supported.

4. Why Does It Work Locally but Time Out in Production?

Compare your deployment’s execution limit with the model’s response time. Set an explicit SDK timeout, bound output using the selected model’s supported token-limit parameter, and shorten oversized conversation histories. Streaming improves perceived responsiveness but does not remove hosting execution limits. For genuinely long tasks, consider background jobs with polling.

FAQ: How Can I Keep Quality High and Spending Low?

Should every request use the strongest model? No. Route straightforward extraction or classification to a cheaper suitable model, and reserve larger models for difficult reasoning. Measure success on representative examples rather than judging a single impressive response.

What prevents unexpected bills? Authenticate users before making paid calls, enforce per-user quotas, cap input and output, and monitor token usage. Cache only when privacy and freshness requirements permit.

Why consider 59API? Its inexpensive pay-as-you-go access makes model comparisons affordable without committing to one provider. A referral rebate can add savings, subject to current terms. If that fits your project, consider signing up and testing a small, representative workload first.

मूल्य · दस्तावेज़