← Back to all guides

Build a Production-Ready Chatbot on an OpenAI-Compatible API

A chatbot backend needs to do more than forward a prompt. It should protect API credentials, keep conversation context, handle provider errors, and limit costs. This walkthrough builds a small Python service using FastAPI and the OpenAI SDK, then covers the changes that make it safer to deploy.

1. Create a project and configure credentials

Install the dependencies in a virtual environment: FastAPI, Uvicorn, the OpenAI Python package, and a settings library such as python-dotenv. Keep your provider key in an environment variable, never in browser code or a committed file. For local development, a .env file is convenient; add it to .gitignore.

For example, set OPENAI_API_KEY to the key from your API provider and OPENAI_BASE_URL to https://api.59api.com/v1. The 59API base URL is https://api.59api.com; OpenAI-compatible SDK integrations commonly use the /v1 API path, so confirm the exact path and model identifier in the provider’s current documentation. Treat the model name as configuration rather than hard-coding it throughout your app.

59API is worth considering when you want pay-as-you-go access to GPT and Claude models through an OpenAI-compatible interface. It offers low-cost access to native models, rather than a deliberately downgraded model, and a referral rebate. Compare the current model prices and capabilities with your expected traffic before choosing a default.

2. Build a minimal chat endpoint

Create a FastAPI application and initialize the OpenAI client with the key and base URL read from the environment. Add a POST /chat route that accepts a list of messages, each with a role and content. Restrict roles to user and assistant for client-supplied history; add the system instruction on the server so callers cannot replace your application’s rules.

In the route, call the chat completions method with the configured model and the assembled messages. Return the assistant’s text from the first choice as JSON, for example as an object with a reply field. Set a reasonable max_tokens value and temperature for your use case. A support bot that should answer consistently usually needs less randomness than a creative writing assistant.

Start locally with Uvicorn, then send a POST request containing a user message to /chat. Verify that the response has the expected shape, that an empty message is rejected, and that removing the API key produces a clear server-side configuration error. Do not return the provider key, raw exception details, or internal prompts to the client.

3. Preserve context without trusting the browser

Language models do not remember earlier requests automatically. For a prototype, the client can send recent user and assistant messages with each request. For production, store conversations server-side and associate them with an authenticated user. Load only the relevant recent turns, then append the new user message before calling the model.

Set limits on message length, turn count, and total context. Long histories increase latency and cost, and may exceed the model’s context window. When a conversation grows, summarize older turns or remove them according to your product’s retention policy. Validate request bodies with typed schemas, and reject unsupported roles and oversized payloads.

4. Make failures and usage manageable

Handle timeouts, rate limits, and provider errors deliberately. Return a stable error response to the client, log the request ID and status for diagnosis, and avoid logging full conversations unless you have a clear privacy basis. Retry only transient failures, use a small retry limit with backoff, and avoid retrying a request that may have already completed if your app cannot prevent duplicate charges.

• Protect access: authenticate users, apply per-user rate limits, and keep provider credentials on the server.
• Control spend: cap input size and output tokens, track usage by user, and set provider-side budget alerts where available.
• Observe behavior: record latency, error rates, selected model, and token usage without exposing sensitive content.
• Test changes: use a fixed set of prompts to check response quality before changing models or system instructions.

5. Deploy and verify

Run the service behind HTTPS, configure secrets through your hosting platform, and add a health check that does not make a paid model request. Set request timeouts at both the application and proxy layers. Before release, test invalid input, slow responses, rate limits, and concurrent requests. Keep the provider base URL and model configurable so you can compare models or switch settings without rewriting the endpoint.

This design keeps the first version small while leaving room for authentication, persistence, streaming, and usage controls. If you are comparing compatible providers, you can sign up for 59API and test a small workload against its available GPT and Claude models before routing production traffic.

Pricing · Docs