← Volver a las guías

Build a RAG App with Claude and GPT

What a RAG application does

Retrieval-augmented generation, or RAG, lets an AI answer questions using your own documents instead of relying only on its training data. A typical RAG app converts documents into searchable chunks, finds the most relevant chunks for a user’s question, and places those results into a prompt for Claude or GPT.

This approach is useful for product documentation, internal policies, support knowledge bases, and research tools. It can reduce hallucinations, provide source citations, and keep information current without fine-tuning a model.

Plan the basic architecture

Your application needs five parts: a document loader, a text splitter, an embedding model, a vector database, and a generation model. The embedding model turns text into numerical vectors. The vector database stores those vectors and returns chunks that are semantically similar to a query. Claude or GPT then writes the final answer from the retrieved context.

For a beginner project, a simple stack might use Python, a local folder of Markdown or PDF files, an embedding provider, and Chroma, Qdrant, or PostgreSQL with pgvector. Start with one document type and add more loaders after the retrieval quality is reliable.

Step 1: Ingest and split your documents

Extract clean text before indexing it. Remove navigation menus, repeated headers, and irrelevant boilerplate. Split the content into chunks of roughly 500 to 1,000 tokens with a 10% to 20% overlap. Overlap helps preserve meaning when a sentence or procedure crosses a chunk boundary.

Store useful metadata with every chunk, including the document name, URL, section heading, page number, and last-updated date. This metadata makes citations possible and lets you filter results, such as searching only the current product manual.

Step 2: Create embeddings and retrieve context

Generate one embedding for each chunk and save it in your vector database. At query time, embed the user’s question and retrieve the top five to ten matches. Test different values rather than assuming that a larger result set is better; too much context can distract the model and increase cost.

For higher accuracy, combine vector search with keyword search. A reranker can then reorder the initial results. Always inspect retrieved chunks during development. If the right answer is not being retrieved, changing the prompt will not solve the underlying problem.

Step 3: Ask Claude or GPT to answer

Build a prompt with three clear sections: system instructions, retrieved context, and the user’s question. Tell the model to answer only from the supplied context, say when the information is insufficient, and cite sources using the metadata you stored.

A practical request uses a low temperature for factual support answers and includes a reasonable output limit. Keep the retrieved text clearly separated from instructions because documents may contain malicious text that attempts to override your system prompt. Treat retrieved content as untrusted data.

You can send the generation request through an OpenAI-compatible client by setting the base URL to https://api.59api.com and selecting the Claude or GPT model available to your account. Keep the API key in an environment variable, never in browser code or a committed repository. This makes it easy to compare Claude Opus, Sonnet, Haiku, Fable, and GPT options without rewriting your application’s model layer.

Evaluate quality before launch

Create a test set of real questions with expected source documents and acceptable answers. Measure retrieval recall, citation accuracy, answer faithfulness, latency, and cost. Log the query, retrieved chunk IDs, model, token usage, and final response, while removing personal or confidential data from logs.

Use access controls for private documents, validate uploaded files, and set limits on file size and request frequency. Streaming responses can improve perceived speed, but they do not replace timeouts and retries.

Why use 59API for the model layer?

RAG often involves many development requests, so model pricing matters. 59API provides cheap, pay-as-you-go access to native official-quality Claude and GPT models rather than downgraded substitutes. Its OpenAI SDK compatibility also simplifies switching models, while compatibility with Claude Code and Codex can fit naturally into an existing workflow. If you want to reduce testing costs, you can sign up for 59API and try the relay, with a referral rebate available as an additional benefit.

Precios · Docs