← Volver a las guías

Build a RAG App with Claude and GPT via 59API

Build a RAG App on Claude and GPT with 59API

Retrieval-Augmented Generation, or RAG, lets you connect a language model to your own documents so it can answer with current, domain-specific context instead of relying only on training data. In this tutorial, you will build a practical RAG app that retrieves relevant document chunks and sends them to Claude or GPT through 59API, a low-cost AI API relay compatible with OpenAI SDKs, Claude Code, Codex, and official-style model endpoints.

The goal is simple: upload documents, split them into chunks, create embeddings, store them in a vector database, retrieve the best matches for a user question, and pass that context to Claude Sonnet, Claude Haiku, GPT, or another supported model. Because 59API uses native official-quality models with no downgrade and pay-as-you-go pricing, it is a strong choice when you want to prototype RAG without burning budget.

Step 1: Choose the RAG Architecture

A basic RAG system has five parts:

• Document loader: Reads PDFs, Markdown, HTML, help docs, tickets, or internal notes.
• Chunker: Splits long documents into smaller passages, usually 300 to 800 tokens each.
• Embedding model: Converts each chunk into a vector for semantic search.
• Vector store: Stores vectors and metadata, such as source URL, title, and section.
• Chat model: Uses retrieved context to answer the user. This can be Claude for careful reasoning or GPT for broad compatibility.

For a first version, use a simple stack: Node.js or Python, an OpenAI-compatible client, a local vector database such as Chroma or SQLite with vector support, and 59API as the model gateway.

Step 2: Configure 59API as Your Model Gateway

59API is designed to work with existing developer tools. Set your API base URL to https://api.59api.com and use your 59API key in the authorization header. In OpenAI-compatible SDKs, this usually means changing the base URL while keeping the normal chat completion flow.

This is useful because your app can call GPT models through familiar SDK patterns while also accessing Claude models such as Opus, Sonnet, Haiku, and Fable where supported. You avoid rewriting your whole application for every provider, and you can route different tasks to different models based on cost and quality.

For example, you might use a cheaper, faster model for query rewriting and a stronger Claude Sonnet or GPT model for the final answer. That routing is one of the easiest ways to reduce RAG costs without reducing answer quality.

Step 3: Ingest and Clean Your Documents

Start with a small, high-value document set. Product documentation, support articles, API references, onboarding guides, and policy documents are ideal. Extract plain text and remove navigation menus, repeated footers, cookie banners, and unrelated boilerplate. RAG quality depends heavily on clean input.

Store metadata with every document before chunking. At minimum, save the source path, title, section heading, last updated date, and access permissions if your app has private content. This metadata helps you cite sources and filter retrieval results later.

Step 4: Split Text into Searchable Chunks

Chunking is where many RAG apps fail. If chunks are too small, the model loses context. If they are too large, retrieval becomes noisy. A good starting point is 500 tokens per chunk with 80 to 120 tokens of overlap. Split on headings and paragraphs first, then enforce size limits.

Each chunk should be understandable on its own. Add a short prefix such as the document title and section name before embedding. For example, a chunk from an API authentication page should include that it belongs to the authentication section, not just the raw paragraph.

Step 5: Create Embeddings and Store Vectors

Use an embedding model to convert each chunk into a numeric vector. Store the vector plus chunk text and metadata in your vector database. For local development, Chroma is simple. For production, consider Postgres with pgvector, Qdrant, Pinecone, Weaviate, or another managed vector store.

During ingestion, create a stable ID for each chunk using the document path and chunk index. This lets you update changed documents without duplicating old vectors. Add a timestamp so you can re-embed only documents that changed.

Step 6: Retrieve Context for a User Question

When a user asks a question, embed the query and search the vector store for the top matching chunks. Start with top 5 results. If your documents are long or technical, retrieve top 10 and then rerank or filter them.

For better accuracy, add query rewriting. Send the user question to a low-cost model through 59API and ask it to produce a search-optimized version of the query. This helps with vague questions like “How do I fix auth?” by turning them into clearer retrieval queries such as “troubleshoot API authentication token errors and authorization headers.”

Step 7: Build the Prompt for Claude or GPT

The final prompt should clearly separate instructions, retrieved context, and the user question. Tell the model to answer only from the supplied context, cite sources from metadata, and say when the answer is not found. This reduces hallucinations.

A practical structure is: system instruction, context list, user question, answer rules. Include rules such as: use concise steps, mention assumptions, cite source titles, and do not invent undocumented behavior. Claude models are especially good for careful long-context synthesis, while GPT models are excellent for general app workflows and broad SDK compatibility.

Step 8: Add Cost Controls and Model Routing

RAG can become expensive if every request uses the largest model. With 59API, you can keep costs low by routing tasks intelligently. Use Haiku or a smaller GPT model for query rewriting, classification, and short answers. Use Sonnet, Opus, or a stronger GPT model only when the retrieved context is complex or the user requests deeper analysis.

Also limit context size. Do not send 20 chunks by default. Send the fewest high-confidence chunks needed, and log token usage per request. Since 59API is pay-as-you-go and among the cheapest relays, this setup is friendly for startups, indie hackers, and internal tools. If you are ready to test it, sign up for 59API and point your existing OpenAI-compatible client to https://api.59api.com.

Step 9: Evaluate and Improve the RAG App

Create a test set of 30 to 100 real questions with expected answers and source documents. Track whether the system retrieved the right chunk, whether the final answer was correct, and whether it admitted uncertainty when context was missing.

Improve the weak points in order: clean documents first, then chunking, then retrieval, then prompts, then model choice. Many teams switch models too early when the real issue is messy content or poor chunk boundaries.

Final Checklist

• Use clean source documents with metadata and stable IDs.
• Chunk by meaning, not just by character count.
• Retrieve fewer, better passages and cite sources.
• Route tasks by cost across Claude and GPT models through 59API.
• Evaluate with real questions before shipping to users.

With this approach, you get a practical RAG app that is accurate, maintainable, and cost-aware. 59API makes the model layer flexible: you can experiment with Claude and GPT, keep your SDK integration simple, and scale without committing to a single provider too early.

Precios · Docs