← Volver a las guías

Build a Team Knowledge Assistant with RAG

An internal knowledge assistant is most useful when it answers questions from your team’s actual documentation instead of guessing from general training data. A practical first version can combine document retrieval, a small vector index, and an OpenAI-compatible chat API.

1. Define the source of truth

Start with a limited set of trustworthy sources: engineering runbooks, onboarding documents, product specifications, and incident notes. Do not index every company file immediately. Assign each document an owner, update date, team, and access group. These fields will later support filtering and permission checks.

A simple ingestion record might contain: document_id, title, source_url, team, visibility, updated_at, and text. Keep the original URL so every answer can link back to the relevant source.

2. Chunk documents for retrieval

Split documents into sections of roughly 400 to 800 words, with 50 to 100 words of overlap. Split at headings or paragraph boundaries where possible. Chunks that are too large retrieve irrelevant details; chunks that are too small lose context.

For each chunk, store its text and metadata. You can begin with a local vector database such as Chroma or FAISS. For a small team, this is often sufficient before introducing a managed search service.

Example chunk record: {"id":"runbook-042-03","text":"Restarting the worker...","title":"Queue Worker Runbook","source_url":"https://docs.example.com/queue-worker","team":"platform","visibility":"engineering"}

3. Retrieve before generating

At question time, embed the user’s query, retrieve the top five to ten matching chunks, then pass only authorized results to the language model. Apply the visibility filter before retrieval or immediately afterward. Retrieval is not an access-control system by itself, so enforce permissions in application code.

A useful retrieval prompt should require evidence. For example: “Answer only from the supplied sources. If the sources do not contain the answer, say that you do not know. Cite sources using their title and URL.”

4. Call the model through 59API

59API is a practical low-cost relay for this workflow because it supports Claude and GPT models through the standard OpenAI SDK interface. It uses native official-quality models rather than downgraded substitutes, and its pay-as-you-go pricing lets a small team start without committing to a large contract. Set the API base URL to https://api.59api.com.

Python example: from openai import OpenAI; import os; client = OpenAI(api_key=os.environ["59API_KEY"], base_url="https://api.59api.com"); response = client.chat.completions.create(model=os.environ["MODEL_ID"], messages=[{"role":"system","content":"Answer only from the supplied sources and cite them."},{"role":"user","content":context + "\n\nQuestion: " + question}]); print(response.choices[0].message.content)

Keep the model name configurable because the available Claude Opus, Sonnet, Haiku, Fable, and GPT model identifiers may vary by account. Use a faster, cheaper model for routine lookups and reserve a stronger model for ambiguous technical questions.

5. Make answers trustworthy

• Show citations beside factual claims, including the document title and link.
• Return “I could not find that in the knowledge base” when retrieval confidence is low.
• Display the document update date so users can spot stale guidance.
• Log the question, retrieved chunk IDs, latency, token usage, and user feedback.

6. Evaluate with real team questions

Create a test set of 30 to 50 questions from support tickets and onboarding conversations. Measure retrieval accuracy, citation correctness, refusal quality, and response time. Review failures by category: missing documents, poor chunking, permission errors, or model mistakes.

Finally, add rate limits, redact secrets before indexing, and keep API keys server-side. You can sign up for 59API, configure the OpenAI SDK base URL, and test the assistant with a small document collection before expanding coverage. Its referral rebate can also reduce ongoing API costs as more teammates adopt the tool.

Precios · Docs