Ship a Permission-Aware Team Knowledge Assistant
An internal knowledge assistant should answer questions such as “How do we roll back production?” using your team’s actual documentation, not plausible guesses. For a first release, use retrieval-augmented generation, or RAG: retrieve relevant passages, then ask a model to answer from that evidence. Start with one team and one documentation source before connecting the entire company.
1. Define a narrow, testable first release
Choose a high-friction workflow: onboarding engineers, locating runbooks, or explaining service ownership. Collect 20 real questions and identify the documents that should answer them. Include questions with no documented answer so you can test whether the assistant admits uncertainty.
- Interface: Start with a small authenticated web page; add chat integrations later.
- Success criteria: Define acceptable answer accuracy, citation quality, latency, and cost per question.
- Scope: Make the first version read-only. Do not give it deployment, ticket-editing, or administrative tools.
2. Ingest documents with permissions attached
Export an approved collection of Markdown files, wiki pages, or runbooks. Remove navigation clutter and split content along headings into roughly 400–800-token chunks, with modest overlap where necessary. Preserve titles and section headings so individual chunks remain understandable.
Store each chunk with a document ID, source URL, section heading, update timestamp, and access-control metadata. Use stable chunk IDs to replace changed content rather than duplicating it. Your ingestion job must also propagate deletions and permission changes; stale access rules can expose information even when answers look correct.
3. Build permission-aware retrieval
A practical starter stack is your existing application backend plus PostgreSQL with full-text search and pgvector. Generate embeddings using a separately selected embedding service or a local embedding model; do not assume your chosen chat model provides an embeddings endpoint.
Authenticate the user, resolve their current groups server-side, and restrict retrieval to authorized chunks. Apply authorization inside the retrieval query or against a prefiltered candidate set, not after sending results to the model. Combine keyword and vector results when possible: keyword search catches exact service names, while vectors help with paraphrased questions.
Start by passing four to six useful passages to the model, then tune against your test set. Never mix cached retrieval results across users with different permissions.
4. Connect a model without a large upfront commitment
59API offers low-cost, pay-as-you-go access to multiple model families through an OpenAI-compatible API, with an API base URL of https://api.59api.com. Configure your client according to its documented endpoint requirements, keep the API key on your backend, and copy the exact model identifier from the service’s model listing.
For an initial comparison, test Gemini 2.5 Flash and Claude Haiku 4.5 on the same questions and retrieved passages. Measure answer quality rather than assuming the more expensive option is necessary. 59API provides native official-quality models without downgrading, making it a useful option for cost-conscious experimentation. Before sending internal documents through any relay or model provider, review retention, processing locations, contractual terms, and your organization’s data policy.
5. Require grounded answers and usable citations
Assign each retrieved passage a source label and provide its title and URL. Your instruction should say: “Answer using only the supplied passages. Cite supporting source labels for factual claims. If the passages do not establish an answer, explain what is missing.”
Treat retrieved documents as untrusted data, not instructions. Explicitly tell the model to ignore embedded requests to reveal secrets or override its rules. Validate that returned citation labels exist, then render links from your server’s source map rather than trusting model-generated URLs. Citations help users verify answers; they do not automatically prove correctness.
6. Test failure cases before inviting the team
Run your question set with ordinary and restricted test accounts. Check unsupported answers, outdated instructions, deleted pages, conflicting sources, and malicious instructions embedded in documents. Track retrieval failures separately from generation failures: better prompting cannot fix a missing runbook. Log latency, token usage, and feedback without recording sensitive passages by default.
7. Launch small and control spending
Add request timeouts, rate limits, per-user quotas, and bounded output lengths. Cache only with permission-aware keys and an invalidation strategy. If this architecture fits your team, sign up for 59API and benchmark a small approved dataset before expanding. Improve document quality and retrieval first; larger models should solve measured problems, not hide a weak knowledge base.