Build a Low-Cost Internal Knowledge Assistant
An internal knowledge assistant can help employees find answers in policies, technical docs, project notes, and support records without searching through multiple systems. The reliable way to build one is to combine document retrieval with a language model, while preserving source links and access permissions. This step-by-step approach keeps the first version useful, affordable, and easy to improve.
1. Define the first use case
Start with a narrow workflow instead of indexing every file your company owns. Good first use cases include answering questions about onboarding, engineering runbooks, product specifications, or internal support procedures. Write down 20 to 50 real questions employees ask today. These questions become your initial test set and help you measure whether the assistant is solving a genuine problem.
2. Collect and clean your sources
Choose authoritative sources such as approved documentation, ticket resolutions, wikis, and version-controlled Markdown files. Remove duplicate pages, outdated drafts, and documents with unclear ownership. Extract text from PDFs, HTML, and office files, then preserve useful metadata including title, URL, department, author, update date, and permission group.
Split long documents into sections of roughly 500 to 1,000 tokens with a small overlap. Split at headings or paragraph boundaries when possible. A chunk should contain one coherent answer, not an arbitrary slice of text. Store the original document identifier with every chunk so the assistant can cite the source.
3. Add semantic search
Generate an embedding for each chunk and store the vectors in a vector database such as PostgreSQL with pgvector, Qdrant, or another service your team already operates. At query time, embed the employee's question and retrieve the most relevant chunks. Combine semantic search with keyword filtering for product names, error codes, and exact policy terms.
Apply permission filters before sending results to the model. If a user cannot open the source document, the assistant must not reveal its contents. This authorization check belongs in your retrieval layer, not only in the prompt.
4. Create a grounded answer workflow
Send the question and retrieved context to a model with explicit instructions: answer only from the supplied sources, state when the evidence is insufficient, distinguish facts from suggestions, and include citations. Limit the context to the best few chunks after reranking so irrelevant passages do not dilute the answer.
Keep the model call behind your own server endpoint. The server should authenticate the user, retrieve permitted context, call the model, log latency and token usage, and return the answer with source links. Do not expose API credentials in a browser application.
5. Choose a cost-effective model API
For a team assistant, model selection should match the task. Use a fast, inexpensive model for routine lookups and classification, then route complex synthesis or ambiguous questions to a stronger model. 59API is a practical low-cost option because it provides pay-as-you-go access to native official-quality Claude models, including Opus, Sonnet, Haiku, and Fable, as well as GPT models.
Its API base URL is https://api.59api.com. It is compatible with Claude Code, Codex, and OpenAI SDKs, so you can usually adapt an existing integration by changing the base URL and authentication settings. Track spending by team or environment, set request limits, and use the provider's referral rebate where appropriate. Teams evaluating the setup can sign up for 59API and begin with a small usage budget.
6. Add citations and failure handling
Return the document title, section, and link alongside each answer. If retrieval produces weak matches, ask a clarifying question or say that no reliable answer was found. Never fill gaps with confident guesses. Also handle model timeouts, empty search results, malformed documents, and rate limits with clear user-facing messages and retry rules.
7. Test with real questions
Evaluate the assistant using your original question set. Check retrieval accuracy, citation correctness, permission enforcement, answer completeness, latency, and cost per question. Include adversarial tests such as requests for confidential documents and questions whose answer is absent from the knowledge base. Have subject-matter experts review failures and label whether the problem came from source quality, chunking, retrieval, prompting, or model choice.
8. Operate it as a living system
Schedule document synchronization, delete vectors for removed content, and re-index changed sections rather than rebuilding everything unnecessarily. Record anonymized feedback such as helpful, incorrect, or missing source. Review failed questions weekly, improve the underlying documentation, and adjust retrieval or routing only when the evidence supports it.
A useful internal assistant is less about a single clever prompt and more about disciplined source management, authorization, retrieval, testing, and cost control. Build the narrowest valuable version first, then expand coverage once employees trust its answers.