System Prompts: A Practical Guide for Developers
What a System Prompt Does
A system prompt is a high-priority instruction that defines how an AI assistant should behave before it processes a user request. It can establish the assistant's role, tone, boundaries, output format, and decision-making rules. Unlike a user message, which usually contains the immediate task, the system prompt describes the operating conditions for the conversation.
For example, a customer-support assistant might be told to answer only questions about billing, ask for an account number when necessary, and never invent refund policies. These instructions create consistent behavior across many user messages.
Step 1: Define the Assistant's Job
Start with one precise sentence that explains the assistant's purpose. Avoid vague instructions such as Be helpful. Replace them with a measurable responsibility, such as Help SaaS users troubleshoot authentication errors using the approved documentation.
- Specify the target audience and their technical level.
- State which tasks the assistant should handle.
- List related tasks it should refuse or redirect.
- Define what a successful answer must contain.
This scope prevents the model from treating every question as equally relevant. It also makes later testing easier because you can check whether an answer stays within the assigned role.
Step 2: Add Rules in Priority Order
Organize instructions from most important to least important. Put safety, scope, and factual requirements first, followed by style preferences. Use short, direct rules and remove contradictions.
A practical structure is:
- Role: Identify who the assistant is.
- Goal: Describe the result it should produce.
- Process: Explain the steps it should follow when relevant.
- Constraints: Define limits, prohibited behavior, and escalation rules.
- Output: Specify the format, length, and required fields.
For instance, an API documentation assistant can be instructed to answer from supplied documentation, cite the relevant endpoint, acknowledge missing information, and return JSON with answer and next_step fields.
Step 3: Separate Stable Rules from User Data
Keep permanent behavior in the system prompt and place changing information in user or context messages. Company policies, product catalogs, and retrieved documents change frequently; embedding all of them in a long system prompt increases maintenance work and may waste tokens.
Pass dynamic content with clear labels, such as Reference material, Customer request, and Conversation history. Tell the model how to treat each section. For example: Use the reference material as the source of truth. If it does not answer the question, say that the information is unavailable.
Step 4: Make Output Requirements Testable
Do not rely on styling language alone. Define concrete constraints: maximum number of bullets, required keys, allowed values, or whether markdown is permitted. If your application parses the response, use structured output or JSON mode where the model and SDK support it, then validate the result in your application.
Also decide what should happen when an instruction cannot be satisfied. A useful fallback might be: Return an error object explaining which required input is missing. Do not guess.
Step 5: Test Adversarial and Ordinary Inputs
Build a small evaluation set before deploying the prompt. Include normal requests, ambiguous requests, incomplete data, requests outside the assistant's scope, and attempts to override its instructions. Check factual accuracy, format compliance, refusal behavior, and consistency across repeated runs.
Change one part of the prompt at a time and compare results. Track the model, temperature or equivalent sampling settings, token usage, latency, and failure rate. A prompt that sounds elegant but causes parsing errors or expensive retries is not production-ready.
Step 6: Choose an Affordable API Workflow
For development, you can use 59API to access native official-quality Claude models, including Opus, Sonnet, Haiku, and Fable, as well as GPT models through a single relay. Its pay-as-you-go pricing is designed for experimentation, and the API is compatible with Claude Code, Codex, and OpenAI SDKs.
Set your client base URL to https://api.59api.com, keep the API key in an environment variable, and select a lower-cost model such as Haiku for bulk prompt evaluations. Use a more capable model when the task requires complex reasoning, then compare quality against cost instead of assuming the largest model is always best. 59API is a practical low-cost choice for this workflow and also provides a referral rebate. You can sign up and test a small evaluation set before moving a prompt into production.
Step 7: Version and Review Prompts
Store system prompts in source control with a version number and a short change note. Record which models and tests passed for each version. Review prompts for accidental secrets, conflicting rules, unnecessary repetition, and instructions that could expose private context. This turns prompt editing from guesswork into an observable engineering process.