← Voltar aos guias

Fast Autocomplete Models: A Developer’s Quick Start

Start with small models, not flagship models

The fastest models for autocomplete and inline suggestions are usually small, low-latency models—not the strongest reasoning models. Completing a function call or adding two lines rarely needs heavyweight reasoning. Your goal is a useful suggestion before the developer types past it.

Start by testing GPT-4.1 nano, GPT-4.1 mini, and a currently available Claude Haiku model, wherever your provider lists them. These are candidates, not a universal speed ranking: routing, context length, load, and output length all affect latency. Use Sonnet or another stronger model as a fallback for difficult completions rather than every keystroke.

1. Define a small, measurable completion task

For your first implementation, generate only the next one to three lines. Avoid explanations, Markdown fences, and whole-file rewrites. A narrow task reduces output tokens and makes suggestions easier to accept.

• Trigger: Request a completion after roughly 150–250 milliseconds without typing, then tune using real usage.
• Context: Send the language, file path, nearby imports, roughly 30–80 preceding lines, and 10–30 following lines.
• Output: Begin with a 64-token limit. Increase it only if useful suggestions regularly get truncated.
• Cancellation: Abort stale requests whenever the cursor or document changes.

Do not send the entire repository by default. Include relevant type definitions or function signatures when needed, and exclude credentials, environment files, and unrelated source files.

2. Connect through 59API

59API gives developers low-cost, pay-as-you-go access to native Claude and GPT models without model downgrades. Its compatibility with OpenAI SDKs, Claude Code, and Codex makes it convenient for testing several model families without building separate billing integrations.

The API base URL is https://api.59api.com. In an OpenAI SDK, configure the base URL using the compatible endpoint shown in your 59API dashboard; for a standard versioned OpenAI route, that is https://api.59api.com/v1. Store your key in an environment variable such as API_59_KEY, never in extension source code.

Use the dashboard’s exact model identifier rather than guessing an alias. For an OpenAI-compatible chat-completions request, start with these fields: {"model":"MODEL_ID_FROM_DASHBOARD","messages":[{"role":"system","content":"Complete the code at CURSOR. Return only the inserted code, without explanations or Markdown. Do not repeat surrounding code."},{"role":"user","content":"Language: TypeScript\nBefore cursor:\nconst doubled = values.map(\nCURSOR\nAfter cursor:\n);"}],"stream":true,"max_tokens":64}. Check the selected model’s supported parameters; some models require a different output-token limit field.

3. Preserve both sides of the cursor

Inline suggestions must fit existing code, not merely continue a conversation. Always label the code before and after the cursor. In the example above, the model should insert something like value => value * 2 without repeating the closing parenthesis.

If an endpoint explicitly supports fill-in-the-middle completion, test its native prefix-and-suffix format. Otherwise, clearly labeled chat context is a practical starting point. Do not assume every Claude or GPT endpoint accepts dedicated suffix parameters.

Before displaying a suggestion, remove obvious overlap with following text and reject responses containing explanations. Associate each request with a document version so delayed responses cannot insert code into a newer buffer.

4. Measure visible latency, not marketing claims

Benchmark each candidate on the same 100–200 cursor positions from representative, non-sensitive code. Reuse identical prompts and output limits. Record:

• Time to first token: How quickly the stream begins.
• Time to usable suggestion: When enough valid code exists to display.
• P50 and P95 latency: Typical and slow-tail performance.
• Acceptance rate: Suggestions accepted and retained after subsequent edits.
• Cost per accepted suggestion: Total request cost divided by retained acceptances.

Streaming improves responsiveness, but does not guarantee faster complete output. Buffer enough text to avoid flashing broken fragments. Benchmark from your actual deployment region, since network and relay routing contribute to latency.

5. Choose the cheapest model that stays useful

Pick the lowest-cost candidate that meets your latency and acceptance targets. Escalate selectively when broader context or harder logic warrants a stronger model; sequential fallback adds delay, so avoid it on every request.

59API’s inexpensive pay-as-you-go pricing suits frequent, short autocomplete calls, and its referral rebate can provide additional savings. Sign up when you are ready to benchmark your shortlist, then choose using measured cost per accepted suggestion—not model reputation alone.

Preços · Docs