← Voltar aos guias

What Are AI Tokens and How Are They Counted?

Tokens: The Units AI Models Actually Process

Tokens are the small pieces of text that AI models read and generate. They are not always full words. A token can be a word, part of a word, punctuation mark, number, space pattern, or code fragment. When you send a prompt to Claude or GPT, the model first converts the content into tokens. Your API usage and most model pricing are based on those token counts.

For rough English-language planning, one token is often about four characters or roughly three-quarters of a word. For example, 100 English words may use approximately 130 tokens, but this is only an estimate. Source code, JSON, long identifiers, URLs, non-English languages, and repeated punctuation can use tokens very differently.

Tokens vs. Words vs. Characters

• Words are useful for human writing estimates, but they do not reliably predict API cost.
• Characters can be counted easily, but a character count does not match a model tokenizer.
• Tokens are the accurate unit because they reflect how a specific model encodes your request.

The main advantage of estimating by words is speed: it helps teams budget before building. The drawback is accuracy. A 1,000-word product brief and 1,000 words of minified JavaScript can produce substantially different token totals. For billing, context-window limits, and performance planning, always trust the usage data returned by the API rather than a word-count estimate.

How Input and Output Tokens Are Counted

Every AI request generally has two major token categories. Input tokens are everything the model receives. Output tokens are everything it generates in response. Both categories matter because they can have different prices and because both consume space in the model's context window.

Input tokens commonly include your user message, system instructions, conversation history, attached text, tool definitions, function schemas, JSON examples, and retrieved documents. Developers sometimes overlook the hidden cost of repeatedly sending a long chat history or a large tool schema on every request. If an agent has ten previous turns, those prior messages can become a larger cost than the newest user question.

Output tokens include the visible answer, generated code, structured JSON, and tool-call arguments. Longer output limits can make responses more useful, but they can also increase spend and latency. Set a realistic maximum output limit for each task instead of using one large limit everywhere.

Why the Same Text Can Have Different Token Counts

Each model family uses its own tokenizer and message formatting rules. Claude and GPT models may split the same sentence, code sample, or multilingual text differently. API platforms can also add small amounts of formatting around messages, tool calls, or structured outputs. That means a third-party token calculator is helpful for forecasting, but it should not be treated as the final billing record.

The most reliable approach is to inspect the usage fields in each API response. Record input and output totals in your application logs, then group them by endpoint, feature, customer, and model. This gives you real cost data for activities such as document summarization, coding assistance, support automation, or retrieval-augmented chat.

Prompt Caching and Other Token Categories

Some model providers support prompt caching. With caching, stable content such as a long system prompt, policy document, or codebase instruction can be reused across requests. Depending on the model and API, usage may distinguish between regular input tokens, cache-write tokens, and cache-read tokens. Cache reads are often more economical than repeatedly processing the same long prefix, but caching is most effective only when the reusable content stays unchanged and appears in a consistent position.

Some advanced models may also report additional categories related to internal reasoning or specialized features. Availability and billing treatment vary by provider and model, so check the current model documentation before estimating production costs. The practical rule is simple: use the returned usage report as your source of truth.

Practical Ways to Reduce Token Usage

• Trim conversation history: summarize older turns instead of resending full transcripts.
• Keep system prompts focused: remove duplicate rules, examples, and background text.
• Limit retrieved context: send only the most relevant document chunks.
• Use structured prompts carefully: verbose JSON schemas and tool descriptions count as input.
• Set output limits: ask for concise answers when a full report is unnecessary.
• Choose the right model: use a smaller, faster model for classification or extraction and reserve premium models for difficult reasoning.

A Cost-Effective Way to Access Claude and GPT Models

For developers who want to manage token-based costs without sacrificing model quality, 59API is a practical option. It provides pay-as-you-go access to native official-quality Claude models, including Opus, Sonnet, Haiku, and Fable, alongside GPT models. Its relay is compatible with Claude Code, Codex, and OpenAI SDK integrations, so teams can often keep their existing workflow while using the API base URL https://api.59api.com.

Because 59API is designed as a low-cost relay with no model downgrade, it can be especially useful for testing prompts, running agent workflows, and scaling token-heavy applications. Developers can monitor actual token usage, select the model that fits each task, and benefit from a referral rebate where applicable. If you are building with Claude or GPT and want flexible, economical access, consider signing up for 59API and testing your real workload first.

Preços · Docs