Token Counter

Step-by-step guide

1. 1
   Paste your text or prompt
   Type or paste any text: a prompt, system message, document, or conversation: into the Text Input panel. The token count updates in real time as you type.
2. 2
   Read the token estimate
   The large Token (est.) counter at the top shows the approximate token count using a BPE-style heuristic. Character, word, and line counts are also shown.
3. 3
   Select models to compare
   Toggle individual model chips to choose which models appear in the comparison view. You can also click 'Show all models' to see all 12 supported models at once.
4. 4
   Check context window usage
   Each model card shows a progress bar indicating what percentage of that model's context window your text fills. Red means over 90% full: a prompt that may be truncated.
5. 5
   Review cost estimates
   The cost table at the bottom shows estimated input costs at current token count across all models. Useful for budgeting API usage before running large batches.

Frequently Asked Questions

How accurate is the token count?

The token count is an approximation using a BPE-style (Byte Pair Encoding) heuristic. It is typically accurate within ±5% for English text. Exact token counts vary by model: each model family uses its own tokenizer. For precise counts, use the official tokenizer for your specific model.

Which models are supported?

The tool covers 12 models across 5 providers: GPT-4o, GPT-4 Turbo, GPT-3.5 Turbo (OpenAI); Claude 3.5 Sonnet, Claude 3 Opus, Claude 3 Haiku (Anthropic); Gemini 1.5 Pro, Gemini 1.5 Flash (Google); Llama 3.1 405B and 70B (Meta); and Mistral Large and Mixtral 8x7B.

What is a context window?

A context window is the maximum number of tokens an LLM can process in a single request: including your system prompt, conversation history, and the response it generates. Exceeding the context window causes the model to truncate older content, which can lead to incoherent responses.

How are input costs calculated?

Input costs are calculated as (token count / 1000) × (price per 1K input tokens). The prices shown are approximate and based on publicly listed API pricing. Always check your provider's current pricing page for exact rates before budgeting.

What is tokenization?

Tokenization is the process of splitting text into tokens: the basic units an LLM processes. Tokens are not the same as words. A single word like 'tokenization' may be split into multiple tokens, while short common words like 'the' are typically a single token. English text averages roughly 0.75 words per token.

Why does Claude have a larger context window than GPT-4?

Claude 3.5 Sonnet and other Claude models support a 200,000-token context window, compared to 128,000 for GPT-4o. Gemini 1.5 Pro and Flash support up to 1,000,000 tokens. Larger context windows allow processing of entire books, codebases, or long conversation histories.

How do I reduce token count to stay within a model's limit?

Common strategies include: removing redundant context from conversation history, using the Stopwords Remover tool to strip filler words from prompts, summarizing earlier conversation turns, and being more concise in system prompt instructions.

Can I use this to estimate costs before an API call?

Yes. Paste your full prompt (including system message and any context) to get an estimated input token count, then multiply by your model's per-token rate to estimate the cost. The built-in cost table does this automatically for all supported models.

Do different languages use more tokens?

Yes. Non-English languages, especially those with non-Latin scripts like Chinese, Japanese, Arabic, and Korean, typically use more tokens per word or character than English. This means the same content in a non-English language will consume more context window space.