Understanding Tokens: How AI Models Actually 'Read' Your Text
September 17, 2026
Every large language model's pricing, context window and even some of its odd behavior around spelling or math trace back to one concept: the token. Tokens aren't words, aren't characters, and aren't quite syllables — they're a model-specific compression scheme, and understanding how they work explains a surprising amount of AI behavior that otherwise looks arbitrary.
What exactly is a token, if it's not a word?
A token is a chunk of text — sometimes a whole common word, sometimes a fragment of a word, sometimes a single character — determined by a tokenizer that was trained to represent the most frequent chunks of text in its training data as efficiently as possible. Common English words like 'the' or 'cat' are usually a single token; less common or compound words often split into two or three (e.g. 'tokenization' might become 'token' + 'ization'). As a rough rule of thumb for English, one token is commonly estimated at about 4 characters or roughly ¾ of a word, but this varies by model and is only ever an approximation.
Why does AI API pricing get quoted per token instead of per word or per request?
Because tokens, not words, are literally the unit the model processes internally — every token fed in and generated out consumes computation, so it's the most direct and fair unit to bill against. Pricing per token also lets providers charge differently for input versus output (generation is typically more compute-intensive than reading), which is why most AI API pricing lists a separate input-token rate and output-token rate rather than one flat per-request price.
Why do some languages cost noticeably more per sentence than English?
Because most major tokenizers were trained on datasets dominated by English text, English words are disproportionately likely to map to single, efficient tokens. Languages with different scripts or morphology — many CJK languages, or languages with extensive word compounding — often tokenize less efficiently, sometimes needing two, three or more tokens to represent what would be a single token's worth of meaning in English. The practical effect: the same sentence, translated, can cost meaningfully more in tokens (and therefore in API price) in one language than another, purely as an artifact of how the tokenizer was trained, not the content's actual complexity.
What does a model's 'context window' actually limit?
The context window is the maximum number of tokens a model can consider at once — combining your input (system prompt, conversation history, any documents provided) plus the response it generates. It's not a limit on 'how much you can ask' in isolation; it's a shared budget across everything the model has to read and everything it produces in that single exchange. A long conversation history or a large pasted document can consume most of the context window before the model has generated a single word of response, which is why very long conversations sometimes lose track of details from early on — they've simply aged out of the token budget.