A token is the unit an LLM actually reads and writes in, a chunk of text usually shorter than a whole word, and every context window, API price, and generation limit is measured in tokens rather than in characters or words.
Why does an LLM use tokens instead of words?
A model needs a fixed, manageable vocabulary to predict over, and human language has far too many possible words for that to be practical. A tokenizer breaks text into a smaller set of common subword pieces instead: common words often stay whole, rarer words split into a few pieces, and unfamiliar strings like a product code or a typo can fragment into many single characters. That splitting is invisible in a chat interface but directly decides cost, since providers bill per token, not per word.
Where does the token count quietly add up?
Retrieved documents in a RAG system, tool definitions, conversation history, and the model’s own reasoning all consume tokens against the same context window, and a system that reflexively pastes in more text than a question needs pays for every extra token whether it helps the answer or not. Tracking and trimming that spend, rather than assuming a bigger LLM plan absorbs it, is one of the fastest ways to cut a production system’s running cost without touching accuracy.
Frequently asked questions
Is a token the same as a word?
No. A common English word is often one token, but longer or less common words split into several, and code, punctuation, and non-English text tokenize differently. A rough estimate is about four characters per token in English.
Why does the token count matter for cost?
Most LLM providers charge per token, input and output separately, so a longer prompt or a verbose answer costs more directly. It is also the unit the context window is measured in, so token count decides both price and how much fits in a single call.