A token is the unit an LLM actually reads and writes in, a chunk of text usually shorter than a whole word, and every context window, API price, and generation limit is measured in tokens rather than in characters or words.
Why does an LLM use tokens instead of words?
A model needs a fixed, manageable vocabulary to predict over, and human language has far too many possible words for that to be practical. A tokenizer breaks text into a smaller set of common subword pieces instead: common words often stay whole, rarer words split into a few pieces, and unfamiliar strings like a product code or a typo can fragment into many single characters. That splitting is invisible in a chat interface but directly decides cost, since providers bill per token, not per word.
Where does the token count quietly add up?
Retrieved documents in a RAG system, tool definitions, conversation history, and the model’s own reasoning all consume tokens against the same context window, and a system that reflexively pastes in more text than a question needs pays for every extra token whether it helps the answer or not. Tracking and trimming that spend, rather than assuming a bigger LLM plan absorbs it, is one of the fastest ways to cut a production system’s running cost without touching accuracy.
This entry was drafted with AI assistance.
Frequently asked questions
Is a token the same as a word?
No. A common English word is often one token, but longer or less common words split into several, and code, punctuation, and non-English text tokenize differently. A rough estimate is about four characters per token in English.
Why does the token count matter for cost?
Most LLM providers charge per token, input and output separately, so a longer prompt or a verbose answer costs more directly. It is also the unit the context window is measured in, so token count decides both price and how much fits in a single call.
No advertising or tracking cookies, and our visitor counts are anonymous. The Cal.com booking widget loads only if you allow it. Privacy Policy.
The page itself, anything our host sets to serve and secure it, and the anonymous visitor count. Always on, and none of it stores anything on your device.
The Cal.com booking widget. Left off, a booking link opens the booking page instead of a popup, so you can still book a call.