Definition
Token is a token is the small chunk of text, often a word or part of a word, that an AI model reads and writes, and the unit that limits and prices most AI usage.
Language models do not read letters or whole sentences. Before your text reaches the model, a tokenizer cuts it into pieces called tokens. A token can be a whole common word like "the", part of a longer word like "un" plus "believ" plus "able", a single punctuation mark, or a few characters of code.
Think of tokens as the Lego bricks of text. The model never sees your sentence as one object. It sees a row of bricks, each with an ID number, and it predicts the next brick, then the next.
It depends on the tokenizer and the language, so any rule of thumb is rough. In ordinary English, a token is often around three quarters of a word, which means 100 words come to roughly 130 tokens. Code, numbers, and languages other than English usually split into more tokens for the same meaning. If you need an exact count, use the tokenizer or token counter your model provider publishes, because different model families count differently.
Tokens are the unit almost everything else is measured in.
A practical example: pasting a 50-page report into a chat spends a large share of your token budget before the model writes a word. Trimming what you send is the easiest way to cut cost and keep the model focused.
A token is not a word, and it is not a character. It also has nothing to do with the login tokens used for security, or with cryptocurrency. Same word, different idea.
Tokens are the currency of inference, the work of running a trained model to get an answer. Models built as a mixture of experts still count tokens the same way, even though only part of the model works on each one.
When someone quotes an AI price, a limit, or a speed, ask what it is per token and which kind of token, input or output. That one question explains most AI bills.
In ordinary English, roughly 750 words, though it varies by tokenizer and language. Code and non-English text usually use more tokens per word.
Tokens measure the actual work a model does, so pricing per token scales with how much text you send and how much it writes back.
OpenAI said Monday it will add an invisible watermark to eligible ChatGPT and Codex text generated in the EU over the coming weeks, and API customers anywhere can now opt in. It is a compliance signal, not a lie detector, and OpenAI says so itself.
Mistral opened a public preview of Mistral Large 4 on Tuesday, a mixture-of-experts model with about 1 trillion parameters and 49 billion active, and promised weights by the end of October. The API is a preview. The weights are the launch.
Reflection AI introduced Beam on Monday, a 501-billion-parameter mixture-of-experts model with 23 billion active, and promised Apache 2.0 weights later in October. Until the weights ship, it is a waitlist and a claim about efficiency.
Google released Nano Banana 2.1 on October 6, an update to its Flash-tier image model that, by Google's own evaluations, holds multi-character scenes together far better. If you make images with people in them or infographics with real text, test it this week.
OpenAI put its Decisions API in public beta on October 6, with gpt-6-luna at $0.10 per million input tokens and no charge for output. If you classify, route or grade at volume through a chat model, this is the endpoint to price against.