I get asked some version of these two questions all the time:
“What’s a token?” is one of them.
It’s a completely understandable question.
The other question is: “Why am I being limited in my job to the amount of tokens I use when I have no idea what a token is, how many I’m burning, or when I’m out?”
Most people using large language models are operating without the basic instruments we take for granted in almost every other system. So, I've been thinking about an analogy I could use to help explain tokenization. And then I got to thinking about the 1980s when CDs were starting to replace vinyl records. Hopefully, this analogy will work for you.
From vinyl to digital: approximating a continuous curve
An LP record is continuous. The physical groove in the vinyl is a direct analog of the sound wave. The needle rides an unbroken curve of rising and falling pressure, and your ear hears music.
When engineers digitized that sound for CDs, they had to turn the continuous wave into discrete measurements. They sampled the wave 44,100 times per second and recorded the height of the wave at each instant on a scale of 65,536 possible values. From those points, they could reconstruct an approximation of the original curve.
Two decisions controlled the quality of that approximation:
- Sample rate — how frequently they took a measurement.
- Bit depth — how precisely they recorded the height of each sample.
Higher sample rates and higher bit depths produced a closer match to the original continuous wave. The music sounded better, but the file also got larger.
Streaming services like Spotify made the trade-off interactive. You can change the streaming quality on the fly. Higher quality preserves more of the original signal and uses more data. Lower quality throws more of it away and conserves your cellular plan when you're not on Wi-Fi. The mechanism under the hood is compression rather than raw resampling, but the choice you're making is the same one: how much of the original wave to keep, in exchange for cost and bandwidth.
Language faces the same problem
Human language carries continuous structure: meaning, grammar, rhythm, association, and intent flow together. A language model cannot operate directly on that continuous stream. It has to break the text into discrete units it can process. Those units are called tokens.
If we broke language all the way down to individual characters (an extremely high "sample rate"), every sentence would become a very long sequence. A paragraph that fits in 200 tokens today would take roughly a thousand. Computational cost rises sharply, and the relationships that actually matter get stretched across far more units, so the model has to work harder to hold them together.
If we treated every whole word as an indivisible unit (a very low sample rate), the vocabulary would balloon into the millions, and the system would still stumble on any word it had never seen. It would also miss the internal patterns that words share. "Running," "ran," and "runner" would look like unrelated points instead of related points on the same morphological curve.
Modern tokenization sits in the middle as the practical sampling rate. Common words usually stay intact. Frequent prefixes, suffixes, and word pieces become reusable building blocks. The result is a sequence of chunks sized so that the local relationships between them carry useful information, the linguistic equivalent of rise and run. From those discrete pieces, the model can approximate the original continuous structure of meaning well enough to predict what is likely to come next.
Sample too finely and you pay for length you don't need. Sample too coarsely and the reconstruction misses the shape entirely. Tokenization lands in the middle, on both sides of the analogy.
That prediction is the entire game. The model is always estimating the next token. The quality of the tokenization determines how clean and learnable the patterns are, and how much computation (and money) each exchange consumes.
The missing dashboard
Here, the analogy reveals the biggest practical frustration with large language models today.
When you drive a car, you have a gas gauge, an odometer, and a speedometer. You know how much fuel remains, how far you've traveled, and how fast you're burning it.
With most LLM interfaces, you have none of those instruments.
Every prompt you send and every response you receive consumes tokens. The model's context window is a fixed-size tank. Cost is metered per token, whether you're paying an API bill directly or drawing down a subscription's usage allowance. Yet the everyday chat interface rarely shows you:
- how many tokens the current exchange has already used,
- how close you are to the context limit,
- what the burn rate looks like on a longer task, or
- what the cumulative cost will be.
You are effectively driving without a fuel gauge, trip meter, or speedometer.
The measurements aren't missing from the system. They're missing from the interface.
You usually discover the limits only when something breaks. The model begins losing track of earlier parts of the conversation because the context window is full and the oldest material has been dropped or quietly summarized. Or it truncates its reply. Worse, it simply locks you out — "you're out of tokens for the next hour / three hours / the rest of the day" — or asks you for more money to continue. The bill arriving at the end of the month is often the first clear signal of how hard you were actually pushing the system.
Is there a solution today, and what's coming?
The situation is improving, but unevenly.
In developer and API environments, visibility is already much better. Many tools and gateways now surface detailed token counts, cost breakdowns, and usage by model, user, or project. Browser extensions have also appeared that overlay real-time counters, remaining context estimates, and rate-limit countdowns directly onto popular chat interfaces. These are useful stopgaps.
Native consumer chat experiences, however, still largely leave people flying blind. The instruments most users need (a clear gas gauge, an odometer of cumulative use, and a speedometer of burn rate) are not yet standard.
That is likely to change. As more people and organizations rely on these systems for real work, the competitive pressure to provide transparent metering will grow. Real-time remaining context indicators, projected cost, and clearer limit warnings are the kind of features that move from "nice-to-have" to expected. Some platforms are already moving in that direction; others will follow.
Until then, the practical advice is simple: treat tokens as a real resource with real costs and limits. Learn the rough conversion. In English, 1,000 tokens runs about 750 words, or roughly four characters per token. Code, numbers, and non-English text run denser, sometimes far denser, so the same page costs more. Watch for the early signs that the context window is filling. And, when the work matters, use the available extensions or API-level tools that give you the dashboard the default interface still withholds.
Understanding tokens is the difference between using these systems blindly and using them with some sense of the actual road, the fuel in the tank, and how far you still have to go.