What Does AI Actually Cost — The Structure of Token Pricing

They say AI is cheap. Until the day you see the invoice.

The "A Few Dollars a Month" Trap

Ask for AI quotes and the numbers are all over the place. One says thirty dollars a month, another says three thousand. Same technology — why the hundredfold gap?

The answer lies not in the price plan but in how it's used. And the unit of that usage is the token.

In How to Read an AI Project Quote we discussed the traps in quotes. This time we look at the real cost structure underneath them.

A Token Is a Meter

A token is the smallest unit AI handles text in. Think of it as a fragment smaller than a word. In English one token is roughly four characters.

Most AI pricing is charged by this token count. Like a taxi meter, the cost rises with the amount of text exchanged. So the honest answer to "How much is it?" is "How much are you going to use?"

Output Costs More Than Input

There are two kinds of tokens.

Input tokens — the text you feed AI (questions, attached documents) Output tokens — the text AI produces (answers)

Usually output costs several times more than input. Producing something "thought up" costs more than reading. So a task like "write me a long report" can cost more than a hundred short questions.

Why Long Documents Cost More

AI has no memory. Every time you talk to it, the needed context must be fed in again, whole. This is called context.

Attach a 100-page contract and ask a question, and those 100 pages count as input tokens for every question. Ten questions means feeding the contract ten times. That's why "the longer the document, the longer the conversation," the more cost snowballs.

Saving: Caching and Narrowing

Fortunately there are ways to cut it.

Caching — let AI briefly "remember" a document you feed repeatedly, so from the second time it's reused at a discount. For work that keeps using the same material, this lowers cost dramatically.

Narrowing — instead of feeding all 100 pages, select only the relevant few. Giving only what's needed is the surest saving.

The Real Risk Is "Variable Cost"

The scary thing about AI cost is that it's not fixed but variable. The better it goes, the more you use it; the more you use it, the higher the cost.

What was fifty dollars a month in the pilot can become five thousand a month after company-wide rollout. More users, each using it longer. That's why, as stressed in Starting AI on a Small Budget, it matters to start small and measure the unit cost.

Three Principles for SMBs

1. Measure with one representative task Run the one task you'll use most for a month. Once you have "cost per case," a simple multiplication reveals your budget.

2. Set usage caps Set monthly limits and per-user limits. It's the safety valve that stops cost from quietly running away.

3. Self-hosting only after you've scaled At very high volume, running your own model can be cheaper. But that break-even point is higher than you'd think. For most SMBs, renting is still cheaper.

Cost Can Be Controlled

Token cost is not mysterious. It's just a meter charged by the amount of text.

Understand the structure and you can manage it. What you feed, how much you receive, how often you call — hold just these three and AI cost becomes not a surprise but a predictable number.