Sources and leaks from Amazon, Adobe, Atlassian, Citi, and more show what is really happening with AI right now: companies are trying to rein in AI use as costs spiral out of control.
Such a unit exists and it is also called tokens, that can measure the capability of a model and the size of a running operation in a model.
I think you might have it mixed up with parameters, rather than tokens. Parameters are how big the model is, and are an indirect measure of how capable it is. Bigger models tend to be more capable.
But what they use for calculating your bill is something different today.
The tokenizer varies a little, but I don’t think it’s changed measurably from tokens. You pay an amount for a million tokens worth of processing. The tokeniser difference just alters how text is converted to tokens, but the tokens themselves don’t change all that much.
If anything, I’d honestly put the issue more with reasoning chains in models, where they basically babble to themselves inside of a <think> tag, that most interfaces hide/collapse. It makes them work better, but vastly increases the amount of tokens per operation.
They have been getting longer and more sophisticated with newer models. So you might have a model now that basically repeats the output multiple times whilst refining and drafting the non-reasoning output.
If you’re making it generate a lot, that’ll balloon the usage, and thus price.
I think you might have it mixed up with parameters, rather than tokens. Parameters are how big the model is, and are an indirect measure of how capable it is. Bigger models tend to be more capable.
The tokenizer varies a little, but I don’t think it’s changed measurably from tokens. You pay an amount for a million tokens worth of processing. The tokeniser difference just alters how text is converted to tokens, but the tokens themselves don’t change all that much.
If anything, I’d honestly put the issue more with reasoning chains in models, where they basically babble to themselves inside of a <think> tag, that most interfaces hide/collapse. It makes them work better, but vastly increases the amount of tokens per operation.
They have been getting longer and more sophisticated with newer models. So you might have a model now that basically repeats the output multiple times whilst refining and drafting the non-reasoning output.
If you’re making it generate a lot, that’ll balloon the usage, and thus price.