What Actually Happens When Your Code Talks to an AI Model

The useful part

One model billed 497 output tokens for 283 characters of visible text, because 445 of those tokens were invisible reasoning. The same question cost 170x more on the priciest model than the cheapest, in the same minute. Temperature changes how surprising the wording is, not how smart the answer is, and a caching layer can make it look like it does nothing at all.

How it works

• You have no idea why it works, what you just spent, or what any of the settings in those six lines actually do.
• So you change nothing, because changing things you do not understand is how you get paged at midnight.
• Which model to use, why your bill tripled, why the same script that worked on Tuesday returns garbage on Friday when someone pastes in a bigger document, why one team's feature costs a hundred times more...
• If you are working through examples on a laptop, prefer a key with a hard spending ceiling on it over one wired to an open-ended account, so a mistake costs you an afternoon instead of a mortgage payment.
• They generate a private chain of internal reasoning, discard it, and show you only the final answer.

What to take from it

That gap is worth closing, because almost every decision you will make later depends on it. The model lives on someone else's hardware, and you are renting a few seconds of its attention. Four things in that loop are worth understanding properly, because they are the four things that will confuse you later: the key, the tokens, the context window, and the settings.

Example or evidence

• Considerably more: across Anthropic's published price list, every single model charges exactly five times as much per output token as per input token.
• Different labs build different tokenizers, and some providers also wrap your message in their own scaffolding before it reaches the model, which inflates the count further.
• All of it comes back to one loop: a request goes out, some text gets counted, a model reads it and writes back, and a meter moves.
• Here is the whole thing, explained the way we wish someone had explained it on day one.

Details worth keeping

Context window is working memory for one request, not knowledge. Go over it and you get a hard error, not a polite summary. Ends with: you making one call yourself and reading the receipt.

Related coverage

• Kodekloud: How does ChatGPT search the web when the model can't open a connection? Tool calling explained: the model asks, the application runs the search.
• Kodekloud: A language model cannot check your disk usage or query your database on its own. Function calling is the mechanism that changes that, and it takes about forty lines of Python to build.
• Dzone: When teams first integrate large language models (LLMs) into their software platforms, the initial experience often feels surprisingly simple.
• Dev: Part 2 of a series on building an AI-native platform.