Why You're Hitting AI Token Limits (And 9 Habits to Fix It)
Learn why Claude and Codex token limits vanish so fast, and the concrete habits that cut reused input and stretch every usage cap further.
Why do AI chat tools run out of tokens so fast?
Because every message you send drags the entire conversation history along with it. Chat-based AI tools have no real memory between turns. Instead, each time you hit enter, the model resends the whole thread from the start: your first message, its reply, your second message, its reply, and so on. That accumulated history is called reused input, and it can make up the vast majority of tokens processed in a session, even though you only typed a small fraction of it. A single heavy day of Codex or Claude usage can move billions of tokens through a workspace, with the overwhelming majority being material the model already saw before.
TL;DR
- Reused input, not your actual typing, is the main driver of token consumption in long chat threads, since the full conversation gets resent on every turn.
- Editing a bad prompt instead of correcting it in a follow-up message avoids paying for the mistake twice.
- Starting a fresh thread when the task changes is one of the single biggest ways to cut token usage, because long conversations carry enormous reused context.
- Carrying forward only the finished output of a multi-step task (not the drafts, rejected sources, and back-and-forth that produced it) keeps the next step lean.
- Output tokens cost repeatedly, not once, because every response becomes part of the input on the next turn, so asking for exactly what you need (a paragraph, five bullets, JSON) pays off across the whole conversation.
- Searching files yourself and sending lightweight text instead of letting the model dig through PDFs or screenshots avoids burning tokens on formatting and search overhead.
- Connected tools carry hidden token costs, since every tool definition (name, description, arguments) gets loaded into the model’s input before it does anything, and a handful of connected servers can add tens of thousands of tokens per request.
Built like a system. Not vibe-coded.
Remy manages the project — every layer architected, not stitched together at the last second.
How does reused input quietly inflate your token bill?
The mechanics are straightforward once you see them. Your first message in a conversation costs roughly what you typed. Your second message costs what you typed, plus the model’s first reply, plus your original message. By the tenth message, you’re paying for nine previous exchanges every single time, even though you only wrote one new line. By the thirtieth message, the material you just typed is a small fraction of the total tokens being processed.
This isn’t a bug and it isn’t something AI labs are likely to fix for you. The business incentive runs the other way: labs want you using their tokens, up to the point where compute constraints force them to throttle usage. That tension means the responsibility for managing consumption sits with the person typing, not the company running the model.
It also explains why failed or unclear responses are so expensive. If an answer comes back wrong and you ask the model to “fix it,” the retry resends the entire flawed exchange again, then the fix on top of it. Each retry compounds the reused-input problem rather than resetting it.
What habits actually reduce token usage day to day?
A handful of practical habits address most of the waste, and none require installing anything.
Edit mistakes instead of correcting them in the next message. If a prompt was unclear and got a bad answer, use the edit function to fix the original message and resend it, rather than adding a new message saying “that’s not what I meant.” Editing removes the bad exchange; a follow-up message keeps it in the history forever.
Batch related questions and specify the output format up front. If you have several questions about the same document, ask them together in one message and state exactly how you want the answer to look: a one-pager, 150 words, just bullet points. This cuts down on the back-and-forth the model would otherwise need to disambiguate what you want.
Start a new thread when the task changes. This is the habit with the biggest measured impact. Long conversations are useful while you’re deep in one problem, but they carry enormous reused context that compounds fast. Modern models can handle disambiguation across a long context window, but that convenience costs tokens. Starting fresh doesn’t delete the old thread; it just stops the new task from dragging that history along.
Carry forward the result, not the process. In multi-step work (research, then drafting, then revision), pass along the finished artifact from one stage into the next, not the full history of drafts, rejected sources, and reasoning that produced it. Handing over a clean research summary is far cheaper than handing over every round of back-and-forth that created it.
Everyone else built a construction worker.
We built the contractor.
One file at a time.
UI, API, database, deploy.
Ask for only the output you need. Output tokens are expensive in a way that’s easy to overlook: they cost once when generated, then again every time they get resent as part of the input on subsequent turns. A request for a five-page summary when a five-bullet answer would do gets billed repeatedly for as long as the conversation continues.
Why does file handling matter so much for token costs?
Two habits here make an outsized difference. First, search files yourself before handing them to the model. Letting an AI search a large document is convenient, but it means the model has to read and process far more than it needs to. Pulling out the relevant snippet yourself and pasting just that section is dramatically cheaper.
Second, send the lightest usable form of a source. If the content of a document matters but the formatting doesn’t, convert it to plain text or markdown before sending it. Pasting a PDF or a stack of screenshots because that’s how the source arrived is a common shortcut, and it works, but it inflates token usage substantially compared to sending clean text.
A related habit for repeated lookups: keep answers and reference data somewhere retrievable, whether that’s a personal database or notes system. If you’re going to need the same figure or fact across multiple conversations, storing it once and retrieving it is far cheaper than making the model recalculate or re-derive it every time.
How do connected tools and integrations add hidden token costs?
Every tool connected to an AI assistant, whether it’s a project tracker, a messaging platform, or a monitoring dashboard, comes with a description: what the tool does, when to use it, what arguments it takes. That description gets loaded as input before the model does anything with your actual request. Anthropic has published findings showing that a typical setup with several connected tool servers can burn roughly 55,000 tokens on tool definitions alone, before any real work happens.
The practical takeaway is to load only the tools a given job actually needs. Connecting everything “just in case” means paying a token tax on every single request, whether or not those tools ever get used.
Is there a limit to how much you can fix by hand?
Yes. For long-running or debugging-style work where restarting a thread isn’t practical, some solutions require relying on the model provider’s own mechanisms. OpenAI supports compaction for long-running tasks, which carries forward a condensed state and reduces token needs without a full restart. Anthropic supports context editing, which clears out old tool results and stale reasoning blocks before the next request goes out. Both help, but both mean you’re working with an approximated version of the original conversation rather than the full history, which is a tradeoff worth understanding rather than assuming is invisible.
Beyond that, habits alone only go so far. Most of the gains come from consistency: editing instead of correcting, starting clean threads, sending lean sources, and asking for precise outputs, applied every time rather than occasionally.
Frequently Asked Questions
What is “reused input” in AI token usage?
Reused input is the portion of every request that consists of conversation history the model has already processed before. Because chat tools resend the full thread on every turn, reused input can account for the large majority of tokens in a long conversation, even though the user only typed a small new message.
Does starting a new chat actually save tokens?
Yes, and it’s typically the single most effective habit. A new thread doesn’t carry forward the accumulated history of a long conversation, which is usually the largest source of reused input. Some baseline system instructions still get sent, but the bulk of prior back-and-forth is gone.
Why does asking for a shorter answer save more than it seems?
Because output tokens get billed more than once. The response itself costs tokens to generate, and then it becomes part of the input on every subsequent turn in that conversation, getting billed again and again. A longer answer than necessary compounds its cost with every follow-up message.
Should I always convert PDFs to text before uploading them?
When the content matters more than the layout, yes. Plain text or markdown is far more token-efficient than a PDF or screenshots, which carry formatting overhead the model has to process. This is especially worth doing for documents you’ll reference repeatedly.
Do connected tools and integrations cost tokens even when unused?
Yes. Every connected tool sends a description of its function to the model as part of the input, regardless of whether that tool gets called during the conversation. Connecting only the tools relevant to a specific task avoids paying that overhead on every request.
