Context Rot: Why AI Gets Worse the Longer You Talk to It
You've probably felt it: a long AI session that started sharp slowly gets dumber
You've probably felt it: a long AI session that started sharp slowly gets dumber. The model contradicts something it said earlier, forgets a constraint you set, or confidently states something you'd already corrected. You're not imagining it. It's called context rot, and understanding it changes how you work with AI.
What is context rot?
The gradual breakdown of a model's ability to correctly use earlier context as a conversation or prompt gets longer. Crucially, it's not "forgetting" in the human sense — all the tokens are still there, inside the context window. The model just starts to misunderstand, ignore, or inconsistently apply them.
So you get a model that has the information available and still acts as if it doesn't: contradicting an earlier statement, dropping a constraint you set twenty messages ago, answering as though an instruction was never given. The data is present; the reliable use of it has decayed.
Why does AI get worse the longer you chat?
Because the ratio of signal to noise in the context window drops as the session grows.
Every exchange adds tokens — your messages, the model's replies, tool outputs, files. As that pile grows, the genuinely important details (the constraint you set, the goal you stated) get diluted among thousands of tokens of conversational filler. The model's attention has more to spread across and a harder time locating what matters.
There's a well-documented version of this called "lost in the middle": models reliably attend to the start and end of a long context but get unreliable about everything in between. Benchmarks bear it out — many models drop sharply in their ability to recall and use information as context length climbs, even when there's technically room left in the window. A bigger window doesn't save you; it often just gives rot more room to set in.
What are the signs of context rot?
It's often subtle, because the model keeps sounding confident. Watch for:
- Contradictions within a session — the model says something that conflicts with an earlier answer or a constraint you gave it.
- Dropped instructions — a rule you set early ("always respond in bullet points," "never use that library") quietly stops being followed.
- Confident fabrication — instead of flagging that it's lost track, the model invents a plausible version to keep going. This is the dangerous one, because it doesn't look like an error.
- Degraded reasoning late in long sessions — answers get vaguer or more generic the deeper you are.
The through-line: the model doesn't announce it's struggling. It plows ahead, which is exactly why context rot erodes trust before you notice it.
How do you prevent context rot?
The tactical fixes are about keeping the context window lean:
- Start fresh sessions often — new topic, new chat. A clean window has no rot.
- Summarize and restart — when a session gets long, have the model summarize, then start over with just the summary.
- Keep the working context small — don't dump everything in; include what the current task needs.
These help, but they share a weakness: they put the burden on you to manage the window manually, every session. And they don't solve the deeper issue — that the context you keep re-explaining (who you are, your standards, your project) is exactly what bloats every conversation in the first place.
How does a context layer help?
By moving your persistent context out of the conversation entirely.
The information most prone to causing rot — your background, your project's rules, your preferences — is the stuff you'd otherwise restate in every session, padding the window each time. A context layer holds that outside the chat and loads it fresh at the start of each session through MCP, in structured form. The model gets what it needs to understand you cleanly, instead of you re-typing it into a window that's already filling with history.
The effect is a higher signal-to-noise ratio from the first message. You're not relying on the model to dig your constraints out of message forty, because they arrived structured at message one — and they don't decay over the session, because they live outside it. You still start fresh sessions for new tasks; you just never start them cold, and you never pay the rot tax on the context that matters most.
Context rot is, at its core, a problem of too much undifferentiated information in one window. Keeping your durable context structured and external is how you stop feeding the problem.
→ How models handle the window: How Do LLMs Use Context?
→ How external context loads in: How to Deliver Personal Context to AI Tools
→ Keep your context out of the rot with Unabyss →
