What Is Context Rot? Why AI Agents Get Worse Over Time

Context rot is the gradual decline in an AI model’s reliability as the information in its working context grows. The model may still be inside its advertised context-window limit, yet become worse at finding the right fact, following an earlier instruction, or connecting information spread across the session. Last reviewed July 18, 2026. Maintained by the Plannotator project. For an AI agent, context includes more than your latest message. It can include the system prompt, chat history, files, tool instructions, tool results, retrieved documents, plans, errors, and the agent’s own previous work. All of that competes to influence the next answer or action. Context rot does not mean the model has literally decayed. It means the working set has become harder to use well.

​

Think of context as a desk, not a filing cabinet

A large filing cabinet can hold more documents. A large desk can also hold more documents, but covering every inch of it does not necessarily help you make one decision. The model needs the right material on its desk for the current step. A concise specification, the relevant source files, and one failed test may be more useful than the entire repository, a long transcript, every tool result, and several outdated plans. This is the practical distinction:

• Context capacity is how much the model can accept.
• Effective context is how much of that material the model can use reliably for the task.

The first number appears on a model card. The second depends on the model, the task, the position and similarity of relevant information, the amount of distracting material, and how the agent manages its working state.

​

Context windows grew from thousands to millions of tokens

The increase has been dramatic. GPT-3 was trained with a 2,048-token context window. Anthropic expanded Claude from 9,000 to 100,000 tokens in 2023. Google introduced a one-million-token Gemini 1.5 Pro window in 2024. By 2026, current frontier agents had made million-token windows ordinary: Claude Fable 5 supports 1M tokens, and the GPT-5.6 family supports 1.05M. Larger windows are genuinely useful. They let models inspect longer documents, more code, extended video, and richer histories in one request. They also delay the hard limit where old material must be removed or summarized. They do not guarantee that every included detail will influence the result equally well.

​

Context rot can start before the window is full

A context window has a hard boundary: the maximum input the model accepts. Context rot is usually a performance gradient before that boundary. Three problems can overlap:

​

Relevant information becomes harder to select

The model may need to find one constraint among many similar notes, connect facts that use different words, or distinguish the current decision from an older one. This is harder than retrieving an exact phrase from an otherwise unrelated document.

​

Distracting context is not neutral

Old plans, repeated instructions, failed approaches, broad tool output, and almost-correct examples can compete with the current task. A related but wrong statement may be more distracting than completely unrelated text.

​

Long sessions accumulate contradictions

An agent produces new state on every turn. It reads files, runs commands, receives errors, writes plans, revises plans, and explains its actions. If obsolete conclusions remain beside current ones, the model must infer which version still applies. That is why adding more explanation can make a struggling session briefly better, then worse. The repeated instruction receives fresh emphasis, but the session also grows.

​

What does the research show?

There is no single benchmark that captures every real agent workflow. Several lines of research point to the same practical warning.

• Chroma’s Context Rot evaluation tested 18 models. Performance became increasingly unreliable as input length grew, even when the researchers held the task itself constant. The effect varied by model, input structure, semantic similarity, and distractors.
• Lost in the Middle found that models often used relevant information more reliably when it appeared near the beginning or end of a long input than when it appeared in the middle.
• RULER moved beyond simple retrieval to multi-hop tracing and aggregation. Nearly all evaluated models declined as context grew, and only half maintained satisfactory performance at 32,000 tokens despite advertising windows of at least that size.
• NoLiMa removed obvious word overlap between a question and the relevant passage. Eleven of thirteen evaluated models fell below half of their strong short-context baselines at 32,000 tokens.

These tests do not say long context is useless. They show that fitting information into the window and using it reliably are different achievements. They also should not be read as a 2026 model leaderboard. The studies establish recurring long-context failure patterns, but most predate Claude Fable 5 and GPT-5.6. Current frontier models are more capable, and teams should measure them on their own tasks rather than assume an older benchmark score still describes today’s model.

​

What does softmax have to do with it?

Most modern language models use an attention mechanism to decide how strongly parts of the input should influence one another. Softmax helps turn many possible relationships into a weighted distribution. The paper Softmax Is Not Enough proves a narrower result: learned softmax circuits can lose sharpness when they must choose across more items than they saw during training, even on a simple maximum-selection task. The authors describe this as a problem of sharp size generalization. In plain language, a selection that was decisive across a short list can become less decisive as the list grows. That is one useful explanation for why more possible inputs can make selection harder. It is not proof that softmax alone causes every long-session failure. Training distribution, position handling, retrieval quality, task complexity, contradictory information, and the agent harness also matter.

​

Why do AI agents seem to get worse over time?

Long-running agents create context while they work. A coding agent may begin with a clear task and a few files, then accumulate search results, source files, test logs, screenshots, failed patches, review feedback, and its own explanations. Matt Pocock calls the practical symptom attention degradation. The model has not changed, but constraints that worked earlier begin to slip. The agent may ask a question that was already answered, ignore a file it read, repeat a failed approach, or introduce code that conflicts with an earlier decision. Mike Ramos uses two pop-culture metaphors in 50 First Dates with Mr. Meeseeks:

• The model is like a person who must watch a briefing tape to reconstruct what happened before. The current context is that tape.
• A chat is like a short-lived worker created for one task. Keeping the same worker around indefinitely can produce increasingly strange behavior.

The metaphors are intentionally simple, but the operating advice is sound: do not mistake a long transcript for durable memory, and do not keep one session alive merely because it contains history. Agent products may add memory, compaction, files, search, or summaries. Those features can improve continuity. The underlying model still acts on the material loaded for the current inference. Stored knowledge helps only when the right part returns at the right time.

​

Will better models solve context rot?

They can reduce it. Claude Fable 5 and GPT-5.6 are designed for ambitious long-running work and ship with roughly million-token windows. Newer models often handle longer inputs, more distractors, and more complex retrieval better than earlier models. That progress does not turn a long transcript into reliable memory. The same million-token request can contain current instructions, obsolete plans, repeated tool output, and several plausible but conflicting answers. More capacity gives the model room to receive them all; it does not decide which ones the team still trusts. Current agent products are also managing context instead of relying on one ever-growing prompt. Anthropic’s context-window documentation recommends compaction for conversations approaching the limit, while its broader context-engineering guidance recommends structured notes, just-in-time retrieval, and focused subagents. Anthropic says Claude Fable 5 can work for days and delegate to subagents. OpenAI’s GPT-5.6 ultra setting coordinates four agents by default for demanding tasks. The direction is clear: stronger models, larger windows, and better orchestration are arriving together. Long-running systems still need to divide work, preserve durable state, and keep irrelevant intermediate material out of the coordinating agent’s working context. The likely future is not a choice between larger windows and context management. Teams will use both:

• larger windows for richer local understanding;
• better models for more reliable long-context reasoning;
• external files and artifacts for durable state;
• retrieval for loading only what applies now;
• compaction and fresh sessions for clearing accumulated noise;
• evaluations for measuring whether the approach actually works.

​

How to reduce context rot in an AI agent

The practical goal is to replace an undifferentiated pile of session history with a small, current set of sources and a verifiable handoff.

​

Give one session one coherent job

Do not use a single chat as the permanent home for unrelated planning, implementation, debugging, review, and launch work. Start a fresh session when the objective changes materially.

​

Keep the current truth outside the transcript

Record the accepted plan, active constraints, current status, failed approaches, and definition of done in maintained files or artifacts. A new agent should not have to reconstruct them from hundreds of conversational turns.

​

Retrieve context just in time

Give the agent a concise map of available sources, then let it open the relevant files, specifications, or decisions. Avoid loading an entire wiki or repository when the task concerns one subsystem.

​

Remove obsolete tool output

Raw logs and tool results are useful while diagnosing a problem. After the conclusion is recorded, the full output may become noise. Keep the finding and a pointer to the evidence.

​

Compact with a verification step

A summary can omit the fact that matters later. After compaction, check that the objective, constraints, decisions, unresolved risks, modified files, test results, and next action survived.

​

Use focused subagents for bounded investigations

A subagent can explore one question in a clean context and return a short result to the main task. This keeps detailed search trails out of the coordinating agent’s working set.

​

Test the workflow, not only the model

Measure wrong-file edits, repeated questions, ignored constraints, recovery after compaction, and the accuracy of handoffs. The effective context window belongs to the entire agent system, not only the model API.

​

When should you start a fresh agent session?

Reset or hand off when the agent:

• repeatedly ignores a constraint it followed earlier;
• asks for information already present;
• confuses current and superseded plans;
• repeats an approach that already failed;
• keeps adding patches without restoring a coherent design;
• cannot state the current objective, system state, and next verification step clearly.

Do not paste the entire old transcript into the new session. Give the new agent a concise handoff with the objective, current state, decisions, evidence, open questions, and next action. Link to primary files so it can retrieve detail as needed. For longer work, see What Are Long-Horizon Tasks for AI Agents? and AI Coding Agent Handoffs.

​

Frequently asked questions

​

Is context rot the same as reaching the context-window limit?

No. The limit is the maximum amount of input a model accepts. Context rot is declining reliability as the working context grows, and it can appear well before the limit.

​

Is context rot the same as hallucination?

No. A hallucination is an unsupported or false output. Context rot can contribute to hallucinations, but it also appears as missed instructions, weak retrieval, repeated work, or inconsistent reasoning.

​

Does a one-million-token context window prevent context rot?

No. It increases capacity. It does not guarantee uniform use of every token or equal performance across all long-context tasks.

​

Is attention degradation another name for context rot?

They overlap. Context rot names the broader observed decline as input grows. Attention degradation is a useful practitioner term for the way important signals lose influence inside an increasingly crowded session.

​

Should I always start a new chat for every task?

Not necessarily. A coherent task benefits from continuity. Start fresh when the objective changes, the session becomes noisy, or the agent can no longer state and follow the current constraints reliably.

​

Can retrieval or a codebase wiki solve context rot?

They can reduce unnecessary context when they route the agent to a small, current set of sources. They can make the problem worse when they return broad, stale, overlapping, or contradictory material. See What Is an LLM Codebase Wiki?.