Attention budget and the smart zone
Why an agent gets measurably worse as a session grows, long before it runs out of room.
Every token in the context window has a finite amount of influence to distribute over the others — its attention budget. As the number of tokens grows, each meaningful relationship gets a thinner slice. The result is attention degradation: the signal on the relationships that matter weakens as the session fills with relationships that do not.
What degradation looks like
There is no error message and no visible boundary, which is what makes this dangerous. The symptoms are behavioural:
- It forgets a constraint you stated clearly earlier in the session.
- It reintroduces a mistake you already had it correct.
- It states something confidently that contradicts a file it read forty minutes ago.
- It starts producing code in a style the project does not use — drifting back toward its training distribution because your conventions have thinned out.
Same model, same setup, same prompt quality. Only the volume of context differs.
The practical rule
Plan around the smart zone, not around the window limit. The window is what the tool permits; the smart zone is what you should budget. Concretely:
- One task per session. When the task is done, clear.
- Split larger work at natural boundaries and carry it over deliberately (Module 06 on handoffs).
- When you notice any symptom above, do not push through. Pushing through is how you get three hours of increasingly confused work you then throw away.
Watch out
The seductive move at the first symptom is to re-explain the constraint. That adds tokens, which is the direction that caused the problem. If a constraint is being forgotten, the fix is a fresh session with that constraint near the top — or better, a standing instruction file so it is in every session by default.
Try it
Deliberately run one session into the ground. Give it a moderate task, then keep going: more files, more tangents, more corrections. Watch for the first moment it forgets something. Note the token count. That number is your personal threshold, and having seen it once you will recognise it from then on.
Takeaways
- Attention degradation is gradual and silent; there is no warning from the tool.
- The dumb zone starts long before the window fills — budget the zone, not the limit.
- At the first symptom, clear and restart. Re-explaining adds tokens and makes it worse.
A colleague says they never hit this because their model has a million-token window. What do you tell them?
That the window size and the usable zone are different quantities. A bigger container does not give each token more attention to spend; it just removes the hard stop that used to force a restart. Large windows make degradation easier to reach and harder to notice.
A course by Pieter Zandbergen