Context rot: The silent cause of AI agent failure

Comprehension debt usually describes the gap between the code a system contains and the code any human understands. But a second ledger exists that nobody audits: the documentation written to close that initial gap. I found a database documentation section in a monorepo’s agent context file that pointed to the wrong directory for 252 days. It survived 12 subsequent context file edits, spread to 17 other files, and quietly caused five feature branches to write code to a non-existent path. I wrote a checker and ran it across 15 repositories to determine if this was just an isolated incident. Paths in this article have been renamed; every number remains untouched.

An ls that returned nothing

While verifying a coding standard in a large .NET monorepo, I followed the repository's agent context file. The document devoted 34 lines to database architecture: five directories, five file counts, eight worked search commands, and a bolded standing instruction: always check this folder before answering questions about stored procedures.

So I ran the first command listed:

Bash
$ ls docs/sql/procedures/*ProcName*.sql
ls: cannot access 'docs/sql/procedures': No such file or directory

The directory had moved eight months earlier. It was not deleted, merely relocated from docs/sql/ to docs/generated/sql/ in a pull request titled "modern portal endpoint migration setup"—a PR completely unrelated to documentation. All 1,679 files remained intact, but the map pointing to them was wrong.

A broken path is an unremarkable bug. What happened next is the real issue.

Dating the damage

Because Git tracks changes, I measured the exact timeline rather than guessing:

Bash
# when the directory moved out from under the documentation
git log --format='%h %ad %s' --date=short -- docs/sql/procedures

# every edit to a context file since, none of which noticed
git log --format='%h' --since=2025-12-09 -- AGENTS.md | sort -u | wc -l

# how far the wrong path spread
git grep -ln "docs/sql/" -- ':!docs' | wc -l

The commands returned three specific metrics:

• 252 days stale
• 12 later commits touching the two context files shipping with the repository
• 17 tracked files repeating the bad path

Those 12 commits elevated a simple typo into a broader system failure. These were not casual edits. One commit message read "refactor context docs." Another claimed to update areas pointing to an old solution file. A third introduced a consolidated standards document intended as the single source of truth. Engineers deliberately updated the context files 12 separate times, yet the dead reference survived every pass.

The documentation read perfectly. That was the core issue. Prose reviews cannot detect a path that no longer exists on disk.

The issue then compounded in three distinct ways:

1. The file counts drifted independently of the directory move. Cross-referencing the claims against the new directory revealed 662 claimed stored procedures versus 633 actual, 233 views versus 229, 34 functions versus 33, and 15 schemas versus 1. Tables claimed 731 and found 731—the sole survivor among the five counts.
2. The error was inherited. The consolidated standards document, written three months after the move, listed the dead directory as a canonical path. Whoever assembled it worked from existing documentation rather than verifying the underlying file tree. A false claim was promoted to authoritative status by the very project tasked with establishing truth.
3. Engineers began building at the ghost address. Five separate feature pull requests added SQL files to the dead path, creating a five-file directory structure alongside the real 1,679-file tree. The most recent pull request landed a week before my audit. The contributors followed instructions: they checked where the documentation directed them to place files, and put them there.

Plaintext
Day 0:   docs/sql/ created; context file written to match
Day 3:   Unrelated PR moves directory to docs/generated/sql/; docs are not updated
Day 101: "Single source of truth" doc inherits the wrong path
Day 110: First feature PR creates a file at the dead address (4 more follow over 5 months)
Day 252: Issue measured; path remains uncorrected

Between day 3 and day 252, no one updated the documentation, and the repository reshaped itself around the error. That gap is the core problem.

Why nobody noticed

This breakdown falls squarely under comprehension debt rather than basic carelessness.

For decades, documentation maintained accuracy because errors created immediate friction. When a developer followed a broken path, they got stuck and asked for help. That friction acted as an immediate repair signal. It was unscheduled and inconvenient, but effective, because human confusion moves upstream.

Conversely, an AI agent following a broken path receives an empty directory response and immediately moves on. It displays no surprise and files no issue ticket. Instead, it generates a confident answer derived from whatever other context it can gather. The response usually appears correct because the remainder of the context file is accurate.

Agents do not get confused. That is a feature, but it also removes the historical feedback loop that kept documentation accurate.

Engineers designing APIs for AI agent consumption report similar issues, observing OpenAPI specifications rotting quietly because routing agents never complain about discrepancies. What remains unmeasured is the downstream cost of these silent failures.

Was it bad luck?

A single broken section does not prove a system-wide trend. To test this across our ecosystem, I built a checker to validate claims in context files that could be evaluated deterministically without parsing natural language prose. I then ran it across every repository in the workspace.

The tool evaluates three types of falsifiable claims:

Python
import re

# A path claim: a backticked token containing a directory separator
PATH_TOKEN = re.compile(r"`([^`\s]*/[^`\s]*)`")

# A count claim: the first numerical quantity on a line containing a path
COUNT = re.compile(r"\b(\d[\d,]{1,6})\s+([a-z][a-z\- ]{2,40}?)(?=[,.()]|$)")

# A command claim: an ls or grep operand inside a code block,
# resolving to its deepest fixed directory
LS_CMD = re.compile(r"^\s*(?:[$#]\s*)?(?:ls|cat|head)\s+(?:-\w+\s+)*([^\s|>&;#]+)")

Any claim the checker cannot evaluate is classified as unverifiable and included in the final totals rather than discarded. Excluding non-deterministic claims inflates checker accuracy, and inflated accuracy is the precise issue under examination.

Filtering out false positives required four rounds of edge-case handling: removing placeholders like {Domain}, routing templates like Status/PingSystemHealth, illustrative documentation patterns like "each library exposes src/index.ts", and valid cross-repository pointers such as "lives in the Modules repo at .claude/architecture-guide.md."

The final scan across 15 repositories and 23 context files yielded the following metrics:

• 110 falsifiable claims checked
• 18 false claims identified
• 58 unverifiable claims
• 83.6% accuracy rate

Fourteen of the fifteen repositories scored 100%. All 18 false claims resided in a single repository, and 17 of those traced directly to that single directory relocation.

The 18th error was a pointer to a branch-naming guide. The commit that deleted the guide was the consolidation effort itself—the initiative designed to establish a single source of truth. The PR removed the file but left the pointer intact, leaving a dangling reference for 151 days.

Documentation decay is not a slow, uniform process. It is event-driven. Moving a single directory during an unrelated pull request produced every false statement across the code base.

The useless documentation decayed first

A study published by ETH Zurich evaluated the utility of developer-written context files using a benchmark of 138 tasks across 12 repositories.

Developer-written context files improved task completion by roughly 4%. Machine-generated context degraded performance by 0.5% to 2% compared to providing no context at all. Both approaches increased inference costs by over 20%.

Crucially, the study noted that while agents reliably follow explicit instructions in context files, repository overviews—sections describing folder structures and inventories—do not improve performance even when completely accurate.

Consider which documentation sections decay first. Architectural rules like "keep data access out of the service layer" remain accurate over time because they describe design constraints rather than physical states. Conversely, an inventory listing five directories and their contents becomes inaccurate the moment a developer executes git mv. Descriptions of system states expire quickly; descriptions of architectural intent do not.

Decay targets the exact documentation that provides the least return on investment. All 18 false claims identified in the scan were descriptive inventories rather than architectural rules.

While this might suggest deleting repository overviews entirely, that approach overlooks how these sections interact with system instructions.

Descriptive inventories and operational directives often exist in the same context block. For example, consider the directive: always check this folder before suggesting changes. This is an explicit rule placed inside an inventory. While agents follow directives, relocating the underlying folder leaves the directive's authority intact while invalidating its destination. The agent obeys the instruction, but executes against a dead path.

To manage this, separate claims by whether they can be deterministically verified rather than by whether they function as overviews:

• Paths, file counts, directory layouts, and CLI commands: Generate or validate these claims in CI pipelines rather than writing them manually. A hardcoded number in a document is simply an unexecuted test.
• Layering rules, naming conventions, and design rationale: These cannot be checked automatically, but they represent the core rules agents actually follow. Human editorial time should focus exclusively here.

Periodic manual documentation reviews often fail to catch these issues. The audited context files passed 12 manual reviews because reviewing prose confirms readability, not technical truth. Dead paths still read well; catching them requires automated execution.

Managing unverifiable documentation

Automating deterministic path checks is straightforward. The broader challenge is managing the 58 claims the checker categorized as unverifiable—the layer where architectural understanding lives and decays without obvious error signals.

Implement three core practices to govern this layer:

1. Convert structural rules into executable tests

Layering constraints are critical to preserve, yet natural language prose enforces them poorly. If a standards document dictates that data access logic must remain separate from the service layer, turn that sentence into an explicit test:

C#
var result = Types.InCurrentDomain()
    .That().ResideInNamespace("Company.Services")
    .ShouldNot().HaveDependencyOn("Company.Data")
    .GetResult();

Assert.True(result.IsSuccessful, "Architecture violation: Services depend on Data layer.");

Tools like ArchUnit (Java) and NetArchTest (.NET) formalize these constraints as architecture fitness functions. Because AI agents continuously consult these rules, a single violation can quickly propagate into newly generated code. Backing structural rules with automated tests prevents silent architectural drift.

2. Use code reviews to capture reusable rules

Use code reviews to write architectural rules, not just to catch local defects. A review comment fixing a single pull request loses its value once merged. Promoting that same comment into a documented rule ensures agents read it on every subsequent task.

Track repetition in code reviews. If reviewers repeatedly flag the same pattern, the documentation is missing an explicit rule. For agent-driven development, adding that rule directly updates the context driving future code generation. Review shifts from a gate on a single diff to a mechanism for updating the machine-readable rules of the system.

3. Document non-obvious patterns and counter-intuitive logic

Capture patterns that automated reviewers and developers frequently flag as errors even though they are intentional. This includes valid null-forgiving operators, two-stage pagination queries, or specialized correlated subqueries.

Every entry should document a resolved engineering debate. While these entries cannot be validated by an automated checker—since they assert intent rather than structural facts—they prevent agents from refactoring correct code and spare human engineers from repeating settled arguments.

Knowledge sharing often fails when it merely describes what the code does, as AI agents read code faster than humans can document it. Instead, focus documentation on design choices: why an implementation looks counter-intuitive, what approaches were attempted and abandoned, and what constraints dictated the final design. This context exists nowhere else in the repository; once the engineer who wrote it leaves, that knowledge is lost.

These three practices share a common framework: automate deterministic claims so they cannot decay, and rely on human engineers to capture rationale that cannot be automated. Machines verify the facts; humans document the rationale.

This approach preserves the institutional record in a format that survives employee turnover. In the monorepo audit, the underlying failure was not a loss of system understanding—the engineer who moved the SQL files knew exactly where they went. The failure was that the system record did not update alongside the code, leaving both human engineers and AI agents working from an invalid map for 252 days.

Limitations of the data

This audit did not measure an agent generating an incorrect code output directly from the stale context file. The premise that agents follow invalid paths silently rests on the ETH Zurich benchmark findings alongside the five feature branches created in the non-existent directory. This provides clear observational evidence that both humans and agents acted on an invalid map, rather than a controlled experiment.

Additionally, 58 of the 110 claims remained unverifiable by automated scripts. An automated checker can confirm a directory exists, but it cannot evaluate whether the accompanying description is accurate or worth the token cost to include in the context window. Comprehension debt can easily persist within documents where every file path successfully resolves.

Implementation steps

To audit your repository for comprehension debt:

• Run a basic path-validation script. Extract all backticked paths from the context files your agents read, verify their existence on disk, and fail the build if any path is broken.
• Audit unverifiable claims. Identify inventory lists that can be generated or deleted, and convert testable architectural rules into automated tests.
• Document rationale. Retain high-value rationale explaining non-obvious design choices, explicit constraints, and historical trade-offs.

The five feature pull requests created in the dead directory highlight the hidden cost of invalid documentation. The engineers were not careless; they read the documentation, trusted it, and placed their work in the designated location.

That is the true cost of an unverified map: it does not produce obvious errors during code review. It produces diligent engineers and AI agents carefully building software in the wrong place.

Historically, documentation failed loudly when an engineer hit a dead end and raised an issue. Today, the primary consumer of context files never gets stuck. As a result, validation mechanisms must be explicitly built, automated, and continuously executed.