Detect Stale and Fabricated Docs in Your Codebase

Part of the DeepSeek Harness PR Review guide series

In most codebases a large share of documentation no longer matches the code. Some docs are stale — they described the code once and were never updated. Some are wrong — the description drifted from reality. And some are fabricated — they describe features that have never existed in the code at all. An LLM that reads both the docs and the code can classify every doc automatically.

The four doc statuses

Match — the doc accurately describes the code
Stale — the doc reflects an older version of the code; partially true, partially outdated
Wrong — the doc contradicts the actual code today
Fabricated — the doc describes behavior that does not exist anywhere in the code

How the check works

When a PR changes code, the docs most likely invalidated are ranked first — scored against the diff by path proximity, by the symbols the diff declares, and by which changed filenames they mention — and a dedicated docs agent verifies that ranked list against the code. The ranking is a starting point, not a limit: the agent still reports a doc it finds outside the list. Each candidate is read side by side with the real code:

# example findings
docs/API.md      → STALE      "Documents email/password login only;
                                the code now has an idToken branch"
docs/PAYMENTS.md → FABRICATED "Describes an idempotency-key header that
                                does not exist in the code"
docs/RELEASE.md  → MATCH      "Bundle ID and push config still accurate"

Why "fabricated" matters most

A stale doc misleads; a fabricated doc is worse — it documents a feature that was never built. New team members read it and spend days looking for functionality that does not exist. Automated doc verification catches this at PR time, before the wrong docs propagate further.

Human confirmation for wrong or fabricated docs

Docs flagged Wrong or Fabricated are significant enough that the tool requires a human confirmation before recording them as findings. The confirmation is a single ≤20-word question, and the answer is stored with the report so the decision is traceable.

Turn doc debt into an actionable list

Every PR review produces a docs reality-check table. Over time the web dashboard accumulates per-repo doc error counts — a measurable signal of how much of your documentation you can actually trust. That number is the first step toward fixing the root cause instead of the symptoms.