Detect Stale and Fabricated Docs in Your Codebase

Part of the DeepSeek Harness PR Review guide series

In most codebases a large share of documentation no longer matches the code. Some docs are stale — they described the code once and were never updated. Some are wrong — the description drifted from reality. And some are fabricated — they describe features that have never existed in the code at all. An LLM that reads both the docs and the code can classify every doc automatically.

The four doc statuses

Match — the doc accurately describes the code
Stale — the doc reflects an older version of the code; partially true, partially outdated
Wrong — the doc contradicts the actual code today
Fabricated — the doc describes behavior that does not exist anywhere in the code

How the check works

When a PR changes code, the review agent finds the docs that reference the changed areas and reads them side by side with the real code:

# example findings
docs/API.md      → STALE      "Documents email/password login only;
                                the code now has an idToken branch"
docs/PAYMENTS.md → FABRICATED "Describes an idempotency-key header that
                                does not exist in the code"
docs/RELEASE.md  → MATCH      "Bundle ID and push config still accurate"

Why "fabricated" matters most

A stale doc misleads; a fabricated doc is worse — it documents a feature that was never built. New team members read it and spend days looking for functionality that does not exist. Automated doc verification catches this at PR time, before the wrong docs propagate further.

Human confirmation for wrong or fabricated docs

Docs flagged Wrong or Fabricated are significant enough that the tool requires a human confirmation before recording them as findings. The confirmation is a single ≤20-word question, and the answer is stored with the report so the decision is traceable.

Turn doc debt into an actionable list

Every PR review produces a docs reality-check table. Over time the web dashboard accumulates per-repo doc error counts — a measurable signal of how much of your documentation you can actually trust. That number is the first step toward fixing the root cause instead of the symptoms.