validate-docs

Validate that ALL features claimed in README are documented in specs and tested.

Philosophy

The README is the product's promise to users. Every feature advertised must be:

1. Specified in docs/specs/ (source of truth for behavior)
2. Tested in frontend/tests/ (proof it works)

Workflow

Step 1: Extract ALL features from README

Read README.md and identify every feature claim. Look in:

• How It Works section - core functionality
• UI section - user-facing features
• Agent Workflow section - automation features
• Any diagrams, bullet points, or descriptions that promise functionality

For each feature, note:

• What it claims to do
• Whether it has a linked spec (some won't)

Step 2: Map features to specs

Check if each README feature has a corresponding spec:

| README Feature | Expected Spec | | ------------------------------------------ | ------------------------------------------------ | | Main thread conversation | docs/specs/chat.md | | Sessions/background work | docs/specs/sessions.md | | Mobile/desktop layout | docs/specs/layout.md | | Agent workflow (spawn → work → PR → close) | docs/specs/agent-workflow.md | | Notifications | docs/specs/sessions.md (notifications section) |

Flag any README feature without a spec as ERROR.

Step 3: Verify specs have testable assertions

For each spec file:

• Check it contains specific, testable statements
• Assertions should mention exact UI text, behaviors, or states
• Vague specs like "works well" are not testable

Step 4: Map spec assertions to tests

For each bullet point/assertion in a spec, search for a test that verifies it:

# Example: search for test covering "No sessions yet" message
grep -r "No sessions yet" frontend/tests/

Track coverage for each spec assertion.

Step 5: Output report

## Documentation Validation Report

### README Features → Specs

- [x] Main thread conversation → docs/specs/chat.md
- [x] Sessions → docs/specs/sessions.md
- [x] Layout → docs/specs/layout.md
- [ ] **Agent Workflow → NO SPEC** ← ERROR

### Spec Assertions → Tests

#### docs/specs/chat.md (7/9 = 78%)

- [x] "Input field with placeholder" → e2e/user-journey.spec.ts
- [ ] "Messages persist across reloads" → NO TEST

#### docs/specs/sessions.md (23/23 = 100%)

- [x] "No sessions yet" → sessions/01-session-list-empty.spec.ts
      ...

### Summary

| Category                   | Coverage    |
| -------------------------- | ----------- |
| README features with specs | 3/4 (75%)   |
| Spec assertions with tests | 38/40 (95%) |

### Errors

1. README "Agent Workflow" section has no spec
2. chat.md "Messages persist across reloads" has no test

Step 6: Exit status

• ERROR if any README feature lacks a spec
• ERROR if any spec assertion lacks test coverage
• SUCCESS only if fully covered

Common Gaps to Watch For

1. Advertised but unspecified - README promises feature, no spec exists
2. Specified but untested - Spec describes behavior, no test verifies it
3. Workflow features - Multi-step flows (like agent lifecycle) often lack e2e coverage

Tests to Flag for Removal

Not all tests are good tests. Flag these as problems:

1. Tests without specs - If a test exists but no spec describes the behavior, either:

   • The spec is missing (add it), or
   • The test is testing implementation details (remove it)

2. Too technical / low-level - Tests should verify user-facing behavior, not internals:

   • ❌ "store updates when API returns data"
   • ❌ "component re-renders on state change"
   • ✅ "user sees updated session status after refresh"

3. Testing implementation, not behavior - Tests coupled to code structure:

   • ❌ "calls fetchSessions with correct params"
   • ❌ "dispatches ACTION_TYPE to store"
   • ✅ "sessions list shows new session after spawning"

4. Duplicate coverage - Multiple tests verifying the same user behavior

5. Tests for removed features - Spec was removed but test remains

The test suite should read like a user manual, not a code audit.