docs-builder

Keep project docs current, complete and findable, and split a file when it outgrows its row in docs/index.md.

The honest label: this does NOT make docs cheaper to read. Measured four ways; best case is a tie with doing nothing. Cost tracks findings, not structure — better navigation raises how thorough an agent is willing to be, it does not cut reading. The winning arm wins because synthesis caps cost: the pages already did the reading. Never sell this as a token saving.

Every mechanical step is docs-builder/docs-builder.cjs (vanilla Node, zero deps). A model is used for exactly two things: proposing themes, and writing pages. Bookkeeping done by a script is 100% correct; done by a model it was 27%.

Model tiers — never hardcode a vendor model name

| step | tier | why | |---|---|---| | propose + assign themes | cheapest tier | structured labelling against a fixed list; no synthesis | | write pages | mid tier | semantic synthesis, cheaper/faster than your top reasoning tier |

Use whatever your tool designates as that tier. The measured numbers below were taken on Claude's Haiku 4.5 (cheap) and Sonnet 5 (mid) in August 2026 — the ratios and shapes carry over, the absolute prices do not.

---

Invocation

Locate the script first. docs-builder.cjs is bundled next to this command at docs-builder/docs-builder.cjs — the same directory as this file, whether installed or run from the package. Never search the target repo for it, never reconstruct it from this spec, and if it truly exists nowhere say so and stop. Set DB to its ABSOLUTE path, then cd to the target repo's root:

DB=<absolute path to docs-builder.cjs>
cd <target repo root>

Every command below is node $DB …; everything the script writes (docs/.docs-builder/* JSON state, docs/index.md, the ledger, the log, the config pointer) lands under the target repo. REPO= is optional and only needed when not running from the repo root.

With an argument (reorg, cleanup <file>, or search <query words...>) — run that mode directly, no question asked.

Bare /docs-builder, no argument — ALWAYS ask, never auto-detect. Run due first and put its one-line verdict in the question text so the choice is informed. Then use AskUserQuestion, one question, header Mode, exactly these two options:

Question: What should docs-builder do?

• First run — sort every .md in docs/ into product / archive, then split anything too big into pages and index them. Use when docs are a pile of loose files, or docs-builder has never run here.
• Docs drift — docs moved on since the last run: report what changed, rebuild the index, re-run lint. Nothing is restructured and nothing is split.

Do not offer a third option and do not recommend one. If due cannot run, say so plainly and ask anyway — never guess the mode on the user's behalf.

Auto-detecting was considered and rejected: the two differ in cost (an unreviewed first-run plan vs. a cheap drift check), and a wrong guess on the first one is expensive to unwind.

search is a separate, explicit-argument-only mode — it is NOT a third bare-invocation option. The picker above stays at exactly two; do not add search to it. Typing /docs-builder search <query words...> runs the mode directly (same rule as reorg and cleanup above): it defaults the outline path to docs/.docs-builder/outline.json so the user need only supply query words, and N= overrides the result count (default 10). It is read-only — no model cost, no interview, nothing moves.

What each option runs

First run — three steps, with the classification interview and a stop in between:

1. discover (Mode 0). Nothing moves. It writes reorg-plan.json — one row per file, each carrying a mechanical suggested bucket + reason (a PRIOR, not a verdict), plus h1 and a short content snip, plus an oversized boolean (size no longer decides the bucket). bucket itself starts empty on every row.
2. The classification interview. Feed the model the WHOLE plan table (file, h1, snip, lines, suggested+reason) in one call and have it fill bucket for every row — product/logs/archive — with a one-line reason: honour that a SHOUTED self-declared status is near-conclusive for archive and that suggested is a prior, not an authority. The model writes its answers straight into reorg-plan.json. Then show the user the full resulting table via AskUserQuestion (approve all / correct specific rows / abort) — a correction changes the plan file before anything moves.
3. apply-reorg moves every row, oversized included — size only decides whether a doc is splittable, not whether it gets sorted. It refuses outright if any row's bucket is still empty. Afterward it prints the oversized docs it just moved as a follow-up list, cleanup <NEW path> (N lines), logs/ entries last. If the list is empty (nothing oversized), say so and skip the split question; otherwise show the list, then ask which to split (any, all, none). Only then run cleanup <file> (Mode 1) on each chosen file — cleanup itself prints the estimated split cost for that one file, then a mechanical shape report, then stops for its own interview (Mode 1, step 1b) before anything else runs. Before that first commit, add docs/.docs-builder/ to .gitignore if it is not already ignored: it is machine state, regenerated every run, and the ledger stamp is per-clone by design — it must never ride into history on a later git add -A. Once the moves are committed, run node $DB ledger — nothing in steps 1-3 stamps the ledger, and without the stamp due stays NOT due, the picker's verdict stays uninformed, and /remember's docs nudge never fires.

The two stops are deliberate and different. Step 2 guards correctness — the interview and the user's approval, before a single file moves. Step 3's follow-up guards cost — splitting is ~$0.39 per 1,000 source lines, and the user has seen neither the file list nor the number when they pick "First run". Never split N files in one shot on an unseen list.

Docs drift — run bare reorg (Mode 2, below): its own due-style drift summary prints first, if a ledger stamp exists, then it runs discover. If any row's bucket is still empty (true on a genuine first run, or when new files appeared since the last classification), reorg stops right there and prints what to do next — it never silently proceeds past an unclassified plan. Commit what it changed, then run node $DB ledger to move the stamp. Once the plan is fully classified (an already-sorted corpus's re-run carries its prior classifications forward automatically — see "Discover is idempotent" below), reorg continues straight through apply-reorg → lint, no further stop, so index.md and lint.json stay current. This is the common, cheap case for a corpus that is already sorted: nothing new to classify, so the interview gate never fires.

---

Modes

| Mode | Menu option | Does | Destructive | |---|---|---|---| | /docs-builder reorg (discover, classification interview, confirm, then apply-reorg) | First run, steps 1-3 | classify a WHOLE corpus into product/logs/archive | no (moves are git mv, plan classified and reviewed first) | | /docs-builder cleanup <file> | First run, step 3's split question | measure ONE named oversized doc (cost, scan, heading shape) → stops for the interview | no (measure-only; original preserved) | | /docs-builder reorg (bare docs-builder.cjs reorg) | Docs drift | due's drift summary (if a ledger stamp exists) + discover → (stops here if anything is still unclassified) → apply-reorg → lint, whole corpus | no | | /docs-builder search <query words...> | (none — explicit-argument mode only, never offered in the bare picker) | BM25-rank sections of docs/.docs-builder/outline.json against the query, read-only | no |

reorg and cleanup solve different problems and compose: reorg sorts an entire messy docs/ tree into the four-bucket structure below in one pass and never splits anything itself; an oversized file still moves into its bucket like everything else, but still needs a human to run cleanup <file> individually (below), one named file per invocation, because that step spends real model budget and should never fire without a look first. cleanup is the ONLY entry point to the split pipeline — it refuses more than one file at a time, refuses a missing/non-.md/ protected file, prints its cost estimate, then STOPS for an interview once the shape is measured (Mode 1, step 1b); cleanup-apply (Mode 1, step 4) is the only door back in, and it refuses to run until that interview has produced a labels.json with exactly one theme marked core: true.

Layout

docs/
  README.md          entry point, referenced from CLAUDE.md
  index.md           GENERATED by index-flat/apply-reorg/cleanup-apply. never hand-edited.
                     READER-FACING. The WHOLE-CORPUS map — the only file with a completeness
                     guarantee. ## Product, ## Logs, ## Archive.
  log.md             append-only:  ## [DATE] operation | description — written by
                     `archive`, `apply-reorg`, `validate`, and `reorg`; NOT written by
                     read-only commands (`due`, `search`, `discover`).
  product/           specs, designs, plans — the default. `apply-reorg` MOVES files here
                     (`git mv`); content is never rewritten.
  logs/              pre-registrations, results, learnings, reports — historical, still
                     relevant. Same MOVE discipline as product/archive.
  wiki/              synthesised pages, written by Mode 1 (`cleanup`)'s page writers.
  archive/           what got cleaned up: self-declared dead. Originals are BYTE-FROZEN:
                     nothing under here is ever a rewrite target, so a doc lands byte-identical
                     to what it carried in (a clean R100 rename) and stays that way. Links
                     elsewhere POINTING AT it are still repaired. History via `git mv`.
                     Pruning is `git rm`, the user's own
                     call — nothing here does it automatically.
  .docs-builder/     machine-only working state. Never hand-edited, never read by a human.
    ledger.json        last consolidation SHA + per-doc line counts
    outline.json       Layer 1 scan
    cleanup-shape.json `cleanup`'s mechanical heading-shape report — the interview's proposal
                       is built from this, never a model guess at what a section is "about"
    labels.json        the model's theme assignment (`core: true` on exactly one theme)
    reorg-plan.json    `discover`'s plan (Mode 0): `suggested`+`reason` per row (the script's
                       mechanical PRIOR) plus `bucket` (empty until the classification
                       interview fills it — `apply-reorg` refuses to run while it's empty)
    validate.json      the gate's verdict
    failures.json      LIVE count of current `validate` gate failures, keyed
                       `<check>:<target>` — incremented on failure, DELETED the moment that
                       exact key passes again. Not a history; at 3+ recurrences `validate`
                       adds a STRUCTURAL warning line (message only, never the exit code).
    lint.json          latest lint proposals
    tasks/             one task-<theme>.json per page

index.md deliberately stays visible. It is the thing a reader (or an agent) opens first — the measured winning arm is pages + a coarse index. Hiding it under a dot-dir would break the one mechanism that works. Only machine state goes in .docs-builder/.

index.md has exactly one writer: index-flat (called directly, or from apply-reorg/reorg/cleanup-apply), and it is the corpus's ONLY index. This is load-bearing, not a style choice — a real defect on bareloop is why: a second, themed per-split index used to exist alongside it, and running a PRD split after a reorg silently overwrote the 37-row whole-corpus map with that split's own 7-row view — 30 files vanished from a file that still claimed completeness. Splitting the two apart into two files only moved the problem (it then clobbered outline.json across concurrent splits instead), so the themed index was removed outright, 2026-08-24. One index, rebuilt on every reorg and after every split, is the whole design.

Never moved — enforced in code, not just documented (PROTECTED_NAMES / walkMd):

• Files, at any depth: README.md, index.md, log.md, CHANGELOG.md, LICENSE.md, CONTRIBUTING.md, CODE_OF_CONDUCT.md, SECURITY.md, CLAUDE.md, AGENTS.md, AGENT.md. Bare LICENSE/NOTICE have no .md extension, so the walker never sees them.
• Directories: every dot-dir (.git/, .github/, .claude/, .factory/, .opencode/, .amp/, .docs-builder/) plus node_modules/, and the dirs reorg itself owns (product/, logs/, archive/, wiki/) so a second run is idempotent.

---

Mode 0 — reorg (a whole corpus, not one file)

The old skill (v1) did this job by handing an agent a file list and a prose rulebook ("KEEP/CONSOLIDATE/ARCHIVE", "when uncertain → ARCHIVE") and letting it read, judge and mv every file itself — the exact shape that measured 27% correct on bookkeeping elsewhere in this pipeline. reorg does the same JOB with the same discipline as everything else here: classification is mechanical and script-run; nothing is guessed; nothing moves until a plan has been written and reviewed.

Not rebuilt: v1's CONSOLIDATE (merging two docs' content into one). That rewrites content, a different and higher-risk operation than anything measured so far. Descoped on purpose, not silently dropped.

1. Discover (script) — enriches and PROPOSES, never classifies, never moves

node $DB discover        # defaults to docs/

Recursively finds every *.md under the root (skipping wiki/, logs/, archive/, product/, .docs-builder/, and the protected files), and for each one writes a row with:

• h1 and a short snip (first ~200 chars of body, fence-masked) — reused straight from the same headings()/snippet()/fenceMask() parsers scan uses, no second extraction path.
• oversized — a plain boolean (over the line ceiling, OVERSIZED_LINES, default 500 — an UNMEASURED starting point). Size decides whether a doc is splittable, not whether it's sorted — it is no longer a bucket.
• suggested + reason — a mechanical PRIOR, never a verdict:

| suggested | rule | |---|---| | archive | path already under archive/old/reports/phases, or the doc's own opening declares a SHOUTED status word (CLOSED, DEPRECATED, SUPERSEDED, WITHDRAWN, RETRACTED, REFUTED, ARCHIVAL, ARCHIVED), or the filename matches an archive-shaped prefix (REPORT, STATUS, SUMMARY, FIX_, PHASE_, SPRINT_, DRAFT, WIP, OLD, TEMP followed by - or _) | | logs | filename carries an experiment-record token — PREREG, LEARNINGS, REPORT, RESULTS, POSTMORTEM, RETRO (case-sensitive, word-boundary, checked ONLY after the archive rules above, so a REPORT-old.md still reads as archive, not logs) | | product | has an H1, no archive/logs signal — the default when nothing else applies | | product | no H1, but an include stub — its whole non-blank content (≤3 lines) is nothing but include directives (mkdocs --8<--, {% include %}, {{ .. }}, <!-- include -->) and/or markdown links | a live pointer, not an unknown doc — real-world miss: uv's docs/reference/contributing.md | | product | no H1 at all, and not an include stub — no strong signal either way; the interview decides, same as any other row |

• bucket — empty on any row discover has not classified before (see carry-forward below; a re-run keeps a bucket the interview already set). This is the field the classification interview (step 2) fills, and the ONLY field apply-reorg reads to decide where a file goes. It is shown in the printed table, so a re-run confirms on screen that an earlier classification is still in place.

Discover is idempotent across re-runs, but not blind to prior work. Re-running discover carries an already-classified row's bucket FORWARD for any file it still sees at the same path — it does not re-litigate a decision the interview already made. Only a file discover has never classified before (new since the last run, or reappeared after a manual revert) starts unclassified. suggested/h1/snip/lines/oversized are always freshly recomputed, so the plan stays current even when bucket doesn't move. This is why bare reorg (Mode 2) can compose discover with apply-reorg without a stop on an already-sorted corpus: nothing new to classify, so its own gate never fires.

Measured, why logs exists. Run on bareloop's real product/ (27 files): 11 were experiment records (8 *-PREREG, 2 *-LEARNINGS, others) sitting alongside 14 actual specs and designs — 41% of the bucket was run history, not product, which made the bucket useless for finding specs. logs/ is history that still matters (a prereg or a results doc), distinct from archive/, which is history that is done.

Why the status check requires SHOUTED caps, case-sensitively. Tried case-insensitive first, against a real, uncrafted corpus (not a fixture built to pass). It false-positived three separate ways on real files: "Supersedes **nothing**" (negation), "this rung BUILDS three frozen records" (an input being described, not the doc itself), "archived spines" (data the doc references, not the doc). Same failure species as the lint fix above — a word that means one thing in isolation matches unrelated prose. Restricting to the ALL-CAPS form fixed every one of those, because this corpus's own writing convention (observed, not designed around) SHOUTS a genuine self-declaration — **Status: CLOSED**, (ARCHIVAL 2026-07-25, before any number) — while narrative mentions of the same word stay lowercase or Title Case. FROZEN was in this list too, until 2026-08-23: measured against bareloop's real docs corpus (37 files), 10 of its 12 archive calls were false positives, all from FROZEN — e.g. 2026-08-01-layer-3-reuse-design.md says "design FROZEN... build follows this record" and TYPES-PREREG.md says "FROZEN before any model token is spent" — in that corpus's own convention FROZEN means "locked, do not edit, still current," not "retired." That's the one failure this design promises never to make, so the word was dropped with no replacement heuristic. Traded away: 2 real misses ("Frozen 2026-07-26", "job #4 ... (frozen)") — consistent with precision-over-recall. Neither miss is dangerous: a miss just lands the doc in product, one bucket short of ideal, not mis-archived.

Also dropped from v1's own heuristics, on the same evidence standard: "filename has a date → likely stale." Tested against a real corpus and wrong — 2026-07-28-p-palette- design.md is a current, locked, actively-built spec, not a stale report. A dated filename alone proves nothing.

Output: docs/.docs-builder/reorg-plan.json, plus a printed table. Nothing has moved yet, and nothing has been classified yet either.

2. The classification interview — the model classifies, behind an approval gate

This is deliberately the model's job, not a rule's. The FROZEN incident (above) is usually read as proof a model must not classify — that's the wrong lesson. FROZEN was a mechanical rule, and it did damage precisely because it moved files with no gate at all. The failure was the silent move, not the judgement.

1. Read docs/.docs-builder/reorg-plan.json. Feed the model the WHOLE table — file, h1, snip, lines, suggested+reason — in one call, and have it fill bucket for every row (product/logs/archive) with a one-line reason. suggested is a PRIOR the model is shown, never an authority over it — but a SHOUTED self-declared status (**Status: CLOSED**) is near-conclusive for archive regardless of what the mechanical prior says.

2. Have the model write its answers straight into reorg-plan.json's bucket fields.

3. Show the user the full resulting table via AskUserQuestion — approve all / correct specific rows / abort. A correction changes the plan file.

   Show EXACTLY these four columns, in this order. No extras, no prose padding — the operator is scanning for a row that looks wrong, and every extra column hides it:

   | file | lines | → destination | why |

   • file — the doc's CURRENT path.
   • lines — the plan's line count. Append (oversized) when the row is oversized.
   • → destination — the full destination PATH this row will move to, e.g. docs/archive/PRD.md — never the bare bucket word (archive). A wrong destination is obvious in a path and easy to skim past in a single word. This is the column the operator is actually approving.
   • why — the model's one-line reason, trimmed to one line.

   Sort the rows by destination, so all archive rows sit together, then logs, then product. A misfiled doc is easiest to spot against its neighbours; scattered through a path-sorted list it reads as normal.

Only after approval does apply-reorg run. The approval gate, not the classifier's mechanism, is the safety property here — and it is strictly stronger than a rule that moves files with no gate at all.

3. Apply (script) — an ALREADY-CLASSIFIED plan, verified moves, survives a bad file

node $DB apply-reorg      # defaults to the plan above

Refuses outright if any row's bucket is still empty — the interview-has-not-happened message, not a crash — so nothing can move on an unreviewed plan. A plan from before this version (bucket: 'oversized' or 'review', both gone from the schema) is refused too, with a pointer to re-run discover.

• product → verified git mv to docs/product/<basename>
• logs → verified git mv to docs/logs/<basename>
• archive → verified git mv to docs/archive/<basename>
• Oversized files move too — size decides splittable, not sorted. No bucket is exempt. After the move, every oversized row is printed as a follow-up list at its NEW path, one cleanup <path> (N lines) line per file — run cleanup (Mode 1, below) on each, by hand, one file at a time. The list is ordered, logs/ entries last: a pre-registration is a legitimate split target but rarely the best NEXT one — a prereg is a record of one experiment, meant to be read whole. Auto-splitting N unknown files in one shot would spend real model money with no confirmation; the pipeline never does that unprompted, and cleanup itself refuses to run on more than one file.
• After the scan, apply-reorg writes docs/index.md itself — it calls index-flat (see below) automatically, so a reorg-only corpus ends up indexed without a second command. Runs every time, unconditionally.
• apply-reorg also writes the docs pointer into CLAUDE.md — a marker-wrapped <!-- DOCS_INDEX:START -->/<!-- DOCS_INDEX:END --> block naming docs/index.md as a plain path, never an @-reference: hot-loading a 100-row index into every session is exactly

<!-- Content truncated for initial SEO render. Open the source file tab for the full file. -->