Add an llms.txt index for the documentation (#660)

[phmatray]

Implements #660.

Closes #660.

Generates docs/llms.txt, an llmstxt.org index of the 37 pages under docs/website/, and fails the build when it goes stale.

llms.txt lets an LLM answer questions about Fallout from one fetch instead of crawling the site or falling back on stale NUKE knowledge. That fallback is the part that matters after the rebrand: a model trained before it will say Nuke.* and nuke :setup.

What is here

File	Change
build/Build.Documentation.cs (new)	GenerateLlmsTxt and VerifyLlmsTxt
docs/llms.txt (new, generated)	the index itself, committed
build/Build.CI.GitHubActions.cs	VerifyLlmsTxt added to the build workflow's targets
.github/workflows/build.yml	regenerated from the attribute above
.github/workflows/build-skip.yml	runs VerifyLlmsTxt instead of echoing
.fallout/build.schema.json	regenerated for the two new targets

Not hand-maintained, by design. It follows the GenerateTools / VerifyGeneratedTools pattern already in this repo: generate, commit, and fail CI when a source edit lands without regenerating.

Decisions worth reviewing

The gate is wired into both workflows, and that is the point. build.yml has paths-ignore: docs/**, so a PR editing a page under docs/website/ never runs it. That is exactly the change that invalidates llms.txt. Wiring the gate only there would have produced a check that can never fire on the thing it guards. build-skip.yml is the workflow that handles those PRs, so it now runs VerifyLlmsTxt rather than an echo. It keeps the ubuntu-latest job name that branch protection keys on, and it builds the build project and regenerates one file rather than running test and pack.

Metadata comes from what the site already declares, not from new fields. Section names are read from each directory's Docusaurus _category_.json label, so they read "CI/CD Support" and "Common Tasks" exactly as the sidebar does. Title-casing the slug instead would have produced "Cicd" and "Ide". Titles fall back to the first H1 when a page declares no title, which is what Docusaurus does and what badge.md relies on. Descriptions fall back to the page's opening paragraph, since only introduction.md declares one.

The base URL is https://docs.fallout.build/docs/, verified against the live sitemap.xml. All 37 derived URLs match the 37 live pages exactly, set against set. The /docs/ prefix is easy to miss: see the follow-up below.

Test plan

• All 37 derived URLs compared against https://docs.fallout.build/sitemap.xml. Identical in both directions, so no derived URL 404s and no live page is missing.
• ./build.ps1 GenerateLlmsTxt twice in a row leaves git status clean. The generator is deterministic, which is what keeps the gate from flapping.
• ./build.ps1 VerifyLlmsTxt on a clean tree exits 0.
• Negative test: edited a page's title:, committed without regenerating, and the gate failed with docs/llms.txt is out of sync with docs/website. Run './build.ps1 GenerateLlmsTxt' locally and commit the result. Reverted afterwards.
• Negative test: removed a page's title: and the target failed naming the offending file.
• ./build.ps1 VerifyGeneratedTools VerifyLlmsTxt Test, the same target set CI runs: 814 passed, 7 skipped, 0 failed, and the working tree stayed clean, so both verify gates passed.
• ./build.ps1 Compile exits 0. 8 warnings, all pre-existing NU190x advisories, none in the new file.
• Generated file checked by hand: one H1, one blockquote, 8 ## sections, 37 link lines.

Review round

A high-effort review raised 12 findings. Ten are fixed in 7383c59e; none changes the generated file today, and each closes a way it could go wrong on a page the docs do not have yet. The two that mattered most:

• _category_.json without a label crashed the target. GetPropertyValue throws when the property is absent, so the guard and the slug fallback under it were dead code.
• Underscore-prefixed files were indexed. Docusaurus excludes them from routing and docs/website/_snippets/ already exists, so a .md dropped there would have emitted a URL the site never serves.

Also fixed: .mdx pages are indexed, sidebar_position orders pages inside a section as it does on the site, titles are read outside code fences, unclosed frontmatter fails instead of shipping garbage, and titles sort with StringComparer.Ordinal so a different culture cannot fail the byte-for-byte gate with no real drift.

Verified empirically rather than by reading: added _snippets/partial.md, a .mdx page with sidebar_position: 1, and a label-less _category_.json, regenerated, and confirmed the snippet was skipped, the MDX page indexed and sorted first, and the label-less category fell back to its slug instead of throwing. Reverted afterwards; docs/llms.txt is byte-identical.

One finding dismissed. The review flagged that docs/llms.txt sits outside docs/website/ and might never be published. That path is the deliberate cross-repo contract, not an oversight: it is an artifact of the website sources rather than a page in them, and Fallout-build/docs.fallout.build#14 names the exact path to copy. Worth confirming with whoever owns the site build.

Serving half is a separate repo

docs.fallout.build is built from Fallout-build/docs.fallout.build. This PR generates the file; that repo has to copy it to static/llms.txt. Filed as Fallout-build/docs.fallout.build#14.

Neither half is user-visible alone. https://docs.fallout.build/llms.txt stays 404 until both land. That is expected, not an oversight.

Plan

• Task 1: Walk the docs and derive URLs
• Task 2: Parse frontmatter into title and description
• Task 3: Render and write docs/llms.txt
• Task 4: Gate the generated file in CI
• Task 5: Hand off the serving half

Follow-ups

Not fixed here, to keep this PR to its issue.

• README.md:59 links to https://docs.fallout.build/getting-started/installation, which 404s. It is missing the /docs/ prefix. Found while verifying the URL derivation: the issue cited this link as proof of the mapping, and it turned out to be the stale one. It is the only such link in the repo.
• llms-full.txt (the corpus inlined rather than linked). Cheap now that the walk exists, but a different file with a different size budget.
• build-skip.yml now has a real build step, which makes it the natural home for other docs-only validation, such as link checking. Nothing is broken; it is just no longer true that docs PRs run nothing.