Skip to content

fix: keep llms.txt generation behind one shared guard - #410

Merged
has2k1 merged 3 commits into
posit-dev:mainfrom
tyza66:fix/llms-txt-links-only-when-generated
Oct 23, 2026
Merged

has2k1 merged 3 commits into
posit-dev:mainfrom
tyza66:fix/llms-txt-links-only-when-generated

Conversation

@tyza66

@tyza66 tyza66 commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Summary

#350 points out that every page advertises llms.txt and llms-full.txt, but
those files are only generated for projects that have an API reference. #351
gated the places that advertise the files on _llms_txt_available(), yet the
code that decides whether the files are actually written still had three
separate, slightly different opinions about when to run. This PR removes that
drift, so a page can never advertise a file the build will not write and a file
can never appear on disk without being advertised.

Two holes are closed:

  • _generate_llms_full_txt() only required an api-reference block with a
    package, so a project with an empty sections list still got a
    header-only llms-full.txt written while _llms_txt_available() reported
    False. This is exactly the docs-only-site case from Every page advertises llms.txt and llms-full.txt, but they are only generated for projects with an API reference — docs-only sites ship two dead links #350, where a page
    links a file that is effectively empty.
  • _prepare_for_freeze() called _generate_llms_txt() and
    _generate_llms_full_txt() directly, bypassing the predicate build() gates
    on. The freeze path could therefore leave files behind that a full build()
    never writes, which breaks the invariant _prepare_for_freeze documents:
    that the build directory ends up holding files in exactly the state a full
    build() would produce.

Changes:

  • New _llms_txt_quarto_config() is the single source of truth for the
    structural half of the guard: _quarto.yml exists, parses to a mapping, and
    its api-reference block has a package and at least one section. It
    returns the parsed config, or None when the project is not eligible.
  • _llms_txt_available() delegates to that helper and keeps the importability
    check that only it needs.
  • _generate_llms_txt() and _generate_llms_full_txt() use the same helper
    instead of re-parsing _quarto.yml under their own, narrower conditions.
  • _prepare_for_freeze() now wraps both generator calls in
    if self._llms_txt_available():, so the freeze path and a full build agree.
  • Tests parametrize six _quarto.yml shapes (no file, no api-reference, no
    sections, no package, unimportable package, importable package with
    sections) and assert that _generate_llms_full_txt() and
    _prepare_for_freeze() agree with _llms_txt_available() in every case.
    This extends the existing test_llms_txt_available_matches_generators from
    the predicate alone to the generators and the freeze path.

One deliberate scope note: _generate_llms_txt() is not given an
importability check. Existing tests such as test_generate_llms_txt_writes_file
and test_generate_llms_txt_b9 pin the behaviour that it writes llms.txt for
a package that cannot be imported, and after this change both production call
sites (build() Step 3 and _prepare_for_freeze()) are gated by the predicate
anyway, so the asymmetry is no longer reachable in a real build. I would rather
flag it here than churn those tests in the same PR; happy to follow up if you
would prefer the guard moved into the generator itself.

Related GitHub Issues and PRs

Checklist

Verification

pytest tests/test_great_docs.py -k "llms or margin or freeze" passes with 94
tests. The full suite was run on Windows before and after the change: 7886
passed / 112 failed before, 7898 passed / 112 failed after, which is exactly the
12 new parametrized tests and no new failures. The 112 failures are pre-existing
and environment-specific (PTY recorder, layout migration, lint, skill install,
page status, interlinks) and reproduce identically on a clean checkout.
ruff format reports both files already formatted and ruff check adds no new
findings. Reverting only great_docs/core.py makes 3 of the new tests fail, so
they do pin the fix.

AI disclosure

This change was produced with the help of an AI coding agent. The agent used
Alpha as its tool: Alpha
drives the editing session, and the agent worked through it to inspect the
repository, make the edits, and run the test suite. Every hunk above was
reviewed by me, and the numbers in the Verification section were produced by
running the suite locally on the branch in this PR.

Copilot AI balanced review requested due to automatic review settings October 22, 2026 18:03

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

posit-dev#350 reported pages advertising llms.txt and llms-full.txt for projects that
never generate them. posit-dev#351 gated the homepage margin, the SKILL.md resources
and the build log on `_llms_txt_available()`, but the checks that decide
whether the files exist still drifted apart:

- `_generate_llms_full_txt` only required a `package`, so an `api-reference`
  block without sections still wrote a header-only `llms-full.txt`.
- `_prepare_for_freeze` called both generators directly, bypassing the
  predicate `build()` gates on, so the freeze path could leave files behind
  that a full build never writes.

`_llms_txt_quarto_config()` is now the single source of truth for the
structural half of that guard, shared by the predicate and both generators,
and `_prepare_for_freeze` honours the same predicate as `build()` so the
build directory really does end up in the state a full build produces.
@tyza66
tyza66 force-pushed the fix/llms-txt-links-only-when-generated branch from 0b886a1 to d4f580b Compare October 22, 2026 18:06
The shared guard only lets the llms.txt generators run when the API
reference configuration names a package and lists at least one section,
so the empty fallbacks they still carried for both settings could never
apply.
Preparing a build directory for freezing still generated the API
reference with quartodoc, which great-docs no longer depends on. The
import failed silently, so the freeze path never wrote reference pages
even though it promises to leave the build directory as a full build
would. It now generates them with the same renderer as a full build, and
only when the project has an API reference.
@has2k1
has2k1 merged commit 81bb715 into posit-dev:main Oct 23, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants