Migrate documentation from Sphinx to Quarto by stephengreen · Pull Request #391 · dingo-gw/dingo

stephengreen · 2026-07-02T13:35:33Z

Summary

Migrates the documentation from Sphinx/MyST to Quarto + quartodoc, done in place on docs/source, hosted on Read the Docs via a custom build.commands build — keeping RTD's versioning, pull-request preview builds, and the existing dingo-gw.readthedocs.io URLs.

Preview: every push to this PR builds a preview on Read the Docs — see the docs/readthedocs.org:dingo-gw check below (https://dingo-gw--391.org.readthedocs.build/en/391/).

This branch is now Quarto-only — the Sphinx toolchain (conf.py, docs/Makefile/make.bat, and the Sphinx/MyST deps in the dev extra) has been removed, and .readthedocs.yaml is rewritten for the Quarto build. RTD keeps serving the old Sphinx docs from main until this migration lands there.

Scope note: prose is carried over verbatim from the old docs — this PR is about the migration, tooling, and look, not rewriting content. A content pass is a follow-up.

What's done

Content

All 16 prose pages converted .md → .qmd, prose preserved: admonitions → callouts, {cite}/{eq} → Quarto citations/equation refs, cross-links, tables, figures.
5 tutorial notebooks render natively (no myst_nb); inline autoclass embeds replaced by links into the API reference.

API reference

Auto-generated by gen_api.py (the sphinx-apidoc analogue): walks dingo and lists every public class/function as its own page — 253 objects across 30 sub-package groups — so new objects appear automatically without a hand-maintained list.
Matches the old Sphinx completeness. quartodoc renders only the class docstring, so dingo's classes (which document constructor args on __init__) came out bare. build_api.py wraps quartodoc's Builder to merge each __init__'s Parameters onto its class — the merge_init_into_class behaviour quartodoc lacks, with no dingo source changes — and include_empty: true lists methods/attributes. Class pages now show a full Parameters table plus their methods.

Look & navigation

Theme: the matched flatly (light) / darkly (dark) Bootswatch pair with a dark-mode toggle.
Native system font stack (no external web-font requests) and a restrained deep-blue "scientific" palette with neutral greys.
Top navbar carries the major sections (Getting Started / Examples / User Guide / API Reference) with a correct per-section active highlight; the full docs tree lives in the left sidebar.

Hosting

.readthedocs.yaml overrides the RTD build with build.commands: install Quarto (pinned 1.9.38), pip install --group docs jupyter (the quartodoc/griffe pins are single-sourced in pyproject.toml), run gen_api.py → build_api.py → quarto render, and publish _site/ to $READTHEDOCS_OUTPUT/html. The API build parses the dingo source statically (ast/griffe), so the docs build never installs dingo — no torch, no GSL.
Hosting on RTD rather than GitHub Pages keeps, for free: versioning (activate tags in the dashboard; readers get the Addons version flyout), PR preview builds (link + changed-files comment on every PR, plus DocDiff visual diffs), and link preservation — and a future custom domain (e.g. docs.dingo-gw.org, for unifying with a larger Dingo site) can be set as canonical later, at which point RTD path-preservingly redirects all old readthedocs.io URLs to it.
The interim GitHub Pages deploy (.github/workflows/docs.yml) is removed: the RTD PR build is the render check, and a second pinned copy of the toolchain would only drift. The gh-pages site is now stale and gets retired once this lands.

Build & deploy

Install Quarto + the doc tooling (pip install --group docs jupyter from the repo root), then from docs/source:

python gen_api.py     # refresh the API object list (the sphinx-apidoc step)
python build_api.py   # generate reference/ pages from docstrings (merges __init__ params)
quarto render         # build the site into _site/   (or: quarto preview)

Read the Docs runs the same three steps via .readthedocs.yaml. The rendered site (_site/) and generated reference/ pages are git-ignored. Also written up in docs/README.md and installation.qmd.

Follow-ups (not blocking)

Content pass — prose is verbatim from the old docs; some pages want updating.
Versioning — no build work needed anymore: activate released versions in the RTD dashboard when wanted.
Executed code blocks — the site currently sets execute: enabled: false (notebooks render from stored outputs). When pages want live-executed {python} blocks — which would need the full dingo runtime — use Quarto freeze: execute locally, commit _freeze/, and CI renders from the frozen results, so the RTD build stays light (still no torch) and build-time limits are a non-issue.
Custom domain — e.g. docs.dingo-gw.org once a main Dingo site exists: one DNS CNAME + a dashboard setting; old readthedocs.io links then redirect automatically.
Undocumented methods (e.g. forward) show a signature but no description; Sphinx filled these from inherited base-class docstrings. Best fixed by writing docstrings in the source, not by tooling.
GitHub Pages retirement — once this lands: unpublish the Pages site (repo Settings → Pages) and delete the gh-pages branch.

cc @nihargupte-ph — would appreciate a look at the overall approach, especially the auto-generated API reference (gen_api.py + build_api.py + quartodoc) and the RTD build.commands recipe. Thanks!

🤖 Generated with Claude Code

Convert the docs in place on the docs-quarto branch (prose preserved verbatim): - Rename all 16 pages .md -> .qmd and convert MyST syntax to Quarto: admonitions -> callouts, {cite:p}/{footcite} -> [@..], {eq} -> @eq- refs, cross-links, tables, figures. - Convert the 5 tutorial notebooks' markdown cells (list-form source so Quarto renders callouts); drop inline autoclass embeds now covered by the API reference; convert the mermaid caption to a Quarto fig-cap. - Add _quarto.yml: ReadTheDocs-style docked left sidebar + minimal top navbar, cosmo + custom.scss theme (tidy links/code blocks), mermaid neutral, bibliography. - gen_api.py auto-discovers the package (the sphinx-apidoc analogue) and writes a comprehensive quartodoc reference: 81 modules / 91 pages. - Replace the Sphinx build instructions in installation.qmd with Quarto ones. Sphinx (conf.py, dingo.*.rst) is left intact; a render allowlist makes Quarto ignore it during the migration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

- Left docked sidebar (ReadTheDocs-style) with the full docs tree; top navbar minimal (Home / API reference / GitHub). - Auto-generate the quartodoc API reference via gen_api.py (the sphinx-apidoc analogue): every public module, regenerated on demand (81 modules / 91 pages). - Notebooks: convert markdown-cell sources to nbformat list-form so Quarto renders the callouts (they were collapsing); restore dropped code-block captions; convert the mermaid :caption: to a Quarto fig-cap. - Code fences: retag unlabeled ``` blocks as default (directory trees), sh (commands), or python, so they render as styled panels instead of plain text. - Fix 9 enumerations that were missing the blank line pandoc requires. - custom.scss: lighter link underlines + center/cap the mermaid flowchart. - installation.qmd: document the gen_api.py -> quartodoc build -> quarto render flow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

API reference: - gen_api.py now lists individual public classes/functions (AST-extracted), so each gets a full-docstring page and a sidebar entry (254 reference pages, was ~82). - Remove two redundant NumPy "Methods" docstring sections (cfnets.PositionalEncoding, samplers.Sampler): quartodoc 0.11.1 raises on a DocstringSectionFunctions, which was silently dropping those classes. quartodoc auto-generates the methods table anyway. Rendering fixes: - Retag unlabeled code fences as default (directory trees) / sh (commands) / python. - Add the blank line pandoc needs before 9 enumeration lists. - custom.scss trimmed to link + mermaid styling. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

This PR drafts an in-place documentation migration from the existing Sphinx/MyST setup to a Quarto + quartodoc website under docs/source, aiming to preserve the current doc content while switching rendering/build tooling and introducing an auto-generated API reference.

Changes:

Converted prose docs from .md/MyST patterns to Quarto .qmd and updated notebooks to use Quarto-native callouts, citations, equation labels, and cross-links.
Added Quarto website configuration (_quarto.yml), styling (custom.scss), and docs build instructions (notably in installation.qmd), plus Quarto-specific gitignore entries.
Introduced gen_api.py to auto-generate quartodoc API sections in _quarto.yml, and adjusted some docstrings to better fit the new API reference output.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
docs/source/_quarto.yml	Adds Quarto website config, nav/sidebar, render allowlist, and quartodoc API sections.
docs/source/gen_api.py	New script to walk `dingo` and regenerate quartodoc `sections` in `_quarto.yml`.
docs/source/custom.scss	Adds Quarto theme tweaks (links + mermaid sizing).
docs/source/.gitignore	Ignores Quarto and quartodoc generated build artifacts under `docs/source`.
docs/source/index.qmd	New Quarto homepage content (replacing the old Sphinx/MyST index).
docs/source/index.md	Removes the old Sphinx/MyST index content and toctree.
docs/source/installation.qmd	Updates documentation build instructions from Sphinx to Quarto/quartodoc workflow.
docs/source/overview.qmd	Updates links and formatting for Quarto rendering, including GNPE section references.
docs/source/quickstart.qmd	Retags code fences (e.g., `sh`) and updates internal links for Quarto.
docs/source/training.qmd	Updates YAML example formatting and converts admonitions to Quarto callouts.
docs/source/inference.qmd	Replaces Sphinx autodoc embeds with links into the quartodoc API reference; updates callouts.
docs/source/gnpe.qmd	Updates citations/figures/callouts for Quarto and links into the quartodoc API reference.
docs/source/result.qmd	Replaces Sphinx autodoc embeds with API-reference links; converts notes/citations for Quarto.
docs/source/dingo_pipe.qmd	Updates INI example formatting, callouts, and internal references for Quarto.
docs/source/sbi.qmd	Converts equation labels/references and updates spacing/formatting for Quarto.
docs/source/code_design.qmd	Converts admonitions and inline API references for Quarto.
docs/source/asimov.qmd	Retags code fences and updates formatting for Quarto.
docs/source/example_toy_npe_model.qmd	Updates links, code fences, and class references for Quarto conventions.
docs/source/example_npe_model.qmd	Updates links, code fences, and callouts for Quarto conventions.
docs/source/example_gnpe_model.qmd	Updates links and code fences for Quarto conventions.
docs/source/example_injection.qmd	Updates inline class references and code fence language for Quarto.
docs/source/waveform_dataset.ipynb	Removes Sphinx `eval-rst` embeds and converts admonitions to Quarto callouts.
docs/source/training_transforms.ipynb	Updates mermaid figure caption syntax, converts admonitions to callouts, and updates cross-links.
docs/source/noise_dataset.ipynb	Converts admonitions to Quarto callouts and removes Sphinx `eval-rst` embeds.
docs/source/network_architecture.ipynb	Updates citations/links and converts notes to Quarto callouts.
docs/source/generating_waveforms.ipynb	Converts note formatting to Quarto callouts and removes Sphinx autodoc embeds.
dingo/core/samplers.py	Removes redundant “Methods” list from `Sampler` docstring (docs output change).
dingo/core/nn/cfnets.py	Removes redundant “Methods” list from `PositionalEncoding` docstring (docs output change).

Comments suppressed due to low confidence (3)

docs/source/overview.qmd:65

The link to the GNPE page uses an empty link label ([](gnpe.qmd)), which will render as missing/blank link text in Quarto/Pandoc output.
docs/source/training.qmd:6
The links to the training-transforms and network-architecture pages use empty link labels ([](...)), which will render as missing/blank link text in Quarto output.
docs/source/gnpe.qmd:6
This sentence uses an empty link ([](#usage)), which will render as missing/blank link text. It should name the target section for readability and correct rendering.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…n_api hygiene Rendering fixes (verified in _site output): - Empty `[](target)` links rendered as invisible `<a></a>` because Pandoc, unlike MyST, does not auto-fill link text from the target's title. Restore the text MyST used to generate: GNPE / usage / data pre-processing / network architecture / ASD dataset / data conditioning. - MyST `(ref:...)=` anchors and `](ref:...)` links rendered as literal text and dead `href="ref:..."` targets. Convert to Pandoc anchors: `[]{#ref-time}` and heading ids `{#asd-dataset}` / `{#window-factor}`, and point the links at them. Accessibility: - Give body/content links a subtle permanent underline (WCAG 1.4.1 — don't distinguish by colour alone); navbar/sidebar/TOC chrome stays clean. gen_api.py: - Use context managers for all file I/O (no leaked descriptors on large trees). - Assert exactly one AUTOGEN-marker substitution via re.subn instead of a substring check that misses a missing END marker. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

quartodoc renders only the class docstring, so dingo's classes (which document constructor args in __init__) lost their Parameters tables and looked bare next to the old Sphinx/readthedocs pages. - build_api.py: thin wrapper around quartodoc's Builder that copies each __init__'s numpydoc Parameters section onto its class before rendering (the merge_init_into_class behaviour quartodoc lacks). No dingo source changes; fixes all 30 affected classes at once. - _quarto.yml: include_empty: true so class pages list their public methods/attributes (e.g. forward, use_cache) even when undocumented. - gen_api.py: point the build instructions at build_api.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Two trailing spaces after 'as specified in' were a Markdown hard line break (<br>), splitting the sentence before the pyproject.toml link. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Leftover trailing whitespace from the .md->.qmd conversion. These occurrences were all end-of-paragraph or code-fence lines (no visible hard break), but removed for tidiness. Mermaid multi-line labels are unaffected (only invisible trailing spaces before the newlines went). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The unescaped '_' inside \text{batch_size} is a subscript operator in MathJax, so the whole expression failed to typeset. Escape it: \text{batch\_size}. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Retire the Sphinx toolchain on this branch. readthedocs keeps serving the old docs from main until the migration lands there; it can be re-added later if we decide to keep it as a second host. - Delete conf.py, docs/Makefile, docs/make.bat, .readthedocs.yaml. - pyproject: drop Sphinx/MyST deps from the dev extra; the docs dependency group is now quartodoc-only. - docs/README.md: Quarto build instructions instead of sphinx-apidoc. - _quarto.yml: the render list no longer guards Sphinx files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Build the Quarto site (gen_api + build_api + quarto render) and publish to the gh-pages branch. PRs run a render-only build check; pushes to docs-quarto render and deploy. Quarto/quartodoc pinned to tested versions; static build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The docs build failed in CI with TypeError: parse_numpy() got an unexpected keyword argument 'allow_section_blank_line' because pip resolved a newer griffe than quartodoc 0.11.1 expects. Pin griffe to the tested version in the workflow and the docs dependency group. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…e, dark mode Redesign the docs appearance toward a clean, professional-scientific look: - Theme: drop cosmo for the matched flatly (light) / darkly (dark) pair, so the two share metrics (fixes the navbar-height mismatch between modes). - Dark-mode toggle via custom-dark.scss. - Typography: native system font stack — no external web font (flatly/darkly Lato imports suppressed via $web-font-path: false). - Palette: deep scientific-blue accent (#1b5a8f light / #6ab0e3 dark) with neutral greys and near-black text; not a vivid indigo. - Navbar: light/dark surface + hairline border, the major sections (Getting Started, Examples, User Guide, API Reference), and a clear active underline. - Sidebar: full docs tree with an accented active item. - Code/callouts: rounded with hairline borders; neutral inline-code chips. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

stephengreen · 2026-07-06T15:50:35Z

Preview here.

Quarto marks the first section's navbar link active on every page when there's a single full sidebar, so 'Getting Started' was permanently underlined. Stop styling Quarto's .active on the navbar, and instead derive the current section from the active sidebar item (or /reference/ URL) in a small script, tagging the matching navbar link with .nav-section-current. Auto-adapts; no hardcoded map. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The generic chrome-link hover rule (a:hover text-decoration underline) put a text-underline under navbar links that didn't line up with the tab-style active bar. Suppress text-decoration on navbar hover; the accent colour change remains the hover cue. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 37 out of 37 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (2)

docs/source/installation.qmd:95

The docs build instructions here say to run quartodoc build, but this PR introduces build_api.py specifically to merge __init__ Parameters into class pages. Using quartodoc build directly will regress class parameter documentation compared to the intended workflow.
docs/source/example_npe_model.qmd:129
The environment variable is misspelled as CUDA_VISIBILE_DEVICES, which users may copy/paste and then fail to select a GPU. The correct variable name is CUDA_VISIBLE_DEVICES.

- installation.qmd: use build_api.py (not quartodoc build) so __init__ params are merged; pin quartodoc/griffe in the install snippet. - example_npe_model.qmd: fix CUDA_VISIBLE_DEVICES typo (was CUDA_VISIBILE). - waveform_dataset.ipynb: 'which can used' -> 'which can be used'. - noise_dataset.ipynb: 'Guassian' -> 'Gaussian'. - docs.yml: correct the now-stale header comment (branch is Quarto-only; RTD still builds from main). - pyproject.toml: pin quartodoc==0.11.1 (build_api.py hooks its 0.11.1 internals). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Quarto emits `code { background: #f8f9fa }` into both the light and dark stylesheets (it isn't theme-aware), so inline code not covered by the old `p code, li code, td code` override — the right-margin TOC, headings, and quartodoc `dd` parameter descriptions — showed a near-white box in dark mode. Reset that base fill to transparent and broaden the chip to all article-body inline code (`main :not(pre) > code:not(.sourceCode)`), symmetrically in both themes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Hosting on RTD keeps versioned docs, pull-request preview builds, and the existing dingo-gw.readthedocs.io URLs — features the Pages deploy lost. - .readthedocs.yaml: override the build with build.commands (install Quarto 1.9.38, pip install --group docs jupyter, gen_api -> build_api -> render, copy _site to $READTHEDOCS_OUTPUT/html). No dingo install needed: the API build parses the source statically. - Delete .github/workflows/docs.yml: the RTD PR build replaces the render check, and a second pinned toolchain would only drift. - site-url -> dingo-gw.readthedocs.io; install instructions now use the single-sourced docs dependency group. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

stephengreen and others added 2 commits July 2, 2026 14:46

stephengreen requested a review from nihargupte-ph July 2, 2026 13:35

stephengreen requested a review from Copilot July 5, 2026 17:33

Copilot started reviewing on behalf of stephengreen July 5, 2026 17:33 View session

Copilot AI reviewed Jul 5, 2026

View reviewed changes

stephengreen and others added 9 commits July 6, 2026 00:33

Docs: fix stray line break in installation.qmd

1f34dab

Two trailing spaces after 'as specified in' were a Markdown hard line break (<br>), splitting the sentence before the pyproject.toml link. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Docs: fix batch_size math rendering (escape underscore in \text)

1fc0d71

The unescaped '_' inside \text{batch_size} is a subscript operator in MathJax, so the whole expression failed to typeset. Escape it: \text{batch\_size}. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

stephengreen and others added 2 commits July 6, 2026 20:05

stephengreen marked this pull request as ready for review July 6, 2026 21:54

stephengreen requested a review from Copilot July 6, 2026 21:55

Copilot started reviewing on behalf of stephengreen July 6, 2026 21:56 View session

Copilot AI reviewed Jul 6, 2026

View reviewed changes

Comment thread .github/workflows/docs.yml Outdated

Comment thread pyproject.toml

Comment thread docs/source/waveform_dataset.ipynb

Comment thread docs/source/noise_dataset.ipynb

stephengreen and others added 3 commits July 6, 2026 23:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Migrate documentation from Sphinx to Quarto#391

Migrate documentation from Sphinx to Quarto#391
stephengreen wants to merge 17 commits into
hackathon-1from
docs-quarto

stephengreen commented Jul 2, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stephengreen commented Jul 6, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

stephengreen commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What's done

Build & deploy

Follow-ups (not blocking)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stephengreen commented Jul 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

stephengreen commented Jul 2, 2026 •

edited

Loading