Skip to content

Use SciMLTesting public API docs QA - #475

Merged
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:public-api-doc-coverage
Jul 13, 2026
Merged

ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:public-api-doc-coverage

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Jul 11, 2026 •

Copy link
Copy Markdown
Member

Ignore this PR until reviewed by @ChrisRackauckas.

Summary

  • Deletes the repo-local test/qa/public_api_docs.jl scanner.
  • Uses SciMLTesting's shared rendered public-API docs check via run_qa(...; api_docs_kwargs = (; rendered = true, rendered_ignore = REEXPORTED_API)).
  • Raises only the isolated QA environment's SciMLTesting compat floor to 2.1.

Validation

  • Runic --check test/qa/qa.jl passed.
  • GROUP=QA julia +release --project=. -e 'using Pkg; Pkg.test()' passed: Quality Assurance 20/20 and JET static analysis 75/75.

@ChrisRackauckas-Claude
ChrisRackauckas-Claude force-pushed the public-api-doc-coverage branch 2 times, most recently from e43e744 to b855218 Compare July 12, 2026 23:09
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Updated the QA compat to SciMLTesting 2.2 in both test environments. Local validation passed on Julia 1.12 with GROUP=QA timeout 3600 julia +1.12 --startup-file=no --project=. -e "using Pkg; Pkg.test()": Quality Assurance 20/20 and JET static analysis 75/75.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Fixed the downgrade failure by raising the test OrdinaryDiffEq lower bound to 7.1.2. The failed CI downgrade was resolving OrdinaryDiffEqCore v4.3.0 with ImplicitDiscreteSolve v2.1.1; using ImplicitDiscreteSolve reproduces too many parameters for type at ImplicitDiscreteSolve/src/controller.jl:50 before DiffEqGPU code runs. With OrdinaryDiffEq v7.1.2, the resolver selects OrdinaryDiffEqCore v4.6.0 and ImplicitDiscreteSolve v2.1.2, which imports cleanly.\n\nLocal validation:\n- timeout 3600 julia +1.10 --project=.scratch/... -e "... Pkg.add([OrdinaryDiffEq@7.1.2, ModelingToolkit@11.24.1]); using OrdinaryDiffEq, ModelingToolkit": passed\n- GROUP=QA timeout 3600 julia +1.12 --startup-file=no --project=. -e "using Pkg; Pkg.test()": QA 20/20 and JET 75/75 passed\n- git diff --check: passed

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Updated this PR to use SciMLTesting = "2.2" in both QA/test environments.

The reason it was not enough to only bump SciMLTesting is that the previous follow-up pin OrdinaryDiffEq = "7.1.2" made the downgrade job incompatible with downgraded SciMLLogging v1.10.1. I reverted the test compat to OrdinaryDiffEq = "7" and verified the downgrade resolver/import path on Julia 1.10 with:

  • SciMLTesting v2.2.0
  • OrdinaryDiffEq v7.0.0
  • SciMLLogging v1.10.1
  • ImplicitDiscreteSolve v2.1.3

I also raised the test OpenCL floor to 0.10.3; the local resolver selected OpenCL v0.10.9, and the OpenCL mass-matrix target passed. The previous Julia 1.12 OpenCL failure reproduced with OpenCL v0.10.2 as a pointer arithmetic issue in OpenCL.jl, so this avoids that broken test dependency set.

Local verification run before pushing fe6228a:

  • GROUP=QA JULIA_PKG_PRECOMPILE_AUTO=0 timeout 3600 julia +release --startup-file=no --project=. -e 'using Pkg; Pkg.test()' -> QA 19/19, JET 75/75
  • GROUP=OpenCL JULIA_PKG_PRECOMPILE_AUTO=0 timeout 3600 julia +release --startup-file=no --project=test -e 'using Pkg; Pkg.develop(PackageSpec(path=pwd())); Pkg.instantiate(); using Test; include("test/gpu_kernel_de/stiff_ode/gpu_ode_mass_matrix.jl")' -> passed
  • Julia 1.10 downgrade resolver/import check with the package developed locally -> passed
  • git diff --check -> passed
  • Runic --check test/Project.toml test/qa/qa.jl -> passed

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Updated this branch so the QA environments use SciMLTesting = "2.2" and added the compat floors needed for the strict downgrade job to keep a compatible OrdinaryDiffEq stack:

  • root RecursiveArrayTools compat is now 4.2, matching the OrdinaryDiffEqCore >= 4.4 compat requirement.
  • test/Project.toml now declares OrdinaryDiffEqCore = "4.4" directly so the downgraded test environment does not resolve the old OrdinaryDiffEqCore v4.3 / ImplicitDiscreteSolve combination.

Local checks run on commit 33a3c44:

  • Exact downgrade workflow reproduction passed:
    • timeout 3600 julia +1.10 --startup-file=no .scratch/julia-downgrade-compat/downgrade.jl ... "." "alldeps" "lts" "Mooncake"
    • GROUP=CPU timeout 3600 julia +1.10 --startup-file=no --project=. -e 'using Pkg; Pkg.test(; coverage=true, julia_args=["--check-bounds=yes", "--compiled-modules=yes", "--depwarn=yes"], force_latest_compatible_version=false, allow_reresolve=false)'
    • Result: Testing DiffEqGPU tests passed.
  • QA passed with SciMLTesting v2.2.0:
    • GROUP=QA timeout 3600 julia +release --startup-file=no --project=. -e 'using Pkg; Pkg.test()'
    • Result: Quality Assurance | 19 pass / 19 total; JET static analysis | 75 pass / 75 total.
  • git diff --check passed.
  • Runic: no Julia files changed in this commit.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Updated the branch to restore the QA compat floor to SciMLTesting = "2.1" in test/Project.toml and test/qa/Project.toml.

Local validation:

  • git diff --check passed.
  • GROUP=QA timeout 3600 /home/crackauc/.juliaup/bin/julia +release --startup-file=no --project=. -e 'using Pkg; Pkg.test()' passed: Quality Assurance | 19 pass / 19 total, JET static analysis | 75 pass / 75 total.
  • Exact floor smoke with SciMLTesting v2.1.0 passed after resolver downgrade v2.2.0 => v2.1.0: Quality Assurance | 19 pass / 19 total.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Pushed a cleanup commit to narrow the QA/docs PR back away from root package compat changes. The package-level compat floors for CommonSolve, RecursiveArrayTools, and SciMLBase are restored to master; the SciMLTesting public API docs migration remains in the test/QA surface.

Local validation after this change:

git diff --check
# passed

GROUP=QA timeout 3600 julia +release --startup-file=no --project=. -e 'using Pkg; Pkg.test()'\n# Quality Assurance | 19/19\n# JET static analysis | 75/75\n# Testing DiffEqGPU tests passed\n\ntimeout 3600 julia +release --startup-file=no --project=test/qa -e 'using Pkg; Pkg.develop(PackageSpec(path=pwd())); Pkg.add(PackageSpec(name="SciMLTesting", version="2.1.0")); Pkg.instantiate(); include(joinpath("test", "qa", "qa.jl"))'\n# resolver forced SciMLTesting v2.2.0 => v2.1.0\n# Quality Assurance | 19/19\n```\n\nI also checked the remaining red Buildkite state before this push: GitHub Actions were passing; the remaining failure was the external Buildkite AMDGPU matrix on commit 01836e0. Buildkite exposes the failed AMDGPU step metadata publicly, but the raw job log requires authenticated Buildkite API access from this environment, so I could not inspect its stacktrace here.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Pushed a follow-up that restores the package compat floors that the downgrade workflow actually needs:

  • CommonSolve = "0.2.6"
  • RecursiveArrayTools = "4.2"
  • SciMLBase = "3.10"

The failed downgrade job had downgraded root to SciMLBase v3.1.0, then failed against test/Project.toml's SciMLBase = "3.10" requirement. The prior passing downgrade job used SciMLBase v3.10.0, CommonSolve v0.2.6, and RecursiveArrayTools v4.2.0, so this restores that state while keeping the shared SciMLTesting QA setup.

Local validation run:

GROUP=QA julia +release --project=. -e 'using Pkg; Pkg.test()'
Quality Assurance | 19/19
JET static analysis | 75/75
julia +release --project=test/qa -e 'using Pkg; Pkg.develop(PackageSpec(path=pwd())); Pkg.add(PackageSpec(name="SciMLTesting", version="2.1.0")); Pkg.instantiate(); include(joinpath("test", "qa", "qa.jl"))'
SciMLTesting v2.2.0 => v2.1.0
Quality Assurance | 19/19

Also ran the CI-equivalent downgrade path in a throwaway worktree:

julia +1.10 ../../tools/julia-downgrade-compat-v2/downgrade.jl <effective stdlib skip list> . deps 1 Mooncake
GROUP=CPU julia +1.10 --project=@. -e 'import Pkg; Pkg.test(; coverage=false, julia_args=["--check-bounds=yes", "--compiled-modules=yes", "--depwarn=yes"], force_latest_compatible_version=false, allow_reresolve=false)'

That resolved the expected downgraded stack (SciMLBase v3.10.0, CommonSolve v0.2.6, RecursiveArrayTools v4.2.0, SciMLTesting v2.2.0) and passed the CPU tests locally, including the previously failing downgrade setup.

Replace the local public API docs scanner with SciMLTesting's rendered API docs check, and keep the change scoped to the QA docs environment.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review July 13, 2026 10:11
@ChrisRackauckas
ChrisRackauckas merged commit c4dff14 into SciML:master Jul 13, 2026
17 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants