Skip to content

docs: JOSS paper draft - #621

Merged
ChrisRackauckas merged 5 commits into
SciML:masterfrom
sathvikbhagavan:sb/joss
Nov 4, 2026
Merged

ChrisRackauckas merged 5 commits into
SciML:masterfrom
sathvikbhagavan:sb/joss

Conversation

@sathvikbhagavan

@sathvikbhagavan sathvikbhagavan commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

No description provided.

@sathvikbhagavan

Copy link
Copy Markdown
Member Author

@ChrisRackauckas, can you take a look at this?

@ChrisRackauckas

Copy link
Copy Markdown
Member

🤖 Automated review from an AI agent running as @ChrisRackauckas — not written or reviewed by Chris. It is posted so that you can act on it. Chris or a maintainer may disagree.

Verdict: changes needed before merge (reviewed at ccbe196).

Adds a JOSS paper draft (joss/paper.md, joss/paper.bib) for Surrogates.jl, with you as last author. The bibliography is clean, but the paper leaves out four sections JOSS now requires, and it makes two false claims about the API. Fix those before submission; merging the files as they stand is harmless, since no code changes.

Risk assessment

  • Risk: low. The PR only adds files under joss/ and doesn't touch package code, tests, docs build or Project.toml.
  • Blast radius: none for users. The risk is to the JOSS submission: it would be bounced at pre-review for missing required sections.
  • Evidence:
    • CI on head ccbe196: all 13 checks pass, including Core (1, lts, pre, x86), QA, Downgrade, docs build, Runic, typos and both benchmark jobs. With no failures, there was no need to compare against master.
    • Local checks (artifacts in work/SciML__Surrogates.jl__621/scratch/jossbuild/):
      • pandoc --citeproc → HTML ran with zero warnings.
      • All 41 keys cited in paper.md match the 41 entries in paper.bib: none missing, none unused, no duplicates.
      • All 33 DOIs resolve (HTTP 302 from doi.org).
      • Prose is ~876 words excluding the table.
    • Not verified: the official inara PDF build. The Docker daemon wasn't running and there's no pdflatex on this machine.
    • Claims in the paper checked against src/, ext/, docs/src/ and test/AD_compatibility.jl at the PR head (see findings).
    • Current JOSS format requirements read at https://joss.readthedocs.io/en/latest/paper.html.
    • No other maintainer has reviewed it. The only comment is the author asking @ChrisRackauckas to take a look.
  • Merge: needs changes. JOSS required sections are missing (1), plus two factual errors (2, 3) and metadata gaps (4–6).

Findings

  1. joss/paper.md (structure) — The paper doesn't follow the current JOSS format. JOSS now requires these sections: Summary, Statement of need, State of the field, Software design, Research impact statement, AI usage disclosure, Acknowledgements, References. Length is now 750–1750 words. Four are missing: "Software design", "Research impact statement" and "AI usage disclosure" are absent, and "Comparison with existing software" (line 57) needs renaming to "State of the field". That section also needs the explicit "build vs. contribute" justification JOSS asks for. The fixes:
    • Software design: trade-offs such as the SurrogatesBase abstract interface, package extensions for heavy backends, Optimization.jl-based theta fitting, and the ask/tell batch API.
    • Research impact: concrete downstream users or publications, e.g. the SciML packages that depend on Surrogates.jl and papers that cite or use it.
    • AI disclosure: it cannot say "none". The git history includes 13 commits by ChrisRackauckas-Claude, 27 by "Chris Rackauckas - Beep Boop Edition", and 91 commit-message lines mentioning Claude. It should also say whether AI tools helped draft the paper itself.
    • Adding all of this stays well under 1750 words.
  2. joss/paper.md:53 — "surrogate_optimize!(...) drives the loop, optionally proposing a batch per iteration" is false. None of the surrogate_optimize! methods in src/Optimization.jl accepts a batch or ParallelStrategy argument. Batching is a separate ask/tell API, potential_optimal_points(alg, strategy, lb, ub, surrogate, sampler, n_parallel) (src/Optimization.jl:635–870), and works only with SRBF, EI and LCBS (docs/src/parallel.md:17,29). The Summary's batch sentence (line 42) should also say it covers SRBF/EI/LCBS only. Fix: describe batching as the potential_optimal_points + update! ask/tell loop.
  3. joss/paper.md:52 — "all built-in surrogates share this convention [Kriging(x, y, lb, ub)] regardless of … response type (scalar- or vector-valued)" is false in two ways:
    • Signature: KPLS/KPLSK are (x, y, n_comp, lb, ub, theta) (src/KPLS.jl:208, src/KPLSK.jl:125), and GEKPLS is (x, y, grads, lb, ub; …) (src/GEKPLS.jl:177).
    • Response type: Wendland rejects vector-valued responses (src/Wendland.jl:27). docs/src/index.md:66 lists only RadialBasis, Neural, SecondOrderPolynomial, InverseDistance, Lobachevsky and AbstractGPs as multi-output (Linear also supports it).
    • Fix: "most surrogates follow Surrogate(x, y, lb, ub; kwargs...); gradient-enhanced and PLS variants take extra arguments; a subset supports vector-valued outputs."
  4. joss/paper.md:25 — The affiliation "Google Summer of Code Contributor" is not an institution. Use each author's real affiliation, or "Independent Researcher". Bessi and Narayan also have no ORCID; JOSS strongly encourages them.
  5. joss/paper.md:74 — JOSS requires acknowledgement of financial support. The Acknowledgements section mentions no funding: not Google Summer of Code, which funded the student contributors, and not the grants usually credited on SciML papers.
  6. joss/paper.bib — adams2020dakota (techreport) has no DOI. Sandia/OSTI reports usually have a 10.2172/… DOI, so add it if one exists. The other 7 DOI-less entries (NeurIPS/ICML papers and Julia software misc entries) are fine.
  7. Authorship (for the authors to decide, not a defect) — By commit count, ludoro/LudovicoBessi is the largest contributor (412 commits across three identities: 239 ludoro + 129 LudovicoBessi + 44 Ludovico Bessi; vs. Sathvik's 133 before this PR and Chris's 386 as "Christopher Rackauckas"), and he is listed second. Other substantial contributors aren't listed, e.g. Thomas Marks, marcoq, Ranjan Anantharaman, Rohit Singh Rathaur. JOSS leaves this to the authors, but the order and inclusion should be a deliberate choice.
  8. Minor:
    • \autoref{tab:comparison} renders blank in non-LaTeX previews. It should resolve under inara, but I couldn't confirm that here.
    • The table uses "LCBS" (the package's type name) while the prose uses "lower confidence bound". Either is fine, but pick one.
    • The claim that it "is the only Julia package spanning…" is unsupported beyond the comparison table. Soften it or keep it tied to the table.
    • The table claims (SMT multi-objective "no", BoTorch GEK "no", etc.) were not independently verified.

Push a fix and the PR is reviewed again automatically at the new head.


🤖 Posted by an AI agent — harness: Claude Code · model: claude-opus-5-5[1m] (fleet master); review by Devin CLI (Mac) / fusion-claude-opus-5-5-high-sidekick-swe-2-medium
Conversation: local Claude Code session 3cd6500a-1f81-46b5-ac0b-c466e15b6a53 on Chris's Mac (session ID, no URL)

Comment thread joss/paper.md Outdated

# Statement of need

Surrogate modeling and surrogate-based optimization are used across aerospace design, hyperparameter tuning, and simulation-based engineering, and mature toolkits exist outside Julia [@bouhlel2019python; @saves2024smt; @eriksson2019pysot]. Julia lacked a single package spanning this range under one interface, so users comparing, say, Kriging against radial basis functions against a neural network on the same problem had to learn several APIs or hand-assemble packages such as `GaussianProcesses.jl` [@GaussianProcessesjl] and `Flux.jl` [@Fluxjl] around their own optimization loop.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But nothing connects to differentiability: the core need of Surrogates.jl is that you need not just surrogates but things that connect to acausal modeling (ModelingToolkit, Modelica, etc.) and are also differentiable. There's no mature surrogate packages since nothing has this, just isolated things that aren't generally useful in composable modeling contexts

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I think we need to have a good example in the docs to demonstrate integration with MTK to prove this point. What kind of example can be good for demonstrating this?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build a surrogate of some PDE and have it drive the input to some circuit model

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added it in aa0804a
But need this for CI SciML/PolyChaos.jl#199

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A gentle bump on this @ChrisRackauckas

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tagged

@sathvikbhagavan

Copy link
Copy Markdown
Member Author

A reminder on this @ChrisRackauckas
I have added an example of integrating with MTK and updated the paper

@ChrisRackauckas
ChrisRackauckas merged commit adeed82 into SciML:master Nov 4, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants