Skip to content

Latest commit

 

History

History
415 lines (269 loc) · 93.8 KB

Design.md

File metadata and controls

415 lines (269 loc) · 93.8 KB

What is SED2?

SED stands for Simulation Experiment Description. SED2 is a language for recording simulation experiments in JSON form that can be translated and exported to a number of different systems, where the core simulations and other analyses can be performed by a variety of tools. It can be viewed as a generic workflow language, where instead of specific commands to a particular operating system and environment, it instead contains abstract commands that could be executed on a variety of operating systems and environments.

SED-ML was an earlier language (Simulation Experiment Description Markup Language), was XML-based, and tightly constrained. The goal of SED2 is to change the format to JSON instead of XML, and expand the scope of what is possible to describe.

SED2 is designed to facilitate reproducibilty for the biological modeling community, much like SBML was desgned to facilitate model exchange for the biological modeling community. In 20 years, the same SED2 document should be readable by a new library using new simulators and programs that don't exist today, and they should still produce the same results.

Ultimately, a SED2 'orchestrator' should be able to read a SED2 file, figure out who to ask to do each task, then stitch everything together to perform an efficient execution of all the tasks described in the document.

The SED2 specification originated as a Google Doc (with UML diagrams as Google Draw documents). This monolithic document has since been split into core-spec.md - a short document describing the overall shape and design of a SED2 document - plus a specsheets/ directory of per-class "Data Sheets," one per Task, Output, and supporting class, organized into core/, tasks/, outputs/, and auxiliary/ categories. Each Data Sheet lives in its own versioned folder (specsheets/<category>/<ClassName>/v1.0.0/, with a change to that class getting a sibling folder named after the new document version - see Versioning) and contains a standalone, cross-referencing JSON Schema file, a UML diagram (or a DIAGRAM-PENDING.md placeholder where none exists yet - see core-spec.md Section 10 for the current list), and a description.md explaining what the class does and its attributes/outputs. This split - core-spec.md plus specsheets/ - is the canonical version of the spec. Future spec changes should be made directly against core-spec.md/specsheets/.

General Strategy

We want a code generator that produces three self-contained libraries - one each for C++, Java, and Python - that read and validate SED2 files, and can be used to create or otherwise manipulate a SED2 file. Each generated library is a complete, independent codebase in its own language; none of them depend on each other or on the generator at runtime. The SED2 design may change, and it definitely will expand with time, so the generator itself also needs to be flexibly changeable.

In a future project, we will use these libraries to write a translator of SED2 to a python script, a Julia script, and a Process Bigraph JSON file. These libraries are constrained to serve as the base for those future translators.

Testing

Everything should be test-driven. We need tests for valid and invalid SED2 documents, and for each function (i.e. get/set), in each of the three generated libraries. We will use each language's standard test framework: GoogleTest for C++, the same way that libantimony does (C:\Users\Lucian\Desktop\antimony); JUnit for Java; and pytest for Python.

This section covers document-level fixtures (fixtures/), which exercise the generated libraries - the same real specsheets/ classes a document author actually uses. A separate, smaller fixture tree, test-specsheets/ (see Repository Layout), exercises the generator itself: one or two synthetic examples of each schema-composition pattern specsheets/ uses (the x-generated-oneOf discriminator, a directly-instantiable-and-subclassed Common mixin, a standalone schema-only mixin, a namespace new/ and updated/ example, and so on), kept deliberately small and stable so generator changes can be tested without depending on the size or churn of the real spec. See test-specsheets/README.md for the full list of patterns it covers. test-specsheets/fixtures/ carries that tree's own document-level fixtures - generate the three libraries from test-specsheets/ and run them the same way the real fixtures/ tree below exercises the generated libraries against the real spec, just scoped small enough to test a generator change in isolation. Same naming convention, same pass/fail semantics; see test-specsheets/fixtures/README.md for the one exception (six _type-discriminator-const rules that can only be tested against their own class's schema directly, not embedded through a parent discriminator - see Schema-Pass Errors below for why).

Document-level tests - valid and invalid SED2 documents, and the validation errors an invalid one is expected to produce - live in a single, language-independent directory of input files and expected outcomes. This directory sits outside the per-language template and generated directories described under Code Generation below, and all three test suites read from it rather than keeping their own copies, so the three libraries can never drift out of sync on what counts as valid, or on what error a given invalid document should produce.

Fixture files are named after the validation rule(s) they exercise, adapted from the convention libSBML's test suite uses: <rule-id>-<pass|fail>-<count>-<test>[-<rule-id-2>-<count-2>...].sed2.json (see Repository Layout for the extension). The leading rule ID identifies the constraint under test; pass or fail says whether the document should validate; count is how many times that rule is expected to fire (00 for a pass case); and test distinguishes multiple fixtures for the same rule. A document crafted to fail more than one rule at once chains additional <rule-id>-<count> pairs onto the end - for example ExplicitODESimulation-0001-fail-01-02-BoundedODESimulation-0002-01 fails rule ExplicitODESimulation-0001 once and rule BoundedODESimulation-0002 once. Unlike libSBML's convention, every chained rule carries its own count, not just the first, so a fixture can exercise interacting rules that each fire more than once. Fail-case fixtures are checked against the specific rule IDs and counts encoded in the filename, not just a total error count, and each fail case should violate only the rule(s) named in its filename so that check stays meaningful. Severity doesn't change any of this: a warning-severity rule counts and fails exactly like an error-severity one, so a fixture crafted to trip only a warning-severity rule is still a fail fixture, named and counted the same way.

One rule is deliberately excluded from this chaining: SEDDocument-0012 (duplicate JSON keys) isn't implemented by any of the three generated libraries in v1 (see its own validation file), so it should never appear chained onto another rule's fixture filename. Beyond the fact that nothing would actually detect it, a duplicate key's own "last value wins" resolution can silently overwrite whatever content a paired fixture relies on to trigger its other rule, so combining the two would make that fixture's intent unreliable even by accident.

Pass-case fixtures need no separate expected-output file: the check is that parsing and re-serializing reproduces the same document. That comparison is done as JSON values - same keys, same values, same key order, checked recursively - rather than as literal text, since the three languages' serializers won't agree on whitespace or formatting. Key order matters throughout, not just for tasks: SED2 relies on it so a naive executor can walk the document top-to-bottom without needing a value before it's produced, so round-tripping has to preserve it exactly (see JSON Libraries).

Numeric values compare by value, not by JSON literal form - 1, 1.0, and 1e0 are the same double and must match. Beyond that, equality itself isn't exact: a double round-tripped through three different languages' JSON serializers isn't guaranteed to land on the same bit pattern, so numeric comparison anywhere in the generated libraries - round-trip fixtures here, and any future validation rule or generated code that needs to compare two computed or resolved numbers - uses one standard relative tolerance rather than bitwise equality, shared rather than reinvented per spot. Two numbers a and b are considered equal when |a - b| <= max(RTOL * max(|a|, |b|), ATOL), with RTOL = 1e-6 and ATOL = 1e-12 (the absolute floor only matters near zero, where a purely relative comparison is meaningless). RTOL is deliberately permissive - about six significant digits, far looser than a double's own ~15-17 - since the only thing this tolerance needs to absorb is incidental serialization noise between languages, not real precision loss; an actual bug should still overwhelm it.

Test Generation and Spec Evolution

Fixtures split into two tiers, maintained differently, so that spec changes don't force a wholesale revamp of the test suite.

Schema-derivable fixtures - for constraints the JSON schema captures directly, like required fields, types, enums, and patterns - are generated mechanically by the same generator run that produces the class code (see Code Generation below), into their own generated directory alongside it. A required field implies a missing-it fixture; an enum implies one fixture per invalid value outside it; and so on. These are never hand-edited and can't go stale: whenever the schema changes, they regenerate along with everything else.

Semantic fixtures - for the hand-written numbered rules described under Validation, such as cross-reference resolution, math well-formedness, and interactions between rules - stay hand-authored, since a machine has no way to know which cases are actually interesting to test. To keep spec evolution from forcing these to be rewritten, rule numbers are append-only and never silently redefined: if an existing numbered rule's real behavior needs to change, that mints a new rule number and deprecates the old one, rather than editing the old number's meaning in place. This append-only policy is enforced per class, inside that class's own validation/ folder (see Rule Documentation under Validation) rather than through any centralized registry. A deprecated rule's fixtures move to an archive directory, kept for history but excluded from the active suite, rather than being deleted or rewritten. Under this discipline, most spec growth - including behavioral changes, not just clean additions - becomes writing new fixtures for a new or superseded rule number, which is ordinary incremental work rather than a revamp.

This append-only, never-delete discipline is a release-stability guarantee, and like every release-stability guarantee in this document, it only has to hold from SED2's first actual release onward. Pre-release, nothing outside this repository depends on a rule number, a fixture, or a schema shape staying put, so a rule that turns out to be redundant or wrong is deleted outright - its file removed, its fixtures removed - rather than marked deprecated and archived. Once SED2 reaches its first real release, this reverts to strict deprecate-and-supersede, since an outside consumer may by then depend on what a given rule number means.

Pre-release deletion leaves numbering gaps behind - a rule ID removed outright, never refilled, per the discipline above. The last step before SED2's first real release is a one-time renumbering pass: close every gap in every class's validation/ folder, so each sequence is dense with nothing skipped before the append-only, never-delete discipline above takes over for good. This pass is itself pre-release-only - it never recurs, since after release a gap left by a deprecation is exactly what the discipline above requires staying open.

The one case that legitimately requires editing an existing fixture is a genuine bug fix to a rule that was implemented incorrectly - normal, bounded maintenance, not something spec evolution should routinely cause. CI running on every push (see CI below) is the backstop either way: a rule whose behavior quietly shifted shows up immediately as a named, specific failure - a rule number and test number - rather than silent drift discovered later.

Because the fixture directory is shared across all three generated languages (see Testing above), hand-authoring a semantic fixture is a cost paid once, not three times.

Writing fixtures/'s hand-authored semantic tier for the real spec is deliberately deferred, not started alongside the generator. Schema-derivable fixtures cost nothing extra, since the generator produces them automatically; a semantic fixture is real, one-time authorship effort per rule, and the spec is still changing. The append-only discipline above already bounds the rewrite cost once these exist - only a rule whose own behavior changes needs new fixtures, never the whole tree - but starting that authorship now, before the spec has settled, would still mean writing a substantial number of fixtures for rules likely to be renumbered or reworked before release. Semantic fixtures for the real spec get written once the spec is stable enough that doing so is ordinary incremental work rather than a moving target. This doesn't block generator work: the generator's own correctness is exercised by test-specsheets/fixtures/ (see Testing above) in the meantime, independent of the real spec's own size or churn.

CI

GitHub Actions runs the pipeline on every push, in two stages:

  1. A single 'generate' job runs the code generator once and uploads the three generated-library directories (see Code Generation below) as build artifacts, so every language is built and tested against the exact same generator run rather than each regenerating independently.
  2. A matrix job, one per language (C++, Java, Python), downloads its generated directory, compiles it, and then runs that language's test suite (GoogleTest, JUnit, pytest) against it, using the shared fixture directory described under Testing above.

Any job failing fails the whole run. Because the matrix jobs are separate, a compile or test failure is attributed to the specific language and stage that broke, and a break in one target's generated code can't hide a break in another's.

Releases

Once a language's test job has passed against the real spec (for JavaScript, test-js-main, which builds the module with emsdk and runs the same fixture suite under Node), a matching release job packages that language's library, smoke-tests the packaged artifact itself (not the source tree), and uploads it as a workflow artifact - on every CI run, so a packaging break is caught on the push that caused it:

  • Python (release-python): one pure-Python wheel, libsed2-<version>-py3-none-any.whl, installed into a clean virtualenv and used to load and validate every fixtures/handwritten/pass-* fixture.
  • Java (release-java): libsed2j-<version>.jar, plus -sources and -javadoc jars (from the generated pom.xml's release profile) and the pom itself.
  • JavaScript (release-js): libsed2js-<version>.tgz, an npm package (wrapper, TypeScript declarations, and the module), and libsed2js-<version>.js, the same single-file module on its own for use from a script tag. The tarball is installed into a clean project and used to validate every fixtures/handwritten/pass-* fixture.
  • C++ (release-cpp): libsed2-cpp-<version>.tar.gz and .zip, the self-contained CMake project (headers, the few ANTLR .cpp files, CMakeLists.txt, LICENSE, VERSION.txt). The library is header-only apart from the math parser and fetches its dependencies with FetchContent, so the release is source, not prebuilt binaries. The smoke test (templates/cpp/release-smoke/) extracts the archive and consumes it through add_subdirectory() and libsed2::libsed2.

All three take their version from VERSION.txt at the repository root, which is the version of the libraries and is unrelated to the SED2 document-format version (see Versioning). generator/generate.py reads it (or takes --version) and stamps it into pyproject.toml, pom.xml, and CMakeLists.txt, so the committed generated/ trees stay in step with it; the format is strictly MAJOR.MINOR.PATCH, the one shape valid unchanged in PEP 440, Maven, and CMake. CI never creates or edits a GitHub release and is not tied to git tags. To release, bump VERSION.txt, let CI run, download the release-python, release-java, and release-cpp artifacts from that run's summary page (kept 30 days), and attach the files to a release by hand; to update a release, replace its files with those from a newer run. Publishing to PyPI or Maven Central is not done by CI.

Repository Layout

SED2/
|-- Design.md
|-- Claude.md
|-- README.md
|-- LICENSE
|-- VERSION.txt                      version of the generated libraries (MAJOR.MINOR.PATCH) - see CI's Releases
|-- core-spec.md                     canonical spec: overall shape/design of a SED2 document
|-- specsheets/                      canonical per-class Data Sheets (schema + diagram + description)
|   |-- core/, tasks/, outputs/, auxiliary/
|   |   `-- <ClassName>/v1.0.0/      schema.json (+ inline./common.schema.json where merged), <ClassName>.png or DIAGRAM-PENDING.md, description.md, validation/*.md, outputs.json (tasks/ concrete classes only - see core-spec.md Section 8)
|-- test-specsheets/                 synthetic generator-test fixtures, same shape as specsheets/ + namespaces/ - see README.md there and Design.md's Testing section
|   `-- fixtures/                    document-level fixtures for that synthetic tree (see its own README.md) - same convention as fixtures/ below, scoped small
|-- tools/
|   `-- build_specification.py       assembles core-spec.md + specsheets/ into SPECIFICATION.md on demand (see Specification Assembly)
|-- namespaces/                      known third-party namespace extensions, composed into the schema at generate time (see Namespaces)
|   `-- <prefix>/
|       |-- new/                     wholly new classes this namespace defines
|       |   `-- core/, tasks/, outputs/, auxiliary/
|       |       `-- <ClassName>/<namespace-version>/   schema.json, description.md, validation/*.md (IDs: <prefix>-<ClassName>-NNNN)
|       `-- updated/                 attributes this namespace adds to an existing class
|           `-- core/, tasks/, outputs/, auxiliary/
|               `-- <ClassName>/<namespace-version>/   schema.json, description.md, validation/*.md (IDs: <ClassName>-<prefix>-NNNN)
|-- schema/
|   |-- predefined-functions.json    math function/constant registry (MathML + distrib + SED2-specific)
|   |-- predefined-functions-meta.schema.json   validates predefined-functions.json's envelope (see Math)
|   `-- outputs-meta.schema.json   validates every tasks/ class's outputs.json envelope (see core-spec.md Section 8)
|-- generator/                       the generator's own hand-written source
|-- templates/
|   |-- cpp/                         hand-written C++-only code
|   |   `-- release-smoke/           consumer project CI uses to smoke-test the C++ release package
|   |-- js/                          hand-written JavaScript-target code: src/api.cpp (the fixed Emscripten entry points), index.js (wrapper), tests/fixtures.test.js
|   |-- java/
|   |   `-- tests/                   hand-written JUnit fixture-harness glue
|   `-- python/
|       `-- tests/                   hand-written pytest fixture-harness glue
|-- generated/
|   |-- cpp/                         fully self-contained, generated C++ library
|   |-- java/
|   |-- python/
|   |-- js/                          the C++ library compiled to WebAssembly with Emscripten (see Code Generation); generated sources only, the module is a build product
|   |-- schema/                      assembled 2020-12 schema tree: specsheets/ + registered namespaces/, composed (see Namespaces) - read directly by all three languages' validators (networknt, jsonschema, jsoncons)
|   `-- rules-v*.json                one generated file per document version (see Rule Documentation)
|-- fixtures/
|   |-- generated/                   schema-derivable pass/fail fixtures, regenerated every run
|   |-- handwritten/                 semantic-rule fixtures, hand-authored
|   `-- archive/                     deprecated-rule fixtures, excluded from the active suite
`-- .github/
    `-- workflows/
        `-- ci.yml

Each templates/<language>/ directory mirrors the internal path layout of its corresponding generated/<language>/ directory, so the generator can merge template and generated content by relative path without a separate mapping file.

Fixture files use a .sed2.json extension - unambiguous as a SED2 document while every tool still treats it as JSON - rather than the bare .sed2 extension the old prototype used. A fixture's full filename is self-describing (see Testing), so no companion expected-output file is needed.

specsheets/ - plus any registered namespaces/ extensions (see Namespaces) - is the canonical source every piece reads. The generator composes them at generate time into generated/schema/, the complete assembled 2020-12 schema tree that all three languages' validators - Java's networknt, Python's jsonschema, and C++'s jsoncons - load directly, since all three support 2020-12, so a namespace's schema reaches all three validators exactly the way a built-in class's does. Since none of the schema validators can map their own errors back to a rule number anyway (see Validation), the canonical schemas also carry generator-only metadata - per-attribute rule IDs - as custom x--prefixed keywords next to each constraint, which JSON Schema validators are required to ignore. Accessor legality and dimension/label/index bounds aren't schema metadata at all; they're read at validation time from each task's own outputs.json (see SEDBase-0008 through -0011). schema/predefined-functions.json stays separate, since it's a flat function/constant registry rather than an object-shape constraint - its own folder's fate (whether anything else still belongs alongside it under schema/) hasn't been revisited since specsheets/ took over the class schemas.

Specification Assembly

core-spec.md and specsheets/ are split for authoring (see Repository Layout: a short document describing the overall shape plus one Data Sheet per class), but nothing stitches them back into a single document meant to be read end-to-end. tools/build_specification.py does that: it discovers every class directly from specsheets/ - the same way the generator discovers AbstractTask/AbstractOutput branches (see Classes) - picking each class's highest version directory, and assembles core-spec.md's sections plus every class's description.md, diagram, and numbered validation rules into a single SPECIFICATION.md at the repository root. Headings are shifted so the whole thing nests under one document outline, same-folder and cross-class links are rewritten to resolve from the repository root or to in-document anchors, and a table of contents is generated, so the result reads as one page on GitHub and also converts cleanly to HTML or PDF with pandoc. Bread crumbs run the other way: each core-spec.md section, each class, and each validation rule carries a "Source:" link back to the .md file it came from (with a line anchor for core-spec.md sections), so a reader can jump from the assembled document to the file to edit. The output is deterministic (no dates), which lets CI regenerate it on every run and commit only when it changed.

SPECIFICATION.md is a generated file - never hand-edited, regenerated by re-running the script - and isn't checked into the repository or produced by CI (see CI); it's built on demand, whenever a single combined view of the spec is actually wanted. Because the script discovers classes and versions straight from specsheets/ rather than from any hardcoded list, it keeps working as classes are added, renamed, split, or restructured, with no changes of its own required. What can go stale, and needs hand-maintenance as the spec evolves, is anything the script assumes about shape rather than content - core-spec.md's section structure, the heading levels inside a description.md, validation/*.md's YAML-frontmatter-plus-prose format - so the script should be revisited whenever one of those conventions changes.

Design

Language Versions

  • C++: C++17.
  • Java: Java 21 (LTS). Java 17 leaves premier support this year, and Java 21 has premier support into 2028 while still matching the modern (Java 17+) build of networknt/json-schema-validator.
  • Python: Python 3.13 or later.
  • JavaScript: not a separately implemented language - the C++ library compiled to WebAssembly with Emscripten (4.0.10 as of this writing); Node 18 or later (22 in CI) or any browser with WebAssembly. See Code Generation.

JSON Libraries

Each generated library uses the JSON parser/DOM and JSON-schema validator most idiomatic for its language, chosen so the schema validator works directly against that language's own JSON tree, without a conversion step:

  • C++: jsoncons::ojson (https://github.com/danielaparker/jsoncons) for both the parser/DOM and the schema pass mentioned under Validation below - one header-only, Boost-Software-License-1.0-licensed dependency instead of two separate ones. jsoncons's jsonschema module is templated over the JSON value type, so it validates the ojson tree directly - no throwaway plain-json copy needed the way the project's previous C++ validator required. It also supports 2020-12 natively, unlike that previous validator, so C++ needs no downgraded mirror of the schema the other two languages don't (see below). ojson preserves object key order exactly as parsed, rather than sorting keys alphabetically the way jsoncons's default json type does - needed because SED2 relies on the order of entries like tasks being meaningful (see Testing). Its dynamic tree is also what lets a namespace's opaque content round-trip unchanged when that namespace isn't registered (see Namespaces) - the schema validation pass still rejects a plain unrecognized field with no namespace at all; only namespace-prefixed content that no registered namespace claims gets this opaque treatment.
  • Java: Jackson's JsonNode tree model (jackson-databind) for the parser/DOM, paired with networknt/json-schema-validator (https://github.com/networknt/json-schema-validator), which validates directly against Jackson's JsonNode. Both are Apache-2.0 and actively maintained; JsonNode holds the same kind of dynamic tree, for the same reason - namespace-opaque content, not general unrecognized-field tolerance.
  • Python: the standard library's json module, which parses directly into native dicts/lists, so there's no separate DOM library to choose. Paired with the jsonschema package (https://github.com/python-jsonschema/jsonschema, MIT-licensed), which validates those same native structures.

Toolchain

Generator language. Python (3.13+, the same floor already set for the generated Python library under Language Versions, so there's only one version matrix to track) - matches tools/build_specification.py's already being Python, and the standard library plus the jsonschema/referencing packages already used for validation methodology (see Validation) make Python a natural fit for a tool that mostly walks and composes JSON.

Package/namespace names. Modeled directly on libsbml's own bindings, since Validation already treats libsbml as this project's reference:

  • C++: namespace libsed2 - libsbml's own C++ namespace is literally namespace libsbml, and libsed2 mirrors that exactly. The CMake library target is sed2 (add_library(sed2 ...)), so the build artifact is libsed2.so/libsed2.a/libsed2.lib via CMake's own automatic lib prefix, rather than a doubled liblibsed2.
  • Python: bare libsed2 - both the PyPI distribution name and the import libsed2 module name, matching libsbml's own setup.py (name = "libsbml", import libsbml) rather than a py-/python- prefix. (A distribution name and an import name are allowed to differ - PyPI names can have hyphens, import names can't - but there's no reason to make them differ here.)
  • Java: libsed2j - libsbml's own Java bindings build a jar named libsbmlj.jar (a suffix j, not a prefix), and libsed2j mirrors that; Maven artifactId: libsed2j. Maven groupId: org.sed-ml - kept from the project's SED-ML lineage even though the format itself is no longer called that, the same way the domain-backed groupIds libsbml (org.sbml) and JSBML use track their own project's name rather than the thing they describe.

Build tools.

  • C++: CMake, with FetchContent pulling jsoncons, GoogleTest, and the ANTLR4 C++ runtime (see Math's Parser Strategy) directly from their upstream repositories - no vendored copies, no package-manager layer (Conan/vcpkg) in front of them.
  • Java: Maven - matches both libsbml's own Java bindings and JSBML, both Maven-published, and is more conventional than Gradle for a library meant to be consumed as a dependency rather than built as an application.
  • Python: pyproject.toml (PEP 621) with setuptools as the build backend - the same conventional, no-surprises choice this project has made everywhere else (see JSON Libraries), and close in spirit to libsbml's own (older-style) setup.py.

math.g4's ANTLR-generated lexer/parser source is a special case for all three languages: ANTLR's own code generator is a Java jar, so requiring it at every consumer's build time would mean every C++ and Python build of libsed2 needs a JVM. Instead, the SED2 generator itself invokes the ANTLR tool as one more build-time step - the same way it already produces generated/schema/ and generated/rules-v*.json - and writes the generated per-language parser source directly into generated/cpp/, generated/java/, and generated/python/. Each generated library's own CMake/Maven/pyproject.toml build then only needs the lightweight ANTLR4 runtime library for its language, never the ANTLR tool itself.

The assembled schema tree (generated/schema/) needs a similar "ships with the library, never fetched" treatment, for a different reason: it isn't a build-time-only dependency like the ANTLR tool, it's read by the validator at runtime, every time a document is parsed. Each generated library embeds its own copy - C++ as a compiled-in resource, Java as a classpath resource under src/main/resources/, Python as package_data in the wheel - and each language's schema validator (networknt, jsonschema, jsoncons) is configured to resolve the schema's own https://sed-ml.org/schemas/... $id/$ref URIs against that embedded copy rather than attempting to fetch them over the network; each library provides its own mechanism for this (a local schema registry, a retrieval-URI-to-resource mapping, or similar), left to the generator implementation to wire up per language. This matters for more than offline use: a library that resolved schema references over the network would validate against whatever's live at sed-ml.org at that moment, not necessarily the version it was actually built and tested against - the opposite of the reproducibility this project already cares about (see Versioning).

Dependency versions. No pinned version numbers - each toolchain points at the latest available release of every dependency, and staying current is treated as ongoing maintenance rather than something to avoid by freezing early. Mechanically this differs by ecosystem:

  • CMake FetchContent: GIT_TAG names each dependency's latest released tag as of whenever it's last touched, rather than staying pinned to whatever tag first got things working. There's no CMake mechanism for "always resolve to whatever the latest tag is" at configure time, so periodically bumping the tag string is the practical equivalent - not a literal floating reference (GIT_TAG main would build against arbitrary unreleased commits, which is worse, not better).
  • Python: no version specifier (or a bare >= floor, if one ever turns out to be needed) on the project's own dependencies in pyproject.toml, so pip/uv resolve to the latest compatible release at install time by default.
  • Java/Maven: Maven's own LATEST/RELEASE meta-versions were deprecated and then removed as of Maven 3, so there's no in-pom.xml equivalent of "always latest" the way CMake/pip allow. The practical equivalent is an explicit version string kept current by an automated dependency-update bot (Dependabot or Renovate) rather than by the build tool itself.

ANTLR is a deliberate exception to "no pinned version numbers": the ANTLR tool (used only by the generator, to produce math.g4's generated lexer/parser source) and the three ANTLR4 runtime libraries (C++/Java/Python, linked into each generated library) have to match versions closely, ideally exactly - see Math's Parser Strategy for why drift between them breaks the build rather than just going stale. Rather than let all four resolve to "latest" independently, the tool version and all three runtime versions are pinned together as one coupled value, bumped as a single deliberate update rather than left to float. The update cadence still follows the same spirit as the general policy above - stay current as ongoing maintenance, not frozen indefinitely - just gated on availability: bump the pinned version semi-regularly, once a new ANTLR release is actually published to every ecosystem this project pulls it from (Maven Central, PyPI, and wherever the C++ runtime is fetched from), rather than the moment any single one of them publishes it first. Pinned value as of this writing: 4.13.2, the latest release on Maven Central (both the tool and the Java runtime), PyPI's antlr4-python3-runtime, and the antlr4 GitHub repository the C++ runtime is fetched from.

One tension worth naming: SED2's own reason for existing is long-term reproducibility ("the same SED2 document should be readable... in 20 years" - What is SED2?), and a toolchain that always tracks latest dependencies is the opposite property at the build level. That's fine for the generator itself (a development-time tool, rerun and recomposed from specsheets/ on every change anyway) and for each generated library's own dependencies, as long as the library's build is reproducible at the moment it's built - it only bites if a build years from now needs to reconstruct an old binary exactly, since "latest" won't mean what it meant then. Worth keeping in mind, not necessarily worth changing the policy over.

Versioning

A SED2 document carries a single semantic version in SEDDocument's version attribute (e.g. "v1.0.0") - while the spec is pre-release this stays at "v1.0.0". There is one version number for the whole document format, not one per class. The attribute must match the pattern v#.#.# (major.minor.patch, with a leading v - the same shape as the version directory names below), enforced both as a JSON Schema pattern and as a validation rule (see SEDDocument's Data Sheet).

Whenever any class changes, the document version increments, but only the class(es) that actually changed get a new version directory under specsheets/// - e.g. v1.0.1/ alongside the existing v1.0.0/. A class that didn't change keeps using its existing directory; nothing forces every class to be re-published at every version bump.

To find the directory a class should use for a document declared at version vD, take that class's version directories and pick the highest one that is not greater than vD. For example, if ExplicitODESimulation has directories v1.0.0/, v1.1.2/, and v1.2.2/, a document declared as v1.2.0 resolves to v1.1.2/ - the newest directory that isn't newer than the document itself.

Directory names are exact semver strings, not just a major number - v1.0.0/, not v1/ - since a class can gain a new directory at any version bump, not just a major one.

Namespaces carry their own, independent version numbers, unrelated to the document's own version - see Namespaces.

Actually converting a document (or a registered namespace's own content) between versions - reconciling what changed between two versions of a core class's schema, or between two versions of a registered namespace's schema, beyond just picking which version directory to source from (see above) - is out of scope for v1. Versioning and cross-version conversion, for both core and namespaced elements, is a goal for a future version of the library, not this one.

Classes

Each SED2 class should have a corresponding class generated in each of the three target languages. The classes are defined by the '_type' child, Each child attribute is owned by its parent, and should have get-, set- isSet- and unset- functions. Each class will have a predefined list of child attributes, defined in the specification.

Some child objects themselves are lists, for example, the 'tasks' child of the document. 'getTasks', then should return an ordered list of tasks, with each task being its own subclass type, defined by its "_type" attribute. List children need a list API, as well: add- (append at end), remove- (remove the child with the given ID), and insert- (place in list at given index).

tasks, outputs, and constants are ordered objects keyed by ID rather than plain index-ordered lists, but the same indexed insert- applies to them too - inserting places the new entry at a given position while it's still keyed by its own ID. setId() renames an entry in place, re-keying it within its parent's ordered collection without changing its position or requiring a remove-then-add. No setter validates eagerly: if a rename produces a duplicate ID, or violates any other rule, that isn't caught until an explicit validate() call, the same as any other constraint (see Validation) - so a document can sit in a temporarily invalid state while it's being built, rather than rejecting every intermediate edit.

An OrRef-typed field (NumberOrRef, StringOrRef, and so on) gets a specialized accessor shape instead of the plain get-/set- pair: for any attribute X of an OrRef type, the accessors are always getXRef(), getXValue(), setXRef(), and setXValue(), plus isXRef() to report whether the field currently holds a reference. getXValue()/getXRef() return the literal value or the reference text respectively, and setXValue()/setXRef() set the field as one kind or the other. Both the getters and the setters are split by kind rather than overloaded, so a caller that only means to handle literals (or only references) can't silently read or write the wrong kind - calling getXValue() while the field holds a reference (or getXRef() while it holds a literal) throws, the same as calling any get- accessor on a field that isn't set at all; there's no sentinel or default return, since for most of these types there's no value outside the legal range that could safely double as "not what you asked for." Numeric literals convert to and from string via getXValue()/setXValue() using each target language's own standard number-to-string conversion; the three generated libraries aren't required to produce byte-identical strings for the same number; callers compare the resolved numeric value, not the string form. Boolean literals stay native JSON true/false - no separate parsing rule is needed, since JSON's own boolean token already round-trips cleanly through all three languages.

Every class instance also carries two backpointers, following libsbml's own convention: one to its immediate parent, and one to the owning SEDDocument itself - needed for operations like getSEDReference and the reference-scoping rules, which have to walk up from any point in the tree. These are plain object references in Java, and the same recursive connectToParent()-style propagation libsbml uses applies in all three languages, not just C++: attaching a whole subtree (e.g. adding a Task that already has its own children) has to set every descendant's document pointer, not just the direct child's. C++ keeps these as raw, non-owning pointers, deliberately outside the parent's own ownership of its children, so no reference-counting cycle forms. Python's plain attribute reference is a strong reference by default, though, so the same relationship there - a parent's child list owning the child, plus the child's own backpointer to that parent - would form a real reference cycle; Python's parent and document fields use weakref for exactly this reason, mirroring the non-owning intent of C++'s raw pointers rather than relying on CPython's cyclic collector to clean it up.

SEDDocument carries static read/write entry points (e.g. a static factory method that parses a file or string and returns a new SEDDocument, and an instance method that serializes it back out), and the library also exposes the same operations as free-standing functions - readFromFile(filename), readFromString(docstring), writeToFile(doc, filename), and writeToString(doc) - for callers who prefer a plain function over a static method call; both forms do the same work.

Every API-contract violation - calling getXValue()/getXRef() on the wrong kind or an unset field (see above), an out-of-range insert-, any other misuse of the generated API - is reported by throwing, never by a return code, a null, or a sentinel value; this is consistent across all three languages (Java exceptions, Python exceptions, C++ exceptions). This is distinct from validate()'s own report: a document is allowed to be temporarily invalid while it's being built (see above), so validate() returns a list of rule violations rather than throwing - exceptions are for misusing the API itself, not for a document that fails a SED2 rule.

clone() produces a fully detached copy: the clone's own parent and document pointers are unset, even though the original object's weren't. Everything below it - the clone's own children, cloned in the same operation - gets its parent pointer set to the new clone, never to the original, so the cloned subtree is internally consistent even though the clone as a whole isn't attached to anything yet.

Method and function names follow each target language's own convention rather than one shared style across all three: camelCase in Java, and snake_case in Python, matching each language's own community convention. C++ also uses camelCase, mirroring libsbml's own C++ API (already this project's reference for C++ conventions elsewhere - see JSON Libraries, Package/namespace names) rather than STL's snake_case.

Not every specsheets/ folder is a class, though. specsheets/core/Types bundles the shared primitive and reference-helper types (SId, SIdRef, NumberOrRef, ScaleType, and so on) that every other Data Sheet's schema.json pulls in via $ref - it has no UML diagram, no _type discriminator, and is never itself instantiated as a document element. The generator should treat it as a source of shared type definitions (mapping each $def to whatever the target language uses for that primitive or reference type - a typedef, a small wrapper/validation class, an enum, etc.), not as a class to generate get-/set-/isSet-/unset- accessors for.

AbstractTask's and AbstractOutput's _type discriminators are themselves entirely generated, not hand-maintained - and the mechanism needs no external list anywhere of which discriminators exist or which Common mixin each one scans for; it's self-describing from the files alone. AbstractTask/v1.0.0/schema.json's $defs/AbstractTask entry carries no oneOf in its hand-authored form. Instead it carries an x-generated-oneOf key whose value is a $ref-style pointer at its Common mixin (common.schema.json#/$defs/AbstractTaskCommon); AbstractOutput/v1.0.0/schema.json's $defs/AbstractOutput carries the same marker, pointing at AbstractOutputCommon. A discriminator's own $defs entry may also carry a sibling x-missing-type-rule-id, naming a rule that fires specifically when _type is absent entirely (rather than present but unrecognized) - narrower than the discriminator's own catch-all, for wherever that distinction is worth its own message; see core-spec.md Section 8 and Validation's Schema-Pass Errors below. To build a discriminator's oneOf, the generator: (1) finds every $defs entry across specsheets/ and registered namespaces that carries an x-generated-oneOf marker; (2) resolves that marker's pointer to find the discriminator's own Common schema; (3) scans every schema.json under specsheets/ - all four categories, so a concrete class's own category placement never has to track which discriminator it belongs to - for any class whose ancestry composes that same Common schema, directly or transitively, and pins a _type with const, and treats each match as a branch. Composition mirrors each class's own UML inheritance chain rather than flattening it at every leaf: a concrete class's allOf typically names just its one most specific ancestor (e.g. ExplicitODESimulation composes only AbstractODESimulation), and each mixin in turn composes the next one up (AbstractODESimulation composes AbstractSimulation, which composes AbstractTaskCommon, which composes SEDBaseFields) - so the discovery scan has to resolve that whole chain, not just a class's own immediate allOf, to find every branch. A class registers itself as a legal branch purely by declaring its parentage somewhere in that chain; the discriminator is a computed view over the resolved composition, never a separately hand-maintained list. This is the same discovery rule namespace new/ classes already use (see Namespaces): adding a new built-in task or output type is exactly as lightweight as registering a namespace one - drop a new Data Sheet folder in the right place, and the generator picks it up, with no edit to AbstractTask's or AbstractOutput's own file ever required. Because the x-generated-oneOf marker is itself how the generator finds discriminators to populate, introducing an entirely new abstract-class family - not just a new branch of an existing one - needs no generator code changes either: give it its own common.schema.json (say, FooCommon) and a schema.json whose $defs entry carries an x-generated-oneOf pointer at that Common schema, and the generator discovers and populates it the same way it does AbstractTask, AbstractOutput, and RangeInline.

RangeInline (specsheets/tasks/Range/v1.0.0/inline.schema.json) is a third instance of the exact same mechanism, at a narrower scope: its x-generated-oneOf marker points at RangeCommon (specsheets/tasks/Range/v1.0.0/common.schema.json), so the generator scans for classes whose ancestry composes RangeCommon, directly or transitively, and pins a _type const - currently Range (which composes RangeCommon directly) and NumericRange/ParameterRange (which reach it through NumericRangeCommon). RangeInline has to live in its own file/$defs entry rather than reusing Range's own schema.json the way AbstractTask reuses its own $defs/AbstractTask: Range, unlike AbstractTask, is itself directly instantiable (a bare Range with just values is a legal standalone task), so specsheets/tasks/Range/v1.0.0/schema.json's $defs/Range entry is already the concrete class, pinning _type const "range" - it can't simultaneously be the union placeholder. Any future class that's both directly instantiable and the root of further subclasses needs this same split: its own common.schema.json for the shared mixin, plus a separate file for the generated union, rather than repurposing its own schema.json the way a never-instantiated abstract root can.

Namespaces

SED2 documents may be extended by third parties with attributes or entirely new task/output types under a namespace@identifier syntax (see core-spec.md Section 5). A namespace the generator doesn't know about is unaffected - its content round-trips unchanged, unvalidated, because it's still syntactically namespace-prefixed even though this particular prefix isn't one the generator recognizes. A field or _type with no namespace at all never gets this treatment - that's always a plain validation failure, the same as any other unrecognized property (see Classes).

A namespace the generator does know about - a known namespace - gets its schema composed directly into the canonical schema tree at generate time, so its content validates exactly like any built-in attribute or class, through the same three validators, with no separate runtime code path.

Each known namespace carries its own semantic version, entirely independent of the document's own version (see Versioning) - a namespace's schema can evolve on its own timeline. Whenever a document uses anything from a given namespace, anywhere, it must also carry a @version attribute on SEDDocument itself (e.g. saurolab@version), naming the namespace version that document was written against - mirroring SEDDocument's own version attribute, one level down, including its format: a namespace version must also match the pattern v#.#.# (e.g. v1.0.0). Since the generator declares @version directly on SEDDocument's composed schema for every registered namespace, that format is enforced the same way SEDDocument's own version is, as a plain schema pattern on the generated property - only the requiredness ("must be present if the namespace is used") is a whole-document conditional a plain schema can't express, and stays a semantic Validation rule (see Rule Documentation).

Known namespaces live in a namespaces// directory (see Repository Layout), structured exactly like specsheets/: two top-level directories, new/ and updated/, each split into tasks/, core/, outputs/, and auxiliary/ the same way specsheets/ is, and then //schema.json below that - for example namespaces/saurolab/new/tasks/FancyAnalysis27/v1.0.0/ for a wholly new task type, or namespaces/saurolab/updated/tasks/ExplicitODESimulation/v1.0.0/ for new attributes saurolab adds to the existing ExplicitODESimulation class. The directory name is that namespace's own version, unrelated to the document format's version, and follows the same v#.#.# format.

Each // leaf gets the same contents as a specsheets/ Data Sheet, not just a schema.json - a description.md, and a validation/ folder. Rule IDs there can't reuse the bare [ClassName]-NNNN scheme without risking collision with that same class name's own core rules (or another namespace's), and a bare namespace@identifier-style ID isn't filename-safe either, so namespace rule IDs are hyphenated instead: an updated/ class's rules are --NNNN (e.g. ExplicitODESimulation-saurolab-0001) - the core class name leads, since that's the "real", pre-existing class being extended - while a new/ class's rules are --NNNN (e.g. saurolab-FancyAnalysis27-0001) - the namespace leads, since the class itself has no core identity of its own.

new/ classes are found by the exact same scan that finds built-in branches (see Classes) - the generator doesn't treat namespaces/ and specsheets/ differently when building AbstractTask's/AbstractOutput's oneOf, it scans both trees (plus any --namespace-dir directories) for classes composing the relevant Common mixin with a _type const, whatever tree they live in; a new/ class's discovered _type is just @ instead of a bare built-in one. updated/ classes compose their declared attributes (each namespace-prefixed, e.g. saurolab@thisFactor) directly into that specific existing class's own schema, via allOf, the same way the class's own inline schema already composes in. Because these are literal, per-class property declarations rather than a blanket pattern, unevaluatedProperties: false rejects anything else automatically - so a namespace@key that a known namespace hasn't declared for that particular class is a real validation failure, with no extra generator logic needed for that case.

Every namespace-prefixed attribute lives in one place regardless of whether its namespace is registered: a generic per-object namespace-attribute store, keyed by prefix and attribute name, which is what already lets an unregistered namespace's content round-trip unchanged (see JSON Libraries). A generic accessor (e.g. getNamespaceAttribute(prefix, key)/setNamespaceAttribute(prefix, key, value)) reads and writes that store directly, for any namespace, registered or not. When a namespace is registered and known, the generator additionally produces typed convenience accessors for its updated/ attributes - getSaurolabInputXValue()/getSaurolabInputXRef()/setSaurolabInputXValue()/etc. for saurolab@inputX on BoundedODESimulation, following the same OrRef accessor shape as any other attribute (see Classes) - but these are generated sugar over that same store, not a second place the value lives. The generic accessor stays available for a known namespace's attributes too, not just unregistered ones: since there's only one underlying store, the typed and generic paths can never disagree, and code that only knows a prefix and key at runtime works identically whether or not this particular build happens to have that namespace registered.

Mechanically, none of this happens by editing canonical files - not AbstractTask's/AbstractOutput's own oneOf (see Classes), and not an updated/ fragment's target class either: specsheets/tasks/ExplicitODESimulation/v1.0.0/schema.json can't carry a $ref to a namespace fragment it doesn't know exists, and shouldn't need hand-editing every time one is registered. Instead, for an updated/ fragment, the generator assembles a third schema per affected class, e.g. {"allOf": [{"$ref": ".../specsheets/.../ExplicitODESimulation/v1.0.0/schema.json#/$defs/ExplicitODESimulation"}, {"$ref": ".../namespaces/saurolab/updated/.../ExplicitODESimulation/v1.0.0/schema.json#/$defs/ExplicitODESimulation"}], "unevaluatedProperties": false} - generated fresh each run from whatever's currently registered, the same way a new/ class becomes a generated oneOf branch rather than something hand-edited into AbstractTask/AbstractOutput. This assembled tree is materialized as generated/schema/ (see Repository Layout): the complete, namespace-composed 2020-12 schema, which all three languages' validators - networknt, jsonschema, and jsoncons - load directly. It's the one generated schema output; specsheets/ and namespaces/ stay pure, hand-authored sources that never need editing just because a namespace gets registered.

Only a namespace prefix the generator has never heard of at all should be accepted as opaque, unvalidated passthrough. That still needs a permissive catch-all: a generated patternProperties entry matching any prefix@identifier-shaped key except the prefixes currently registered (built fresh each generate run from namespaces/ plus any --namespace-dir directories - see below), so an unregistered prefix still passes through opaque while a wrong identifier under a registered prefix still fails. Declared once, on SEDBaseFields (see core/SEDBase) - the same place name/description/notes/annotations are already centralized - it reaches every class through the allOf composition already in place; unevaluatedProperties is defined to see patternProperties evaluated anywhere in that composed tree, not just the local schema object, so this needs no extra logic for any of the three languages - all three now read the 2020-12 schema directly, with no draft-7 mirror or its own flattening step to keep in sync.

Class-level types get the same "unregistered prefix is opaque, registered-but-wrong is an error" split, via a parallel fallback on the _type discriminator itself: a _type whose prefix isn't registered at all falls through to a generated UnknownClass - one per language, a minimal class that stores its contents as opaque literal values and returns its raw _type string from getType(). UnknownClass still subclasses AbstractTask or AbstractOutput, whichever discriminator it fell through, so it satisfies any code elsewhere that assumes its children are AbstractTask (or AbstractOutput) subclasses - for example Repeat's subTasks list. A _type whose prefix is registered but whose identifier isn't one of that namespace's new/ classes does not fall through to UnknownClass - it's a real oneOf match failure, the same as the attribute-level case. For now, referencing any namespaced task's output - whether it fell through to UnknownClass or resolved to a real generated new/ class - is limited to the same three forms any task supports: [id], [id].model, and [id].strings (see core-spec.md Section 4). Nothing namespace-specific is reachable through a reference string, including a known namespace's own added attributes (from new/ or updated/ alike); this can be revisited later if it turns out people need deeper access.

The generator always composes whatever's in the repository's own namespaces/ directory. To register a namespace without forking the project, the generator also accepts a repeatable --namespace-dir flag pointing at additional directories in the same layout, composed the same way as the built-in ones. A namespace prefix must be alphanumeric, and must not collide with any other currently-registered prefix. By default, two sources defining the same prefix is an error ("namespace defined twice") - the generator refuses to run rather than silently pick one; this covers two --namespace-dir sources colliding with each other just as much as one colliding with the repository's own namespaces/. An override flag, --override-namespace <prefix>, lifts this only for a prefix that collides with the repository's own namespaces//: the --namespace-dir version replaces it entirely, while every other, non-colliding prefix already in namespaces/ is still parsed and composed normally. Two --namespace-dir sources colliding with each other has no override - there's no "repository's own" version for either to defer to, so that's always an error.

Code Generation

Nothing in any generated library is ever hand-edited. The repository has two kinds of directories: a set of per-language template directories holding hand-written, target-language-specific code (for example, a C++-only helper that has no Java or Python equivalent), and a separate, fully generated output directory per target language. The generator reads the shared SED2 class/schema definitions plus the relevant template directory and writes the complete, self-contained library into the generated directory - one that can be dropped in and used as-is, without needing the templates directory alongside it. Regenerating always overwrites the generated directories in full; any hand-written code that needs to survive regeneration lives only in the template directories.

JavaScript target. There is no third implementation of the library's logic for JavaScript: generated/js/ is the C++ library compiled to WebAssembly with Emscripten, following the pattern of sys-bio/libantimonyjs - the C++ static libraries are linked by emcc into one self-contained <name>.js (WebAssembly embedded: -sMODULARIZE=1 -sSINGLE_FILE=1, memory growth and C++ exceptions on, -Oz), whose entry points are a flat C API exported through cwrap and wrapped by a small hand-written JavaScript file. The API is strings in, strings out, with JSON as the string format (SED2 documents are JSON already), rather than a binding for every generated class: validate(text), validateObject(key, text) for the classes validated on their own, normalize(text), checkMath(text), ruleCatalog(), plus version and class-name queries. Because it is built on the C++ target, anything that changes with the C++ library (the model, rules, validation logic) reaches JavaScript with no extra work; generator/emit_js.py generates only what depends on the spec or the command line (class names, document version, library version, the CMake build, the TypeScript declarations, package.json), while the fixed entry points, the wrapper, and the fixture test live under templates/js/. generate.py generates js together with cpp (--lang js implies cpp), so the two cannot drift apart.

Math

SED2 elements can carry infix math as string children (e.g. Calculation's math, DrawFromDistribution's arguments). An ASTNode class - borrowing the rough interface from libsbml's ASTNode, minus the XML dependency - parses these strings into a tree; get-/set-/isSet-/unset- accessors on the owning class work with the raw string, and a parse()/toString() pair converts to and from the tree. v1 stops there: no evaluator is generated for this grammar, since nothing in a SED2 document itself calls for computing a math string's value ahead of actually running the simulation. The grammar below is SED2's own: standard infix arithmetic plus function calls (shared with SBML's own MathML dialect, so ASTNode's vocabulary overlaps heavily with libsbml's), extended with the #/:/./[] reference syntax described in core-spec.md Section 4, which has no MathML equivalent since MathML has no notion of "a value produced by another document element."

Grammar

Two grammars compose: expressions (the math itself) and references (the #... addressing syntax embedded inside them). An EBNF sketch follows. The arithmetic, reference, and function-call productions come directly from the spec and worked examples; the relational, logical, and piecewise productions have no SED2 textual precedent and instead follow libsbml's L3 infix parser (src/sbml/math/L3Parser.ypp), so that any formula libsbml accepts reads the same way here.

(* --- expressions --- *)
expression      ::= logical
logical         ::= relational ( ("&&" | "||") relational )*          (* libsbml: one level, left-assoc *)
relational      ::= additive ( relop additive )*                    (* libsbml: chains, see below *)
relop           ::= "==" | "!=" | "<>" | "><" | "<" | ">" | "<=" | ">="
additive        ::= multiplicative ( ("+" | "-") multiplicative )*
multiplicative  ::= unary ( ("*" | "/" | "%") unary )*
unary           ::= ("-" | "+" | "!") unary | power
power           ::= atom ( "^" unary )?              (* right-associative *)
atom            ::= NUMBER
                   | REFERENCE
                   | IDENTIFIER "(" arglist? ")"      (* function, distrib, logical/relational call form, piecewise *)
                   | IDENTIFIER                        (* bare constant: pi, true, ... *)
                   | "[" arglist? "]"                  (* array literal *)
                   | "(" expression ")"
arglist         ::= expression ( "," expression )*

(* --- references (attested) --- *)
reference       ::= "#" IDENTIFIER ( ":" IDENTIFIER )* subaccess*
subaccess       ::= "." IDENTIFIER                    (* named non-structural output, e.g. .model *)
                   | "[" index "]"                     (* structural subvalue of AnnotatedData *)
index           ::= signedInt | "'" STRING "'" | signedInt ":" signedInt   (* signedInt ":" signedInt is a range, e.g. [2:5] or [-3:-1] *)
signedInt       ::= "-"? DIGIT+

IDENTIFIER follows the existing SId pattern (specsheets/core/Types/v1.0.0/schema.json): ^[A-Za-z_][A-Za-z0-9_]*$. A reference's first IDENTIFIER must name one of SEDDocument's own ID-keyed children (tasks, outputs, constants currently - styles once it's more than a placeholder); every following colon-delimited IDENTIFIER walks one level down that element's own containment tree (#tasks:repeat:subTasks:sim1); .IDENTIFIER accesses a named non-structural output a task declares in its own outputs.json (.model, .strings, .aggregates, ...); [index] indexes into AnnotatedData, positionally ([3]), by label (['S1']), by negative (Python-style, from-the-end) index ([-1]), or by range ([2:5], [-3:-1]) - matching core/Types's SIdRef pattern, which already accepts all four forms. All four are zero-indexed and follow Python conventions throughout, including a range's own bounds: [2:5] selects positions 2, 3, and 4 - three entries, end-exclusive exactly like Python's a[2:5] - so it needs a dimension of at least 5, not 6; a bare [3] needs a dimension of at least 4. A single positional or label index drops that dimension entirely from the result's shape, the way indexing an array does in most languages; a range keeps it, sized to however many entries the range actually selects. hasSubvalue() (see Cross-references) validates a subaccess* chain against the referenced task's outputs.json without needing to run anything.

Relational, logical, and piecewise syntax follows libsbml's L3 infix parser (decided 2026-09-24):

  • Precedence, lowest to highest: &&/|| (one shared level, left-associative, so a && b || c is (a && b) || c); then the relational operators; then +/-; then *///%; then unary -, +, and !; then ^. So !a == b is (!a) == b, and -2^2 is -(2^2).
  • Relational chains collapse the way libsbml collapses them: a < b < c becomes lt(a, b, c), and a mixed chain such as a < b <= c becomes and(lt(a, b), leq(b, c)). != is always binary and never joins a chain. <> and >< are accepted as alternative spellings of != and serialized back as !=.
  • Repeated && or || flatten into one n-ary node: a && b && c becomes and(a, b, c).
  • Every logical and relational operation also has a call form, which is an ordinary function call in the grammar above: and(...), or(...), xor(...), not(x), implies(a, b), eq(...), neq(a, b), lt(...), gt(...), leq(...), geq(...). xor and implies exist only in call form.
  • piecewise is an ordinary function call with libsbml's argument order: piecewise(value1, condition1, value2, condition2, ..., otherwise). Each value comes before its condition, as in MathML's <piece>; the trailing unpaired otherwise is optional. Condition-first order has the same argument count and would parse without complaint, so the order cannot be validated - every example and doc must show value-first.
  • % is infix rem, as in libsbml with its L3v2 modulo setting: x % y is rem(x, y), and the result takes the dividend's sign (-7 % 3 is -1). C++ fmod and Java's % already behave this way; Python's % does not (it takes the divisor's sign), so the Python library evaluates it with math.fmod.
  • Not adopted from libsbml: lambda.

schema/predefined-functions.json records every operator and call form, with its arity.

Lexer

The one real lexical hazard is .: it means "decimal point" inside a bare number (1.3) and "named-output accessor" inside a reference (#tasks:sim1.model) - the same character, two different meanings, and MathML's own token grammar has nothing like it since MathML has no reference syntax at all. The fix is to never let the two lexing paths interleave: on seeing #, the lexer hands off to a small reference sub-scanner that consumes the entire reference production (IDENTIFIER, colon-groups, then any run of .IDENTIFIER/[index] subaccesses) as a single opaque REFERENCE token per the grammar above, and returns control to the general lexer only once that chain ends (at whitespace, an operator, a comma, or an unmatched closing paren/bracket). Outside of a REFERENCE token, . only ever appears inside a NUMBER, so ordinary decimal lexing is unambiguous. This also means the reference grammar's own internal structure never has to round-trip through the general expression lexer/parser at all - it's effectively a separate, embedded language.

Predefined Functions

schema/predefined-functions.json (validated by schema/predefined-functions-meta.schema.json) is the registry of every name callable or usable from SED2 math: the full SBML Level 3 Core MathML subset - arithmetic/relational/logical operators and their call forms (plus Level 3 Version 2's implies), piecewise, the trig/arithmetic function set, and the constants pi/exponentiale/true/false/notanumber/infinity (sbml-specifications/sbml-level-3/version-2/core/spec/preliminary.tex, sec:mathmlsubset) - plus all 12 SBML distrib package csymbol draw functions with their argument-count variants (sbml-level-3/version-1/distrib/sbml-level-3-distrib-package-proposal.tex, tab:csymbols: normal, uniform, bernoulli, binomial, cauchy, chisquare, exponential, gamma, laplace, lognormal, poisson, rayleigh), plus sum, the one SED2-specific math-grammar function with direct textual evidence (the Calculation compare_summary example). Each entry records its arity, category, and an attested flag distinguishing confirmed vocabulary (from spec text/examples, or by explicit decision, as for the libsbml-style operators - see Grammar above) from proposed defaults; every current entry is confirmed. Two things SBML defines that this registry deliberately leaves out: the time/delay/avogadro/rateOf csymbols, which describe quantities internal to a running SBML model rather than a SED2 task output, and any of KISAO:0000824's other aggregation-function children beyond sum (mean, standard deviation, ...) - AggregationCalculation already exposes those as a whole Task selected by kisaoID, and nothing in the source text calls them by name inside a math string the way sum is. This is the same registry the generator's ANTLR grammar (see Parser Strategy) validates function calls' arity against, and the same one that would back a getSEDReference/math-eval implementation's builtin table in each generated library.

Parser Strategy

libsbml's own infix parser is a hand-written bison grammar, but bison only targets C++ - Java and Python would each need their own parser regardless, and a bison-generated C++ parser wouldn't help either of them. Rather than hand-write and hand-maintain three independent recursive-descent parsers (one per target language, for the exact same grammar), the generator uses a single ANTLR4 grammar (math.g4, covering both the expression and reference productions above) and lets ANTLR generate the C++, Java, and Python lexer/parser from that one source - the same "one canonical definition, three generated targets" shape every other part of this project already uses (JSON Schema classes, numbered validation rules, oneOf discriminators). The cost is a new runtime dependency in all three generated libraries: the ANTLR4 runtime (BSD-3-Clause, actively maintained, with first-class C++/Java/Python targets) alongside whatever's already listed under JSON Libraries. math.g4 lives under generator/ (not under specsheets/, since it isn't per-class data - see Repository Layout), and the ANTLR toolchain (a Java jar) is a build-time-only dependency of the generator itself, not of any generated library.

Validation

The JSON schema pass rejects structurally malformed documents (wrong types, missing required fields, pattern mismatches) with a human-readable message, but a rule number is this project's own invention - no JSON Schema validator library knows what one is on its own - so numbering isn't something any of them hand back directly (see Schema-Pass Errors below for how validate() numbers them anyway).

Instead, every constraint that a person filling out a SED2 document might get wrong - including ones a schema could also catch, like a required field or a pattern - gets its own stable, numbered rule and a message tailored to that specific mistake, the same way libSBML doesn't rely on its XML parser's diagnostics for numbering either: a generic schema error or a class's catch-all still tells the user something's wrong, but not what, and a numbered rule is what turns that into something actionable. Numbers are appended to the end of the spec as they're discovered. Every class has its own validate() function returning that list, fully recursive (see Validation Results below).

Every rule earns its own numbered file and message this way, but not every rule needs its own hand-written detection code in validate() to go with it. A rule whose entire content is something JSON Schema keywords already express - a required field, a type, an enum, a const _type, the reference-pattern half of an X-or-reference field - is caught by the schema pass alone (see Schema-Pass Errors), and the x-rule-id mapping delivers that rule's own specific message without validate() needing to check it separately; the rule still gets written up and numbered, since that's what lets the schema pass report something more useful than the class's generic 0000 catch-all, but no bespoke detection logic is needed behind it. A rule is only mandatory to hand-write detection logic for - and so has to be generated as real validate() code - when no combination of schema keywords could ever catch it at all: because it has to resolve a reference and check what it resolves to (every "if a reference, must resolve to a number/string/array/..." rule - schema validates the reference's own shape, never its target), walk the containment tree (SEDBase's reference-scoping rules, Repeat's subTasks-only restrictions), parse an embedded grammar (Types-0001 through Types-0004), reason about the document as a whole (SEDDocument's duplicate-key, constant-ordering, and namespace-usage checks), or read an external model or data file. In short: every rule gets written up, but only the ones the schema can't catch have to be generated as real validate() logic - the rest ride along on the schema pass's own x-rule-id mapping.

Among the rules that do need library logic, two more kinds split further, recorded in each rule file's own frontmatter as a check field: ref-type - the formulaic "if a reference, must resolve to type X" rules, which the generator derives directly from the field's own declared type (NumberOrRef, StringOrRef, and so on), needing no per-rule authorship - and handwritten, the genuinely bespoke ones (containment-scoping and reference-resolution checks, document-wide namespace/duplicate-key/ordering/version checks, the math grammar) that have to be authored once per language and then reused, the same way every other piece of hand-written, language-specific code in this project survives regeneration: as a file under templates/<lang>/ (see Code Generation), here templates/<lang>/rules/<RuleID>.*. Every rule file's own check field is the authoritative record of which kind it is; this paragraph names categories, not a maintained roster, so it doesn't go stale the way a hardcoded class or branch count would (see Classes). The generator fails its run if a handwritten rule is missing its file for any one of the three languages - the same discipline the stub rule (Claude.md) already asks for, just enforced mechanically instead of by memory. Decided: a fixed function-name convention, not a spliced fragment - each templates/<lang>/rules/<RuleID>.* file defines one function the generated code calls by name, so the hand-written logic stays independently callable and testable rather than textually dependent on whatever local variable names the generator's own codegen happens to use at a given splice point. Where the generated caller invokes it from follows directly from the rule's own scope, itself readable from where its file already lives: a rule filed under one specific class's own validation/ folder that references only that class's own fields (AbstractTask-0003, Repeat-0008 through -0010, LoopVariable-0004) is called from that class's own generated validate(), once per instance; a rule filed under SEDBase or Types that applies wherever a field of a given declared type occurs, regardless of which class carries it (SEDBase-0005 through -0015 for every SIdRef/*OrRef-typed field, Types-0001 through -0004 for every math-bearing field), is called from a shared per-type helper that every class's generated validate() invokes automatically for each of its own fields of that type; a rule filed under SEDDocument that needs the whole document (SEDDocument-0009 through -0014) is called once, from SEDDocument's own validate(). Every handwritten function returns the same list<ValidationProblem> shape as everything else validate() produces (see Validation Results below), so its result folds straight into the caller's own list with no translation. Still left to decide when the generator is actually being written: a single handwritten function's exact parameter shape, and exactly when the missing-file check runs.

Overall, the model here is the SBML specification (C:\Users\Lucian\Desktop\sbml-specifications\sbml-level-3\version-2\core\spec\sbml-level-3-version-2-release-2-core.pdf) and the libsbml library (C:\Users\Lucian\Desktop\libsbml), though for JSON instead of XML.

Validation Results

validate() returns a list of ValidationProblem records - never throws, since a document is allowed to be temporarily invalid while it's being built (see Classes) - empty when there's nothing to report. Each record carries five fields: ruleId (e.g. SEDBase-0005), severity, rule (that rule's own stated constraint, copied verbatim from its frontmatter, so a caller can print something meaningful without a side lookup), message (that rule's message template with every placeholder filled in for this specific violation), and location (an RFC 6901 JSON pointer to where in the document it occurred, rooted at whichever element validate() was called on - see below).

severity is a two-valued, ordered enum: warning, then error. No third tier is defined below warning - an "info" level would be dead vocabulary today, since no rule anywhere uses one - though the ordering leaves room to insert one later if a real rule ever needs it. validate() takes one optional parameter, severityAtLeast, defaulting to warning - the lowest defined severity - so an unfiltered call returns everything; passing error returns only errors.

Every class has a validate() method, not just SEDDocument, and it's fully recursive: calling it on any element returns that element's own violations together with every descendant's, so calling it on the document root returns the complete list for the whole document, and calling it on some task in isolation returns just that task's own subtree. Concretely, a class's generated validate() does three things. First, it runs the schema pass using that class's own standalone schema.json (every class has one - see Classes) against its own local JSON representation, mapping failures to rule IDs the usual way (see Schema-Pass Errors below). Second, it runs its own ref-type and handwritten checks, dispatched as described above. Third, it merges those two local result sets, de-duplicating by (rule ID, location) - the only point in the tree where the same violation could genuinely be found by both mechanisms at once (see below) - filters by severityAtLeast, and then recurses: it calls validate() on each of its own SEDBase-derived children, rewrites each returned location by prepending its own path segment (so a violation reported three levels down still carries a full pointer once it reaches the caller), and appends the results. A constant, or any other plain non-SEDBase value, isn't itself recursed into - there's nothing further to validate about a bare number or string beyond whatever rule already covers the field holding it.

This describes the observable contract, not a mandated implementation: nothing requires literally running a separate schema validation per node. A generator is free to run the schema pass once over the whole document and internally slice results by JSON-pointer prefix per node instead, as long as calling validate() anywhere still returns that subtree's own de-duplicated, correctly-located list.

Schema-Pass Errors

Schema-pass errors belong in the same list validate() returns, not as a separate preliminary gate: a document that fails the schema pass still gets every hand-written numbered check run against it too, and the two result sets merge into one. Two things justify running both passes rather than trusting hand-written checks alone: hand-written coverage is only as complete as whoever wrote it - a schema constraint (or a namespace's own composed-in schema fragment, see Namespaces) can exist without a matching numbered check ever having been written for it - and, unlike a hand-authored check, the schema pass by construction always exactly matches whatever the schemas at generate time actually say, so it's the one mechanism guaranteed not to silently drift out of sync with schema authoring.

Every numbered rule still lives in its own class's validation/ folder as usual (see Rule Documentation) - what's new is that each Data Sheet folder's rule set also carries exactly one open-ended catch-all rule, reserved as 0000 outside that folder's normal append-only 0001+ sequence (e.g. ExplicitODESimulation-0000), for a schema-pass failure attributable to that class but not to any more specific rule. This applies to every folder with a validation/ directory, not just concrete task/output classes - AbstractTask, AbstractOutput, SEDBase, and every other schema-only mixin's own folder gets one too (see the fallback rule below for why). This does not extend to Types (or any similar shared-type bundle that is never composed into a class's allOf chain, only $ref'd at individual property definitions): the fallback walk below only ever climbs a class's own composed ancestry, so a schema-pass failure inside one of Types's primitive definitions always resolves to whichever class actually owns the property that used it, never to Types itself. Types's own validation/ folder exists only to hold its four handwritten math-grammar rules (Types-0001 through -0004, see Math), which the schema pass was never going to attribute to it either - so it needs a validation/ folder without ever needing a 0000 catch-all to go with it. A namespace-owned class's own validation/ folder gets one too, using that namespace's own hyphenated ID scheme (see Namespaces): a new/ class's <prefix>-<ClassName>-0000 (e.g. acme-AcmeWidget-0000), or an updated/ fragment's <ClassName>-<prefix>-0000 (e.g. SimpleWidget-acme-0000) for a schema-pass failure attributable to that namespace's own additions specifically - distinct from the base class's own <ClassName>-0000, which still covers the base class's own fields. A schema-pass failure is numbered one of two ways:

  • Where the validator library can resolve which schema location/keyword failed - all three languages' validators (Java's networknt, Python's jsonschema, and C++'s jsoncons) expose enough structured error detail to do this - the generator looks that location up against the same x-rule-id metadata a hand-written check would use (see Rule Documentation in section 8), and reports the schema-pass failure under that exact rule ID.
  • Where it can't - a location the lookup can't resolve to any specific x-rule-id (a oneOf/allOf combinator failure spanning several branches, a namespace's dynamically-composed fragment with no rule written for it yet, or any other schema-visible constraint nobody's minted a number for) - the failure is reported under the most specific class's catch-all rule the instance location can still be attributed to. A _type mismatch inside tasks that doesn't match any of AbstractTask's discovered branches, for instance, can't be pinned to a specific concrete class, but it's still known to be an AbstractTask, so it's reported as AbstractTask's own catch-all rather than falling all the way back to SEDBase's. One sub-case of this gets its own dedicated rule instead of the generic catch-all: when _type is missing from the instance entirely, rather than present but unrecognized, and the failing oneOf's own $defs entry carries an x-missing-type-rule-id (see core-spec.md Section 8) - AbstractTask-0002, for instance, fires with its own message naming the failing location, ahead of falling through to AbstractTask-0000's generic one. The fallback walks up the class hierarchy - concrete class, then whatever abstract/mixin ancestor's own folder the location can still be resolved to (AbstractTask/AbstractOutput/AbstractSimulation/etc.) - stopping at SEDBase's own catch-all as the last resort for anything resolvable no further than "some element in this document."

A schema-pass failure and a hand-written check can both fire for the same underlying problem and resolve to the same rule ID (that's the point of the first bullet above) - validate() de-duplicates by (rule ID, instance location) before returning its list, so a real violation is reported once even though two independent mechanisms caught it, and fail-case fixtures (see Testing) keep meaning what their filename says rather than a violation appearing to fire twice.

A catch-all rule's message template draws on two new placeholders beyond the table above: {schema-message} (the schema library's own raw error text) and {location} (the failure's JSON-pointer-style instance path, for when it doesn't cleanly name a single {attr}).

Rule Documentation

Each numbered validation rule is documented as its own Markdown file in a validation/ subdirectory inside the rule's own class's Data Sheet folder - e.g. specsheets/tasks/ExplicitODESimulation/v1.0.0/validation/ExplicitODESimulation-0001.md, next to that class's schema.json and description.md. A rule lives with the class it constrains and versions with it, so a new version directory (see Versioning) gets its own validation/ directory rather than threading a change through a separate, centrally-numbered file.

Rule IDs are [ClassName]-NNNN (e.g. ExplicitODESimulation-0001), not a flat global number, since classes aren't drawn from one obviously-ordered list. IDs are append-only within a class's validation/ folder, same as the flat numbering scheme described above. A namespace-owned class's rules follow a variant of this to stay collision-free and filename-safe - see Namespaces.

Each rule file is YAML frontmatter plus a prose body:

---
id: ExplicitODESimulation-0001
rule: The model attribute of an ExplicitODESimulation is required.
message: "Required attribute '{attr}' is missing from {class} '{id}'."
severity: error
status: active
check: schema
---

Longer explanation, edge cases, examples - not surfaced in a validation
error, unlike rule/message above.

rule is the constraint stated plainly, for a rule catalog or docs. message is the template actually used to build a validation error's text, with placeholders filled in per offending instance:

Placeholder Meaning
{class} class name of the element being validated
{id} id of the element being validated
{attr} the attribute that failed
{value} the attribute's literal value as written - a plain literal, or the reference string itself if it's a reference
{resolved-value} what {value} traces back to: itself for a literal, the target's literal value for a reference that resolves statically (e.g. to a constant); omitted when the reference can only be known at run time (e.g. a task's output)
{expected-type} the type the rule requires
{subvalue} the bracket/dot subelement portion of a reference, e.g. ['S1'] or .model
{allowed} comma-separated list of allowed values, for enum-style rules
{expr} a raw math/infix expression string, for AST well-formedness rules
{min} / {max} numeric bounds, for range/cardinality rules
{schema-message} the raw error message text from the JSON schema validator library, for a catch-all rule (see Schema-Pass Errors)
{location} the failing value's JSON-pointer-style instance path, when it doesn't cleanly name a single {attr} - a catch-all rule, a discriminator's dedicated missing-_type rule, or the longest-resolving prefix of a reference that fails to resolve
{prefix} a namespace prefix, e.g. saurolab
{function} a function name called in a math expression
{count} / {expected-count} an actual count and the count a rule requires, for index/argument-count rules
{parse-message} the raw error message text from the math-expression parser, for a parse-failure rule

Generated Rule Index

The generator emits one generated/rules-vX.json per distinct version string that appears as a directory name anywhere under specsheets/ (see Versioning) - e.g. if v1.0.0, v1.1.2, and v1.2.2 exist somewhere in the tree, it produces rules-v1.0.0.json, rules-v1.1.2.json, and rules-v1.2.2.json. Each file aggregates every class's rules as resolved for that specific version - each class contributing from whichever of its own directories is newest without exceeding that version - giving a complete rules manifest for any version a document might declare. It's generated fresh every run, not hand-maintained; the validation/ folders under specsheets/ stay the single source of truth. Nothing at runtime reads it - each generated library's validate() checks compile their message templates directly into code - so it exists for tooling: a rule catalog, coverage checks, external docs.

Cross-references

Almost every attribute in the document can have either a raw value ("5") or a reference to a value, as described in the spec. These references will be stored as text strings in their attributes, and their classes will need the functions 'isReference' ('true' if the value is a string that starts with "#") and 'getSEDReference', which returns a non-owning pointer to the referenced element (or NULL if no such element exists).

The full description of how references work is in the spec, but briefly: document objects are referred to by a hashtag, then their parent/child relationship with colons: "#tasks:task1:subtask2". From there, subobjects might be referenced with dots (#tasks:task1.model) If the object is a vector or a dictionary, subelements can use square brackets (#tasks:task1.model['S1']) Thus, the validator needs to know if those subelement references are plausible, but will not be able to return a pointer to the particular sub-element: they are run-time values that will only become literal values when the experiment is actually being performed. However, they can still be validated: 'hasSubvalue("string")' should return 'true' if (say) "[3]" or "['S1']" are legal subelements of the SED2 object.

Task Output Shapes

hasSubvalue() above needs to know, ahead of ever running anything, whether a given subelement reference is even plausible for a particular task - and a future typed output accessor in the generated libraries would need the same information to know what type to return. That's what each concrete tasks/ class's outputs.json is for: a machine-readable description of what every output suffix ([id], [id].model, [id].strings, and any class-specific ones like Loop's [id].aggregates/[id].range/[id].index) resolves to, replacing what used to be free-text "Valid"/"Invalid"/"Dimensions:" prose in description.md with something code can actually read. schema/outputs-meta.schema.json validates every class's outputs.json envelope; see core-spec.md Section 8 for the full file format and grammar.

The interesting design problem here wasn't the format so much as being honest about what's actually knowable and when, rather than collapsing everything into one "dynamic" bucket. A dimension's size or labels get tagged with where that knowledge comes from: "static" - computed from the task's own field values alone, right now (len(outputVariables), independentVariableRange.numberOfSteps, shapeOf(input) to inherit another reference's whole shape); "input-file" - computable in principle, but only by actually reading whatever external resource a field like model or location points to (JacobianFull's species count lives in the referenced SBML file, not the SED2 document; CsvImport's column headers live in the referenced CSV) - real validation before execution, but a strictly heavier lift than reading the document alone, and one the generator has to implement per file format (SBML, CSV, whatever DataImport.format names) rather than get for free from the grammar itself; or "runtime" - genuinely not derivable without actually executing the simulation, full stop, such as a variable-step-size ODE solver's row count or a Calculation's result shape. A hasSubvalue() implementation (or a future stricter validation pass) can treat these three differently - reject a reference outright on a static mismatch, defer to file-reading validation for an input-file case, accept-but-can't-fully-verify for a runtime one - rather than the single all-or-nothing "Valid"/"Invalid" the old prose gave it.

A shape can also end with a {"trailing": {...}} entry in dimensions: the dimensions of the output's own entries follow the listed ones (a Loop's [id] holds one entry per outputVariableMap key, and each entry may itself have dimensions; a CreateDataBlock result likewise). How many there are, and their sizes and labels, are not statically known, so a validator resolves the entry to an "open" placeholder dimension: it accepts any number of indices at that position without judging them (SEDBase-0009/-0010/-0011/-0014 stay silent), and it never makes a reference "still shaped" on its own (SEDBase-0015). The required of names the attribute whose values supply those dimensions (outputVariableMap, aggregateOutputVariables, data); the generator checks that the class has such an attribute and that the trailing entry is the last dimension. Validators do not use of yet, but it marks where to look once they are meant to work the dimensions out from the entries themselves, following references (including references to other tasks' outputs) and the literal shapes of lists and objects. All entries of such an output must have the same shape (they are stacked into one array); a mismatch is a runtime error of whatever executes the document, not something a validator can see.

The generator implements this notation by parsing it once, at generate time, and compiling it into real code in each target language - shape-inference and hasSubvalue() logic generated directly into each library - rather than shipping a small runtime interpreter for the notation in all three languages. This is the same "one canonical definition, three generated targets" shape the schema, the validation rules, and the math grammar already use (see Math). It also means every expr has to actually parse under the grammar core-spec.md Section 8 defines: a class's outputs.json can't fall back to free-text English the way description.md's old "Valid"/"Invalid"/"Dimensions:" prose could, since the generator has nothing to compile from a sentence.

Prior work

A previous speculative python-based library, along with several tests and example documents, was developed at C:\Users\Lucian\Desktop\sed Much of it is a translator, but the core class organization might be helpful, and the example JSON files should also be helpful, though it might contain used-to-be-valid documents, and shouldn't be taken as gospel a priori.