Skip to content

Latest commit

 

History

History
1418 lines (1023 loc) · 186 KB

File metadata and controls

1418 lines (1023 loc) · 186 KB

xmsconan — Consumer Guide

xmsconan is a build-orchestration toolkit for Aquaveo's XMS C++ libraries. You write one build.toml; xmsconan generates everything else: the Conan recipe, CMake glue, a build.py driver, a Python package skeleton, and a CI pipeline. It also ships the runtime helpers those generated files depend on.

This doc is what a consumer needs to know to set up, build, publish, and depend on an xmsconan-managed library.


1. The mental model

build.toml                       (you write this)
   │
   │  xmsconan gen     →   conanfile.py, build.py, CMakeLists.txt,
   │                       _package/pyproject.toml, .flake8, pytest.ini,
   │                       xms_conan2_file.py
   │
   │  xmsconan ci      →   .github/workflows/<Lib>-CI.yaml  OR  .gitlab-ci.yml
   │
   ▼
python build.py            ← runs the full Conan matrix locally
xmsconan publish           ← build + repair wheel + deploy to devpi + push to Conan

The C++ source you write lives in <library_name>/; tests live alongside. xmsconan gen regenerates the build files from build.toml on each invocation — they are not meant to be edited by hand.


2. Installation

pip install xmsconan
# or, from the Aquaveo dev index
pip install xmsconan -i https://public.aquapi.aquaveo.com/aquaveo/dev/+simple
# with the toolchain the generated CI jobs run
pip install "xmsconan[ci]" -i https://public.aquapi.aquaveo.com/aquaveo/dev/+simple

Conan 2 is a hard dependency and is installed transitively. You also need CMake ≥ 3.21 and a C++17 compiler on the system PATH for actual builds. The ci extra adds what the generated jobs install and nothing else: the conan patch series they pin, cmake, gcovr, and flake8 with the plugins the generated .flake8 expects (§10.3). It is how a workstation and a CI job are made to agree on all of them; the compiler is still yours to provide.


3. Quickstart

# 1. Drop a build.toml into the root of your repo (see §5 for the schema)
# 2. Generate everything that xmsconan owns
xmsconan gen --version 0.0.0 build.toml

# 3. Generate a CI pipeline (one-time; commit it)
xmsconan ci  --version 0.0.0 build.toml

# 4. One-shot Conan setup (adds the aquaveo remote, etc.)
xmsconan conan-setup

# 5. Build the full matrix locally
python build.py --version 0.0.0 --wheel-dir wheelhouse --artifacts-dir test_artifacts

Everything past step 1 is reproducible — re-run xmsconan gen whenever build.toml changes.


4. The unified CLI

All commands live under the xmsconan umbrella:

Command What it does
xmsconan gen Render build files (conanfile.py, build.py, CMakeLists.txt, _package/pyproject.toml, …) from build.toml.
xmsconan ci Render .github/workflows/<Lib>-CI.yaml or .gitlab-ci.yml.
xmsconan profiles Render conan_profiles/ and CMakePresets.json from build.toml. Run automatically by xmsconan gen; useful on its own after a [matrix] or [filter] edit.
xmsconan build Run conan install + cmake configure against a single profile. Used by the generated build.py; also useful for one-off configures.
xmsconan coverage Run the unified C++/Python coverage pipeline and enforce the [coverage] thresholds (see §11). --phase collect stops after producing the reports; --phase report gates an existing set of them (§11.4).
xmsconan test-shards Run a staged gtest runner as N parallel shards inside the current container and merge their JUnit reports into one. What xmsconan job test calls (see §10.2).
xmsconan job Run one generated CI job: build, test, package, deploy, lint, or coverage --pages. Everything each one needs beyond its own leg is read from build.toml and the job's environment rather than passed as flags, which is what lets a generated pipeline be one-line scripts (§10.4) and lets you replay a red job locally (§10.5).
xmsconan vs2019 Drive the manual Visual Studio 2019 (msvc 192) build and publish it to the aquaveo-vs2019 remote (see §16). Never runs in CI itself; GitLab reaches the same matrix through [ci].windows_vs2019 and build.py --platform instead.
xmsconan conan-setup Detect a Conan profile, add the aquaveo remote, optionally login. --remote-name / --remote-url name a different remote, and --append adds it after the remotes already configured rather than first — which is what a special-purpose remote such as aquaveo-vs2019 needs, so it does not become the first stop for every conan install on a shared machine. The password comes from --password-file, $CONAN_PASSWORD, or ~/.xmsconan.toml — there is no --password flag (§17).
xmsconan wheel-repair Run platform-appropriate wheel repair (auditwheel / delocate / delvewheel).
xmsconan wheel-deploy Upload repaired wheels to devpi with uv publish. The password comes from --password-file, $AQUAPI_PASSWORD, or ~/.xmsconan.toml, and reaches uv through its environment — never a command line (§13).
xmsconan conan-deploy Save / restore / upload Conan packages between CI stages. --remote picks the destination; --package-query restricts both the save and the upload to matching binaries (e.g. compiler.version=192). The query is required when publishing to a toolchain-specific remote from a shared runner: conan cache save and conan upload match by reference, and a CI runner's Conan cache is per machine, not per job.
xmsconan publish The full release pipeline (gen → build → repair → deploy).

Run xmsconan <cmd> --help for the full flag set, and xmsconan --version (or -V) for the installed xmsconan version — the first thing a bug report needs. The legacy underscored names (xmsconan_gen, xmsconan_ci, …) still work as aliases; the generated CI calls xmsconan <cmd>.

gen, ci, profiles, build, coverage, test-shards, publish, and every vs2019 verb take -v / --verbose and -q / --quiet (after the verb for xmsconan vs2019 <verb>); the four CI-only tools (conan-setup, wheel-repair, conan-deploy, wheel-deploy) take neither, and neither does job — a CI job's output is a log someone reads after the fact, so it always prints at the default level and folds itself into sections instead (§10.4). The default prints progress and warnings; -q keeps only errors; -v adds debug detail and is what turns a one-line failure into a traceback — see §4.2. Report output — the configuration table, vs2019 build's preflight block and summary table, the --check diff — goes to stdout regardless; progress and diagnostics go to stderr through logging, so xmsconan gen --check > drift.txt captures the diff and nothing else. coverage is the exception to watch: its summary and the Coverage total: line GitLab scrapes are progress, so -q silences the pipeline's coverage number — do not add it to the generated job.

4.1 --dry-run and --check on the generators

gen, ci, and profiles each decide their whole output as one plan — the same set of files a real run writes — and --check compares that plan with the tree instead of keeping a list of its own:

Flag What it does Exit code
(none) Write every file in the plan. 0, or 1 on error
--dry-run Log what a real run would write. Writes nothing. 0, or 1 on error
--check Print a unified diff for every file that is missing or differs, and name any stale profile gen or profiles would delete. Writes nothing. 0 when the tree is up to date, 1 otherwise
xmsconan gen --check --version 0.0.0 build.toml

--check is the enforcement half of "regenerate after any build.toml edit". The generated files are gitignored in the consuming repositories, so a stale one is not a diff anyone reviews — it is a working copy quietly building something the repository no longer describes. Running xmsconan gen --check in CI, or from a pre-commit hook, turns that into a red pipeline instead.

Both sides of the comparison are normalized to LF, so a Windows working copy of an LF-committed generated file is not reported as drift. gen --check and profiles --check also report profiles a real run would delete — a dropped [matrix] entry leaves a profile behind that an IDE will still configure from, and a check that only compared the files it renders would call that tree clean.

4.2 Exit codes and failure reporting

One vocabulary across every command, defined in xmsconan/exit_codes.py. One number means one thing because the generated CI keys off the numbers: GitLab's allow_failure: exit_codes: forgives the coverage gate and nothing else (§11.6), and any wrapper script or && chain reads every nonzero code as "stop".

Code Meaning Produced by
0 The command did what it was asked. all
1 The tool failed: an unhandled exception, a file that could not be written; for gen/ci/profiles --check, drift (§4.1); for vs2019 build, a failed library (§16.4). all
2 A bad request or a machine that cannot honour it. argparse produces it for a bad flag; vs2019 for a missing --root, a failed preflight, or a conan that is not on PATH; build for a conan or cmake it cannot find. all
3 A gate the run was asked to enforce did not clear while the tool itself worked: a coverage layer below its threshold. The one code CI forgives, and only under [ci].split_tests. coverage
4 The run completed but produced nothing: every selected library was skipped. vs2019 build

A child process that ran and failed — conan, cmake, build.py — is reported with its own exit code, so a conan create that failed with 6 is exit 6 from xmsconan build, publish, or vs2019 setup too. A child killed by a signal is 1 from build (and from gen, ci, and profiles, which spawn nothing); the other commands hand the negative code to sys.exit, which the shell reports as 256 minus the signal number.

How a failure is reported is one contract for gen, ci, profiles, and build: one ERROR: line on stderr with the exception's message, and no traceback. Pass -v for the traceback: the same failure is then reported with the full stack, and a child's full command line — which without -v is reduced to the program name, so a credential that reaches a command line by mistake does not reach the log by contract. xmsconan coverage always prints the traceback: its failures are read from a CI job log, where -v cannot be added after the fact. vs2019, test-shards, publish, and the four CI-only tools report the failures they anticipate — a child that failed, a missing tool, a bad request — in one line and let anything unexpected out as a Python traceback, except in the step of publish and vs2019 build that writes the build files, which reports anything the generator raises in one line, as gen does (§15, §16.4). Bringing them under the same contract is planned cleanup.


5. build.toml reference

build.toml is the only file you author for the build system. It controls everything xmsconan generates.

An unknown top-level key is an error. Every tool that reads the file — xmsconan gen, ci, profiles, coverage, publish (with or without --docker) and vs2019 — rejects a key that is not in the tables below, naming it and listing what is accepted. Every optional key has a documented default, so a misspelling otherwise had no symptom at all — the default was kept and the generated artifact quietly was not what the file asked for. The same rule already applied to the [ci], [matrix], conan_profile_variants and vs2019_dependency_overrides sub-tables; it now covers the top level too. The [ci] table's key and type check now runs in every tool that reads the file, not only xmsconan ci. testing_framework, python_binding_type and the keys of xms_dependency_options are checked against their vocabularies at the same point.

5.1 Required

Field Type Description
library_name string Conan / CMake project name. e.g. "xmscore".
description string One-line summary; flows into conanfile.py and the wheel metadata.

5.2 Source layout

Field Default Description
library_sources [] C++ implementation files (.cpp) for the static library.
library_headers [] Public headers exported to consumers.
testing_sources [] .cpp files compiled into the library when testing=True, so dependent libraries can link the testing helpers. Excluded from Python builds — see the note below.
testing_headers [] Test fixture / helper headers (*.t.h for cxxtest).
python_library_sources [] C++ files compiled only when pybind=True.
python_library_headers [] Headers compiled only when pybind=True.
pybind_sources [] Pybind11 binding .cpp files.
pybind_headers [] Pybind11 binding headers.

Paths are interpreted relative to the directory build.toml lives in.

Where testing_sources end up

Under testing=True they are compiled into the library itself, and the test runner picks them up by linking it. This is deliberate: xms libraries publish testing helpers (xmscore's ttEqualPointsXYZ, ttTextFilesEqual, ...) that downstream libraries link from the package they already consume, so the helpers have to live in the library that gets packaged.

Under pybind=True they are excluded from the library and the runner compiles them directly instead. The pybind module links the main library, and these translation units reference cxxtest helpers such as CxxTest::charToString — declared in cxxtest/ValueTraits.h but defined only in cxxtest/ValueTraits.cpp, which is pulled in solely by the cxxtestgen-generated runner.cpp. A pybind module carrying them fails dlopen with an undefined symbol.

The two options are never combined in a generated configuration (the packager fans out wchar_t, pybind and testing independently rather than cross-multiplying), so in practice testing=True always means the helpers are in the library. The IS_PYTHON_BUILD guard exists for hand-built and option-overridden configurations.

5.3 Dependencies

Field Type Default Description
xms_dependencies array[object] [] XMS sister libraries. Object shape: { name = "xmscore", version = "7.0.0", no_python = false }. no_python = true excludes the dep from _package/pyproject.toml. Each entry must be a table with name and version as strings and, if present, no_python as a boolean; any other key is rejected, not ignored.
xms_python_dependencies array[string] [] Extra Python requirements written into _package/pyproject.toml, in pip requirement form ("geopandas", "data_objects>=4.0.0"). For runtime imports that are not XMS sister libraries and so have no entry in xms_dependencies. Appended after the XMS entries, in the order given. The Conan dependency graph is unaffected — nothing is added to conanfile.py — but these are real wheel requirements, so pip resolves them from an index during the Conan build when pybind is on (see §5.3).
extra_dependencies array[string] [] Extra Conan deps in "name/version" form. Each entry is also wired into the generated CMakeLists.txt — a find_package(<name> REQUIRED) in the non-conda branch plus the matching EXT_INCLUDE_DIRS / EXT_LIB_DIRS / EXT_LIBS appends, exactly as xms_dependencies entries are. A header-only package defines no _LIBRARY_DIRS or _LIBRARIES; CMake expands an undefined variable to nothing, so those appends are harmless. Entries the generated CMakeLists.txt already finds are dropped from the CMake side — every xms_dependencies entry, plus boost, zlib, pybind11, cxxtest and gtest. That is not cosmetic: Conan publishes boost's and zlib's configs as Boost and ZLIB, so a lowercase duplicate is a hard configure error, and pybind11 is found inside the IS_PYTHON_BUILD block where Python is available. So the CMake side and the recipe side apply the same rule. Deduplicated by package name against the deps the recipe adds itself (boost, zlib, the test framework, pybind11, every xms_dependencies entry) and against the rest of this list, keeping the first reference. Conan's duplicate check is by name, so a clash is a hard graph error no version dodges — this is what lets a library whose static half needs pybind11 list it here without breaking its pybind=True configurations. An exact duplicate is dropped silently; an entry whose version differs from the reference that wins is a warning naming both, since what is being discarded is a pin somebody wrote on purpose. Entries are whitespace-stripped, and an empty entry is an error.
extra_dependency_cmake_names object {} Override the CMake package name find_package is called with for an extra_dependencies entry, keyed on its Conan package name: { "xmdf" = "Xmdf" }. Needed only when a package's CMake config does not use its Conan reference name. An empty string ({ "somepkg" = "" }) keeps the dependency in the Conan graph but leaves it out of the generated CMakeLists.txt entirely — for a package that ships no CMake config. Values must be strings, and a key naming a package that is not in extra_dependencies is rejected, not ignored: an ignored key's only symptom was a find_package failing on the dependency, pointing nowhere near the misspelled key.
xms_dependency_options object {} Override an XMS dep's options. e.g. { "xmscore" = { "pybind" = false } }. A key naming a package that is not in xms_dependencies is rejected, not ignored — an ignored key meant the override (usually one turning a dependency's pybind off) never applied, and the only symptom was a heavier build than asked for.
vs2019_dependency_overrides object {} Replace an XMS dep's reference — but only on a Visual Studio 2019 (msvc 192) build. e.g. { "xmscore" = "xmscore/[>=6.0.1 <7.0.0]" }. Matched on the package name before the first /; every other toolchain ignores it entirely. An entry may change the version or range only — renaming the package, or naming one that is not in xms_dependencies, fails the msvc 192 build. See §7.4.
conan_profile_conf object see §5.3 note [conf] entries written into the [conf] section of every generated profile. Defaults to { "tools.cmake.cmaketoolchain:generator" = "Ninja Multi-Config", "tools.cmake.cmaketoolchain:user_presets" = "" } — the generator is pinned so it does not vary by machine, and Conan's own CMakeUserPresets.json is disabled because xmsconan writes CMakePresets.json instead. An empty table ({}) omits the section entirely; with no generator pinned there is nothing to express, so no CMakePresets.json is written either.
conan_profile_variants array[object] [] Additional renderings of the same settings under a different generator — the same configuration built with both Ninja and Visual Studio, for instance. Object shape: { name = "vs", platforms = ["windows"], kinds = ["testing", "python"], conf = { "tools.cmake.cmaketoolchain:generator" = "Visual Studio 17 2022" } }. Only name is required. platforms accepts linux, mac_os (note the underscore — it is the profile-filename spelling of Conan's Macos), and windows; kinds accepts library, python, and testing. An omitted filter means "no restriction". A variant overlays [conf] only — settings and options are identical to the base rendering, which is what makes the pair comparable — and each match is written as <stem>_<name>.txt with its own CMake preset. Unknown keys and unknown platforms / kinds values are rejected when the profiles are generated, rather than silently producing no variant.
conan_profile_options object {} Per-package options written into the [options] section of every generated profile. e.g. { "boost" = { "shared" = true } }. The wildcard "*" is supported (e.g. { "*" = { "shared" = true } }); a more specific entry overrides the wildcard.

Boost (1.86.0) and zlib (1.3.1) are added automatically by the recipe. On a Visual Studio 2019 (msvc 192) build the recipe swaps in the legacy stack instead — the same two packages at the versions the aquaveo-vs2019 remote publishes msvc 192 binaries for: boost 1.74.0.3 and zlib 1.2.11. See §7.4 and §16.

5.4 Build configuration

Field Default Description
testing_framework "cxxtest" "cxxtest" or "gtest". Selects the test discovery / runner template in CMake. Any other value is rejected at generate time and again in the recipe's configure() — it used to add no framework requirement at all and fail much later with a CMake "cannot find cxxtest".
python_binding_type "pybind11" "pybind11" or "vtk_wrap". Any other value is rejected the same way; it used to be silent end to end and ship a Python package with no native module in it.
python_namespaced_dir derived The submodule under xms.<...>. e.g. "core" produces xms.core. Defaults to library_name minus the xms prefix when omitted.
pybind_root false Whether this library hosts the root xms namespace.
pybind_advertises_module false Advertise the pybind module's import library (_<name>) to C++ consumers instead of the static library, and install the module to bin/ + lib/. Windows-only in effect, and only for pybind = True packages. Opt-in because a consumer of the module sees only its exported symbols, and because pybind propagates down the dependency graph. See §7.5.

5.4.1 Which configurations get built ([matrix] table)

The fan-out is otherwise fixed: on Windows, build_type × compiler.runtime gives 4 base configurations, plus a wchar_t=typedef copy of each, plus a testing=True copy of each, plus one pybind configuration per Python version — 13 for a single-version library. [matrix] narrows or widens that. Every key is optional; omit the table for the historical fan-out.

Field Default Description
[matrix].compiler_runtime ["dynamic", "static"] Which MSVC runtimes to build. Applied to the base matrix before the product, so the wchar_t and testing copies shrink with it — ["dynamic"] turns 13 configurations into 7. Inert on Linux and macOS, which declare no compiler.runtime: one build.toml serves every platform, so a Windows-only statement must not fail elsewhere. Use it for a library nothing consumes a static-CRT build of.
[matrix].wheel_only false Build only what a wheel release needs: Release with tests, Debug with tests, and the pybind build — three configurations for a single-ABI library, down from 5 on Linux/macOS and 13 on Windows. The two testing legs are fixed; the pybind leg is not, so a library naming two python_versions gets four configurations and one naming two pybind_build_types as well gets six. It drops the library-only configurations (nothing consumes this library's Conan package, so a build with no tests in it proves nothing) and the whole wchar_t=typedef fan-out, and it narrows compiler_runtime to ["dynamic"] unless the key is set explicitly — a pybind module links the dynamic CRT, so a static-CRT leg cannot match it. The three that remain differ only in build_type and pybind, which is the point: a wheel that passes its tests in a configuration nothing else shares is not evidence about the wheel that ships. Combine with pybind_build_types to choose which build type carries the pybind leg.
[matrix].pybind_build_types ["Release"] Which build types get a pybind configuration. Add "Debug" for a library whose consumers link a Debug module (bin/_<name>_d.<abi>.pyd and its import library — that bin/ install and the _<name>_d rename both require pybind_advertises_module; without it the Debug module keeps its plain name under _package/). On Windows the Debug leg publishes no wheel and runs no Python tests (§7.5); Linux and macOS Debug legs build and test theirs as usual. XMS_COVERAGE=1 no longer adds Debug on top: xmsconan coverage takes its Python coverage from the Release pybind build, so this key alone decides the pybind legs.
[matrix]
compiler_runtime = ["dynamic"]              # nothing consumes a /MT build of this library
pybind_build_types = ["Release", "Debug"]   # the desktop application links the Debug module
wheel_only = true                           # only the wheel ships; 3 builds, not 13

An unknown key, a value outside the accepted set ("MD", "RelWithDebInfo"), or an empty list is rejected when the packager is constructed. An empty list would build nothing at all, which looks exactly like a successful build — omit the key instead.

The table reaches the packager as CONAN_MATRIX in the generated conanfile.py, which build.py forwards. xmsconan vs2019 reads the build.toml in each checkout directly, so the same table applies on that track (§16).

5.5 CMake escape hatches

Field Default Description
extra_cmake_text "" Raw CMake injected near the top of CMakeLists.txt.
post_library_cmake_text "" Raw CMake appended after the library target is defined.
extra_export_sources [] Additional files / directories Conan exports with the recipe (e.g. ["test_files"]).

5.6 CI configuration ([ci] table)

These drive the CI templates. All optional.

Field Default Description
ci_type "github" or "gitlab". Required for xmsconan ci. (Lives at the top level, not under [ci].)
[ci].windows true Emit a Windows job.
[ci].linux true Emit the Linux jobs. GitLab only, like [ci].windows — the GitHub templates ignore both. Setting it to false also removes Repair Wheel, Wheel Deploy, Conan Deploy - Linux and the Package stage: those three jobs are Linux-specific and consume the Linux job's artifacts through dependencies:. The Windows wheel jobs are unaffected — each platform stages and publishes its own wheel (§10.2). Intended for libraries that cannot build on Linux at all; see the note below.
[ci].windows_vs2019 false Also build the windows_vs2019 (msvc 192) matrix and publish it to the aquaveo-vs2019 remote. GitLab only — a GitHub project setting it gets a warning and no jobs. Emits Conan Build - Windows VS2019 and (with [ci].deploy) Conan Deploy - Windows VS2019, beside the msvc 194 pair rather than instead of it, so it requires [ci].windows and is rejected with it off. Opt-in because it roughly doubles the Windows half of a pipeline and only the libraries the VS2019-era desktop products consume need msvc 192 binaries. No wheels: see §10.2.
[ci].windows_wheel_repair derived from ci_type: true on github, false on gitlab Repair the Windows wheel. false skips the in-place delvewheel repair in the Windows build job on both GitLab and GitHub, drops the same step from xmsconan publish when it runs on Windows and from the xmsconan vs2019 track, and skips staging the Conan cache's DLLs for a repair that no longer happens. On both forges the flag is read by xmsconan job build (§10.4) rather than rendered into the job, so changing it takes effect without regenerating the pipeline. The unrepaired wheel is still built, uploaded as an artifact, and deployed. Windows-scoped by design — see §12.1 for the default's rationale.
[ci].linux_arm false Emit a Linux ARM job (GitHub only).
[ci].deploy true Emit deploy jobs (only run on tag pushes).
[ci].coverage false Emit a coverage job. On GitLab adds a Coverage stage + Pages upload; on GitHub adds a separate Coverage.yaml workflow. Both delegate to xmsconan coverage. Thresholds and filters come from [coverage] (see §5.7).
[ci].xvfb false Wrap test execution in xvfb-run (use for libraries that link X11/VTK).
[ci].split_tests false Split build and C++ test into two stages so testing artifacts can be reused. The Test stage gets one Run C++ Tests - <label> job per testing configuration the build stages — Release-testing and Debug-testing on the default matrix — each naming its own artifact directory with --label (§10.2). Those jobs are except: - tags, so a tag pipeline never runs the Linux suite: it is left to the branch pipeline of the commit being tagged.
[ci].test_shards 0 When >1, run the C++ tests as N gtest shards inside a single container. The recipe skips cmake.test() in this mode, so the shards are the only thing that runs the C++ tests — a missing runner binary fails the build rather than being skipped. Every xmsconan job build reads the count and shards in place, with one exception: under split_tests = true, GitLab's Linux build skips the C++ tests at any count, and each Run C++ Tests - <label> job runs xmsconan job test, which reads the count from build.toml, forks N runner processes (one when test_shards is 0 or 1) and merges their JUnit XML into one TEST-cxxtest.xml (§10.2). The shard count is per test job, so a two-configuration matrix runs 2 × N processes across two containers. GitLab's Windows builds have no test job, so they shard in place under split_tests too. The count is rendered into neither pipeline, so raising it is a build.toml edit on either host, with no CI regeneration. This was previously GitLab's parallel: N, which bought its concurrency by starting N containers — each one repeating the container start, the pip install and the artifact download to run one Nth of the suite.
[ci].docker_image "" Override the build container image (skips the default Aquaveo images).
[ci].python_versions ["3.13"] Python versions to build on Windows. macOS and Linux take the highest entry unless they are given their own list below. Set to ["3.10", "3.13"] to build a Windows 3.10 wheel + Conan binary in addition to 3.13. See §8.
[ci].mac_python_versions highest of [ci].python_versions Python versions the macOS matrix fans out across. Separate from python_versions so a Windows-only ABI (3.10 ships in Windows wheels for the desktop products) does not silently multiply the mac matrix. See §8.
[ci].linux_python_versions highest of [ci].python_versions Python versions the Linux and Linux-ARM matrices fan out across. Every entry needs a matching conan-gcc13-py<version> container to exist, which is why this tracks the published images rather than python_versions. See §8.

All three lists are checked against the conanfile's python_version option at generation time. A version the recipe does not allow fails xmsconan ci with a message naming the supported set, rather than generating a matrix leg that dies later at conan configure time in CI.

5.7 Coverage thresholds ([coverage] table)

Consumed by xmsconan coverage; only relevant when [ci].coverage = true (or when you run the tool locally). All optional.

Field Default Description
[coverage].parallel false Run the C++ and Python coverage builds at the same time rather than one after the other. The two builds are independent — different build types, different Conan packages, separate build folders — so overlapping them looks like it should halve the coverage job. It defaulted to true and was changed to false, for two independent reasons. Conan 2's local cache is not safe for concurrent writes: two conan creates registering a recipe at once hit a uniqueness constraint, which is what broke xmsvtk's Coverage stage. And the legs contend for CPU — an identical shard, timed by gtest itself, went from 245s to 386s (1.57×) with a second leg beside it — so the overlap spends more wall clock than it saves while holding twice the runner capacity. Set to true only where neither applies: a runner with a private Conan cache and cores to spare. An [ci].xvfb library whose image tests share one display must leave it off regardless. Each leg's output is buffered and replayed as one block when it finishes, so a compiler diagnostic still appears under the file name that produced it. Must be a real TOML boolean: a quoted "false" is a string, and generation rejects it rather than letting a non-empty string silently count as true.
[coverage].cpp_threshold 0 Minimum C++ line coverage percent. xmsconan coverage exits non-zero when gcovr reports below this.
[coverage].python_threshold 0 Minimum Python line coverage percent (pytest-cov).
[coverage].filters ["<library_name>/"] gcovr --filter patterns. Defaults to the library's own source tree.
[coverage].excludes [".*\\.t\\.h$", ".*/_package/tests/.*"] gcovr --exclude patterns. Strips test fixtures and the Python test tree from the C++ measurement. The pybind layer is not excluded any more: it was, back when only the testing build was read — and that build does not compile the bindings, so the exclude removed nothing that existed. Now that the pybind build is instrumented and merged in, excluding it would collect the binding layer's coverage and then throw it away. Name it here to restore the old behavior.
[coverage].python_version highest of [ci].linux_python_versions (default "3.13") The single Python ABI the pybind coverage build is pinned to. The fallback reads the Linux list because coverage only runs on Linux, and on GitLab the resolved version also selects the container image — taking the highest Windows entry could name a version with no published image. xmsconan coverage runs two builds: a testing=True+pybind=False+Debug build for C++ coverage (no ABI dependency) and a pybind=True+testing=False+Release build for Python coverage that gets pinned to this version. Multi-Python fan-out is intentionally collapsed on the pybind side so the Python report is deterministic. Override only when the highest CI version isn't the one you want to gate on. The resolved value is also exported as PYTHON_TARGET_VERSION into both coverage builds, so the matrix build.py generates can actually satisfy the --filter applied to it — --filter narrows an existing matrix and cannot introduce an ABI the matrix lacks. It overrides any ambient PYTHON_TARGET_VERSION (and logs a warning naming both values, plus the [coverage].python_version setting that would honor the one it ignored): the same resolved value drives the pybind --filter and the package lookup, and all three must agree or the build produces one ABI and the lookup searches for another. Like the [ci] lists, an explicit value here is checked against the conanfile's python_version option at resolution time — the coverage run reads this key directly rather than through the [ci] validation, so the check lives with the resolver and fires for xmsconan ci and xmsconan coverage alike.

Both thresholds default to 0, which means "report only, don't gate." Set them to real values once a baseline has been established.

An unknown [coverage] key is rejected, the same as an unknown [ci] key. Thresholds may be written as integers; they are read as floats. A boolean threshold, a filters or excludes value that is not a list, and an unquoted python_version (which TOML reads as a float) are rejected too.

5.8 Build matrix filter ([filter] table)

A baseline restriction on the configuration matrix, for libraries that should never build part of it (no Debug packages, no Python bindings, dynamic runtime only). The table uses exactly the shape build.py --filter takes as JSON: top-level Conan settings plus the nested options and buildenv tables.

[filter]
build_type = "Release"       # a top-level Conan setting
"compiler.runtime" = "dynamic"

[filter.options]
pybind = false               # this library ships no wheel
Key Accepted keys Accepted values
top level os, arch, build_type, compiler, compiler.version, compiler.cppstd, compiler.runtime, compiler.libcxx A single value that some configuration actually carries — "Release", not ["Release"] and not "release". Settings a platform doesn't emit (compiler.runtime off Windows) are ignored there rather than matching nothing.
[filter.options] wchar_t, pybind, testing, python_version, coverage pybind / testing take true or false; wchar_t takes "builtin" or "typedef"; python_version takes a quoted "X.Y" string that appears in any of [ci].python_versions, [ci].linux_python_versions or [ci].mac_python_versions — a version only one platform builds still counts (unquoted 3.13 is a TOML float and is rejected). coverage takes only true, and only in a runtime --filter during an xmsconan coverage run — normal builds omit the option entirely, so a permanent pin in build.toml's [filter] table would exclude every normal configuration and is rejected at generation time with a message saying so.
[filter.buildenv] the names the generated profiles set: XMS_VERSION, PYTHON_TARGET_VERSION, CI_COMMIT_TAG, RELEASE_PYTHON, XMS_TEST_ARTIFACTS_DIR, XMS_TEST_ARTIFACTS_LABEL, MACOSX_DEPLOYMENT_TARGET, _PYTHON_HOST_PLATFORM A single value, compared against what the profile sets. The name is validated, the value is not: most of these are read from the environment at build time (XMS_VERSION, CI_COMMIT_TAG), so whether a pin matches is only knowable in the build environment and xmsconan gen does not judge a [filter.buildenv] pin unbuildable.

Everything above is checked by xmsconan gen and xmsconan ci, including whether the filter as a whole selects anything: a combination no configuration can satisfy (testing and pybind both true, say) fails generation rather than every later build. That check reads [matrix] (§5.4.1) too, since that table decides which configurations exist at all — [matrix] compiler_runtime = ["dynamic"] with [filter] "compiler.runtime" = "static" is two individually valid statements that together build nothing, and is rejected by name.

How it flows through the generated files:

  • xmsconan gen bakes the table into build.py as BUILD_FILTER. Every python build.py invocation applies it first, prints Applying build.toml [filter]: …, then applies any --filter on top — the two AND together. --ignore-build-filter skips it for a one-off build of an excluded configuration.
  • Filters that cancel out now fail. When nothing survives, build.py prints which filters it applied and exits 1 instead of "succeeding" with zero packages built. --preview prints the same message but still exits 0 — it is the flag you reach for to diagnose this.
  • xmsconan ci reads the same table:
    • The build_type matrix in every generated GitHub job keeps only the build types that still have configurations. A pinned build_type is the obvious case, but so is [filter.options] pybind = true: with the default [matrix].pybind_build_types of ["Release"] the Debug leg has nothing to build, and a leg that builds nothing exits 1. GitLab has no build_type matrix — its jobs run the whole matrix through build.py, which honors the filter anyway.
    • The wheel steps (repair, artifact upload, devpi deploy) come out per platform, whenever that platform keeps no pybind configuration — which a filter can do without ever naming pybind. options.pybind = false is the explicit case; build_type = "Debug" against the default [matrix].pybind_build_types and options.testing = true (the testing and pybind variants are disjoint) both do it on every platform, and "compiler.runtime" = "static" does it on Windows only — msvc builds a pybind module for the dynamic runtime alone, while macOS and Linux declare no compiler.runtime at all and keep their wheels. xmsconan job package fails on an empty wheelhouse/ and xmsconan job deploy --wheels-only refuses to publish nothing, so a wheel-less library would otherwise get a red pipeline on every branch. The build step itself needs no such gate any more: it asks the configurations it built whether a wheel exists, so a leg that kept no pybind configuration simply stages none (§10.1). This covers the Windows legs as well, which repair in place inside the build job rather than in a separate one: GitLab drops its Repair Wheel and Wheel Deploy jobs and Wheel Deploy - Windows, while the Conan Deploy jobs — which publish packages, not wheels — stay.
    • os, arch, and compiler* pins are only warned about, not applied. The platform fan-out lives in separate job blocks rather than a matrix axis, so there is nothing to narrow: [filter] os = "Windows" leaves the macOS and Linux jobs to fail with an empty matrix. xmsconan ci names each such job at generation time; drop the pin, or turn the job off through [ci].
  • xmsconan coverage builds Debug + testing=True (C++ report) and Release + pybind=True (Python report), pinned to [coverage].python_version. Because those two pin the inverse of each other, a filter conflicts by requiring an option as much as by excluding it — pybind = true cancels the C++ build just as pybind = false cancels the Python one. They also pin different build types, so any build_type pin cancels exactly one of them (§11). xmsconan ci warns about each conflict when [ci].coverage = true.

Two related changes reach repos with no [filter] table, since --filter and [filter] share one validator:

  • A --filter value that could never match (a list, a misspelled option, "release") now raises instead of silently matching nothing.
  • A --filter naming a setting the running platform doesn't emit (compiler.runtime on Linux) is now a no-op there instead of an "Unknown filter key" error, so one filter can be used across platforms.

5.9 Example

library_name = "xmscore"
description = "Support library for XMS products"
ci_type = "github"

python_namespaced_dir = "core"
pybind_root = true

xms_dependencies = []

library_sources = [
    "xmscore/math/math.cpp",
    "xmscore/misc/StringUtil.cpp",
]
library_headers = [
    "xmscore/math/math.h",
    "xmscore/misc/StringUtil.h",
]
testing_sources = ["xmscore/testing/TestTools.cpp"]
testing_headers = ["xmscore/math/math.t.h", "xmscore/testing/TestTools.h"]
pybind_sources  = ["xmscore/python/xmscore_py.cpp"]

[ci]
linux_arm = true
python_versions = ["3.10", "3.13"]

6. What xmsconan gen writes

After xmsconan gen --version X.Y.Z build.toml you will have (alongside build.toml):

.
├── build.toml
├── conanfile.py              # Conan recipe (extends XmsConan2File)
├── build.py                  # Driver: orchestrates the conan-create matrix
├── CMakeLists.txt            # CMake project — ALL knobs are cache vars
├── xms_conan2_file.py        # Runtime helper imported by conanfile.py
├── pytest.ini
├── .flake8
└── _package/
    └── pyproject.toml        # Python package metadata for the wheel

Don't hand-edit these. Treat them like generated code: regenerate from build.toml on every change. The exception is xms_conan2_file.py, which is copied (not rendered) — it's part of xmsconan itself and updates whenever you upgrade the xmsconan Python package.

xmsconan gen --check verifies that, over every file above plus conan_profiles/ and CMakePresets.json (§4.1).


7. The Conan recipe — what it exposes

The generated conanfile.py is a thin subclass of xmsconan.xms_conan2_file.XmsConan2File. The interesting bits for consumers:

7.1 Settings

Standard Conan: os, compiler, build_type, arch.

7.2 Options

Option Values Default What it controls
wchar_t "builtin" / "typedef" "builtin" MSVC /Zc:wchar_t- toggle. Only "typedef" is built on MSVC (and is excluded from non-MSVC). The recipe sets USE_TYPEDEF_WCHAR_T from it, which is what the generated CMakeLists.txt gates the flag on. Typedef packages published before the release carrying this fix do not carry /Zc:wchar_t- — nothing set that variable between the Conan 2 migration and that release, so those binaries are builtin builds with a typedef package_id. Nothing in the package metadata distinguishes them -- it records the option, not the flag it failed to produce -- so age is the only signal: rebuild and republish the typedef variants of a library before consuming them (see the rollout order in §7.4).
pybind True / False False Build the Python binding module + wheel. Allowed for any build_type.
testing True / False False Build the test runner. Mutually exclusive with pybind=True in the standard packager fan-out — the coverage runner instruments each shape in a separate Conan create rather than combining them.
python_version "3.10" / "3.13" / "3.14" "3.13" Which Python ABI to target when pybind=True. Dropped from package_id when pybind=False, so non-Python builds remain a single binary regardless.
coverage True / False False Instrument the build (--coverage -O0 -g, plus pytest-cov on pybind builds). Part of package_id when True, so an instrumented binary can never satisfy --build=missing for a production build or vice versa. Dropped from package_id when False, so uninstrumented ids are byte-identical to the ids this recipe produced before the option existed. Set by the packager when xmsconan coverage runs — not something to pin by hand.

7.3 Required CMake variables (set by the recipe via tc.variables)

PYTHON_TARGET_VERSION, IS_PYTHON_BUILD, BUILD_TESTING, XMS_TESTING_FRAMEWORK, XMS_VERSION, USE_TYPEDEF_WCHAR_T. The generated CMakeLists.txt already wires these up; only relevant if you write extra_cmake_text.

All of them come from a single _cmake_variables() on the recipe, which feeds both generate() (the toolchain file) and build() (cmake.configure). Those were two hand-kept copies, and USE_TYPEDEF_WCHAR_T is what that cost: neither of them set it. XMS_COVERAGE is the one variable deliberately outside the shared set — set in build() only, and only when the recipe's coverage option is True. The option is part of the package_id, so conan gives the instrumented build its own build folder and the cached -D value can never leak into a production configuration. The generated CMakeLists.txt no longer reads $ENV{XMS_COVERAGE} as a fallback; the option is the sole driver, so instrumentation always travels with the package_id.

7.4 Third-party requirements and the VS2019 fork

The recipe resolves a different third-party stack when it detects Visual Studio 2019 — compiler == msvc and compiler.version == 192. The third-party half of the fork is automatic: there is nothing to add to build.toml for it, and the same conanfile.py builds both stacks. Three class attributes on XmsConan2File define the fork:

Attribute Default Set from build.toml? What it controls
default_requirements ["boost/1.86.0", "zlib/1.3.1"] no Third-party (non-XMS) requirements for the modern toolchains: gcc 13, apple-clang 17, msvc 194.
vs2019_requirements ["boost/1.74.0.3", "zlib/1.2.11"] no Third-party requirements used instead on msvc 192. Same package set as default_requirements, at the versions the aquaveo-vs2019 remote publishes msvc 192 binaries for: the Aquaveo legacy boost the desktop products link against, and zlib 1.2.11 (zlib/1.3.1 exists only on aquaveo-stable, msvc 194 only, so naming it here would force a from-source build or fail). zlib is not optional on either stack — the generated CMakeLists.txt calls find_package(ZLIB REQUIRED) and sources such as daStreamIo.cpp include <zlib.h>.
vs2019_dependency_overrides {} yes — the [vs2019_dependency_overrides] table (§5.3) Per-library replacement of xms_dependencies references on msvc 192, e.g. {"xmscore": "xmscore/[>=6.0.1 <7.0.0]"}. Matched on the package name before the first /. Legacy desktop products pin xmscore 6.x while the Conan 2 line is at 7.x. Every other toolchain ignores this dict entirely — including its invalid entries, since the two rules below are checked only on msvc 192.

Shared by both stacks: pybind11/3.0.1 when pybind=True, and the testing_framework requirement (cxxtest/4.4 or gtest/1.17.0) when testing=True. pybind11 on VS2019 is intentionally 3.0.1 — an upgrade from the Conan-1-era 2.9.1, not an oversight.

The split in that third column is deliberate. default_requirements and vs2019_requirements describe the shared third-party stack — every XMS library resolves the same boost, so they live in xmsconan and are changed there. vs2019_dependency_overrides is a per-library statement about that library's own sister dependencies, so it is a build.toml key:

library_name = "xmsgrid"
description = "Geometry library for XMS products"

[[xms_dependencies]]
name = "xmscore"
version = "7.0.0"

# On msvc 192 only, resolve xmscore from the 6.x line the desktop
# products pin instead of the 7.0.0 above.
[vs2019_dependency_overrides]
xmscore = "xmscore/[>=6.0.1 <7.0.0]"

which xmsconan gen emits onto the generated recipe subclass:

class XmsgridConanFile(XmsConan2File):
    ...
    vs2019_dependency_overrides = {'xmscore': 'xmscore/[>=6.0.1 <7.0.0]'}
    xms_dependencies = [
        "xmscore/7.0.0",
    ]

Two rules are enforced, and only on msvc 192. An entry's key must match a package already declared in xms_dependencies, and it may change the version or version range only — the package name on both sides of the entry must be identical. Either violation raises a ConanException naming the offending entry, rather than being skipped: configure() and run_python_tests() iterate the unresolved xms_dependencies, so a rename would set options on and pip-install from a package no longer in the graph, and a key that matches nothing (a typo in the build.toml table) would produce a VS2019 build that quietly resolves the very versions the override was written to replace. Silent wrong output is worse than a loud failure, and on this build in particular the failure is the only signal you get — nothing downstream would look wrong until the desktop products linked against it.

Omit the table and the attribute is not emitted at all, so the recipe inherits the empty default. Do not hand-edit the value into conanfile.py or xms_conan2_file.pyxmsconan gen rewrites the first and re-copies the second on every run (§6), so only the build.toml key survives regeneration.

One more msvc-192-only behavior, which needs no configuration: the recipe propagates its own wchar_t option into boost's (self.options["boost"].wchar_t). The legacy boost/1.74.0.3 exposes that option and the Conan 1 recipe did the same; boost/1.86.0 has none, which is why the assignment is VS2019-only and tolerant of a boost build that doesn't declare it.

7.5 What a package exposes to a C++ consumer

cpp_info.libs names one library. By default it is always the static library:

pybind_advertises_module Platform cpp_info.libs Where it is installed
false (default) any <name>lib (<name>lib_d on Debug) lib/
true Windows, pybind = True _<name> (_<name>_d on Debug) — the module's import library bin/ holds the .pyd, lib/ its import library
true Windows, pybind = False <name>lib — there is no module to advertise lib/
true Linux / macOS <name>lib lib/

pybind_advertises_module is opt-in per library, and Windows-only. Three reasons it is not a default:

  • A consumer that links the module gets only the symbols the module exportsPyInit__<name> plus anything explicitly __declspec(dllexport). The static library gives it everything. Redirecting a consumer to the module is a deliberate trade, made by the one library whose consumers want it.
  • pybind propagates down the dependency graph (configure() sets it on every xms_dependencies entry), so keying off the option alone would redirect every sister library in a pybind graph.
  • Off Windows there is nothing to advertise. A macOS MODULE target is a bundle that cannot be linked, and a Linux module is _<name>.cpython-313-x86_64-linux-gnu.so, which the find_library(NAMES _<name>) that CMakeDeps emits does not match — so advertising it turns a working package into a configure error.

Where it does apply, the point is msvc-version skew: the consumer links the .pyd dynamically, which is what lets an msvc 192 application consume an msvc 194 build. The MSVC compatibility guarantee runs newer-consumes-older, so a static link across that boundary is unsupported.

Both libraries are in the package either way. An opted-in library installs the module twice on purpose: once under _package/xms/<python_namespaced_dir>/, the tree the wheel is built from, and once into bin/ + lib/, because Conan's generators look only there. The import library is installed by file name, not through install(TARGETS ... ARCHIVE): CMake tracks no ARCHIVE artifact for a MODULE target, so an ARCHIVE clause installs nothing and reports nothing — verified against CMake 3.28 and MSVC v143, with and without ENABLE_EXPORTS.

The _d suffix on the static library comes from set(CMAKE_DEBUG_POSTFIX _d). It does not reach the pybind module target: pybind11_add_module calls pybind11_extension, which overwrites the target's DEBUG_POSTFIX with PYTHON_MODULE_DEBUG_POSTFIX — the NAME_WE of the interpreter's EXT_SUFFIX, which is the empty string for a release interpreter (NAME_WE of .cp313-win_amd64.pyd) — and that overrides the directory-scope value. Verified against the pinned pybind11/3.0.1; 2.9.1 set no DEBUG_POSTFIX at all.

So for an opted-in library the generated CMakeLists re-asserts DEBUG_POSTFIX "_d" on the module target under if (WIN32), giving bin/_<name>_d.<abi>.pyd and lib/_<name>_d.lib. Three properties of that guard are deliberate:

  • WIN32, with no test on the postfix value. set_target_properties assigns rather than appends, so re-asserting _d over a postfix that is already _d — a genuine debug Python build — is idempotent. Testing the value first would skip the re-assert for any other non-empty PYTHON_MODULE_DEBUG_POSTFIX, which a cross-compiling build is required to set and anyone can pass on the command line; the module would then carry a postfix cpp_info does not advertise, and nothing would fail until the consumer's link.
  • Windows only. The import library exists only there, and off Windows the module must keep the name the shipped Python imports it by, because the shipped Python is what imports it; a library that names Debug in [matrix].pybind_build_types still builds and tests that module off Windows.
  • Gated on pybind_advertises_module, exactly like cpp_info. The opt-in is what makes _<name>_d a name anything links. A library that has not opted in keeps pybind11's name, so its Windows Debug _package tree stays importable.

A Windows Debug pybind configuration produces no wheel. For an opted-in library the module there is _<name>_d.<abi>.pyd, and the shipped Python imports it as xms.<dir>._<name>, so a wheel built from that tree installs a module import cannot find and the Python tests fail on it. The generated _package/pyproject.toml does not catch this — its ext-modules entry is sources = [] with optional = true, so setuptools neither compiles nor validates the name, and its package-data glob is *.pyd — the failure is the import itself. Renaming the module is not an option, since consumers link _<name>_d by that exact name.

build() therefore skips the wheel and the Python tests for Windows Debug, and the skip is logged. It is deliberately broader than the rename: it also covers a Windows Debug pybind build that has not opted in, and a vtk_wrap library, which has no _<name> module target at all. Neither is renamed, but every wheel step in the generated CI is gated on Release (§10.2), so a Debug wheel would be built only to be discarded.

Off Windows a Debug module keeps its importable name, and build() builds its wheel and runs its Python tests as usual. Coverage no longer depends on that path — its Python half is a pybind=True, testing=False, Release build (§5.7, §11) — but a library that asks for a Debug module still expects its wheel and its Python tests, so the build runs them.

Naming note for anything migrating off Conan 1. The static library is <name>lib, not Conan 1's lib<name>. Anything reading cpp_info is unaffected; a consumer hard-coding the old file name is not.

7.6 Python tests and dependency shared libraries

run_python_tests() builds a venv, installs the unrepaired wheel into it, and runs pytest. That puts the module outside the build tree, where CMake's build RPATH no longer applies — the installed module keeps only $ORIGIN, and the wheel does not carry its dependencies yet. A dependency built as a shared library (laslib/*:shared=True, say) is then unreachable and the test fails at import with libFoo.so: cannot open shared object file or DLL load failed while importing _<name>.

The recipe puts those libraries back on the loader's path, by a different route per platform:

  • Linux and macOS. generate() declares a VirtualRunEnv, and pytest runs with env="conanrun", so LD_LIBRARY_PATH / DYLD_LIBRARY_PATH name every dependency's library directories. Conan degrades gracefully here: if nothing needs a run environment, no conanrun script is written and the command runs unwrapped. The same run environment also carries the PYTHONPATH entry each pybind dependency exports from package_info (§7.5), so those _package directories are on sys.path during the tests as well as pip-installed into the venv. Same files from the same package folder, so the duplication is harmless.
  • Windows. PATH does not work. Since Python 3.8 an extension module's dependent DLLs resolve only from the module's own directory, the system directories, and directories passed to os.add_dll_directory — so the conanrun PATH cannot carry them. The recipe writes a sitecustomize.py into the venv that registers each dependency's bindirs and libdirs at interpreter startup, before pytest imports anything. This is the same mechanism delvewheel injects into a repaired wheel.

This affects the test only. The shipped wheel is repaired separately (§12), which vendors the dependencies next to the module where $ORIGIN and the Windows module-directory search already find them.

A library whose dependencies are all static needs none of this and is unaffected.


8. Python version support (3.10, 3.13, 3.14)

xmsconan defaults to Python 3.13 only everywhere. Each platform opts in to more versions through its own list, because what limits the fan-out differs per platform:

[ci]
python_versions       = ["3.10", "3.13", "3.14"]   # Windows
mac_python_versions   = ["3.13", "3.14"]           # macOS
linux_python_versions = ["3.13", "3.14"]           # Linux + Linux-ARM

Both mac_python_versions and linux_python_versions default to the highest entry of python_versions, which is the behavior those platforms had when the version was hardcoded — so a project that sets none of them generates the same CI it did before.

Why three lists instead of one:

  • Windows interpreters all come from actions/setup-python (GitHub) or uv venv --python on the GLR-UV runner (GitLab), so a version costs only runner minutes. 3.10 lives here because the desktop products (GMS/SMS/WMS) consume a 3.10 Windows wheel; nothing else needs it.
  • macOS is cheap to fan out too, but inherits nothing from the Windows list — adding 3.10 for the desktop products should not triple the mac matrix.
  • Linux runs in a container, and each version needs a published conan-gcc13-py<version> image. Only 3.13 and 3.14 images exist; there is no conan-gcc13-py3.10/3.11/3.12. Naming a version with no image yields a job that cannot start, which is exactly why this list is not derived from python_versions.

What opting in turns on:

  • CI matrices expand. GitHub Actions: python-version on the mac, linux, linux-arm and windows jobs from their respective lists. GitLab: parallel:matrix over PYTHON_TARGET_VERSION on Conan Build (from linux_python_versions) and on Conan Build - Windows (from python_versions) — plus Conan Build - Windows VS2019 when [ci].windows_vs2019 is set, over the same list. The three Conan Deploy jobs do not fan out: they restore and upload, which is ABI-independent, and dependencies: hands one instance every tarball the fan-out produced (§10.2). On Windows the variable is what uv venv --python is given, so it selects the interpreter directly rather than naming a per-ABI runner image.
  • Linux container images follow the matrix legghcr.io/aquaveo/conan-gcc13-py${{ matrix.python-version }}:latest on GitHub and …/conan-gcc13-py${PYTHON_TARGET_VERSION} on GitLab. An explicit [ci].docker_image still overrides both outright.
  • Artifact names grow an ABI suffix, but only where a platform actually fans out. A platform left on one version keeps the exact MATRIX_NAME, wheel-artifact and release-asset names it published before, since release assets are fetched by exact name.
  • Conan binaries. Each pybind variant carries the python_version option in its package_id, so consumers select xmscore/X.Y.Z@… pybind=True python_version=3.14. Non-pybind builds drop python_version from package_id, so testing/plain-library binaries remain a single shared binary regardless.
  • Wheel output. Each version produces its own cp3XY wheel; pip on the consumer side picks the right one.

Wheel repair is unaffected: xmsconan_wheel_repair only hosts auditwheel, which keys off each wheel's own tags, so the manylinux cp313-cp313 interpreter repairs a cp314 wheel fine.

For local builds (python build.py), the matrix is single-version: it uses PYTHON_TARGET_VERSION from the environment if set, otherwise 3.13. To build several wheels locally, invoke python build.py once per version (or construct XmsConanPackager directly with python_versions=["3.13", "3.14"]).

xmsconan coverage is the exception: it sets PYTHON_TARGET_VERSION itself from the resolved coverage ABI (§5.7) before invoking build.py, so a local coverage run needs no manual export — and an ambient value does not win over build.toml.

Either way, a local pybind build only works from an interpreter of the version being built: the recipe points CMake at sys.executable and the generated CMakeLists.txt requires that exact version. CI never notices because it arranges one: actions/setup-python on GitHub, uv venv --python ${PYTHON_TARGET_VERSION} on the GitLab Windows jobs (§10.2). On a workstation you supply it, which is why the VS2019 wheel workflow is one run per Python version from a virtual environment of that version (§16.8).

Runner / image expectations. On GitLab a Windows opt-in assumes the GLR-UV image can supply the interpreter — it carries uv and pre-loaded 3.10 / 3.13 / 3.14, and uv venv --python downloads anything else it is asked for, so a version outside that set costs a download on every job rather than failing. What it does not have to supply is cmake: the build jobs install it from PyPI into the job's venv, so the image only owns uv and the compilers. A Linux opt-in assumes the matching conan-gcc13-py<version> image exists and is readable by CI — on GHCR the 3.13 image is public while 3.14 is private, and the generated container: block pulls anonymously, so a private image needs its visibility flipped (or a credentials: stanza added) before its leg can start.


9. Local development workflow

9.1 Generate, then build everything

xmsconan gen      --version 0.0.0 build.toml
python build.py   --version 0.0.0 --wheel-dir wheelhouse --artifacts-dir test_artifacts

build.py flags worth knowing:

Flag Effect
--filter '{"build_type": "Release"}' Restrict to a subset of the matrix. Keys and values match the configuration dict (build_type, arch, compiler, options.pybind, options.python_version, …) and are validated the same way as the [filter] table (§5.8). A configuration that does not carry the filtered options/buildenv key does not match it — '{"options": {"python_version": "3.13"}}' selects only pybind configurations, since python_version is set on those alone. Applied on top of [filter], not instead of it.
--ignore-build-filter Drop the [filter] table baked in from build.toml, for a one-off build of a configuration it excludes.
--python-only Equivalent to --filter '{"options": {"pybind": true}}'.
--preview Print the configuration table and exit. Nothing is built.
--build-missing Pass --build=missing to conan create.
--platform KEY Which matrix to build: windows, windows_vs2019, linux, darwin. Defaults to detecting the running machine, which is what every leg other than the msvc 192 ones passes. --platform windows_vs2019 also drops the boost option defaults (the legacy boost/1.74.0.3 recipe does not declare them, and Conan fails a build when a profile sets an option no recipe in the graph defines) and, with --upload, sends the result to aquaveo-vs2019 under a compiler.version=192 package query. All three follow from the one flag, so the matrix and its destination cannot be set differently.
--wheel-dir DIR After the build, copy each pybind package's wheel into DIR. With python_versions=["3.10","3.13"], you get one wheel per version. A run that asked for wheels and got no complete set exits 1 — the flag is a request, not a hint, so only pass it where a wheel is expected. A matrix with no pybind configuration at all (--filter '{"build_type": "Debug"}' with the default [matrix].pybind_build_types, §5.4.1) is that case.
--repair Run repair_linux_wheel after extraction (Docker required).
--artifacts-dir DIR Save per-config test artifacts (LastTest.log, runner binary, _package/, test_files/) for debugging.
--test-shards N|auto Run the gtest suite as N shards after the build instead of during it. The recipe skips cmake.test(), the runner binary is invoked once per shard with GTEST_TOTAL_SHARDS/GTEST_SHARD_INDEX set, and a shard that fails fails the build. auto is half the CPU count (minimum 2) — half rather than all, because each shard is a full test process with its own memory and I/O. Requires --artifacts-dir, which is where the runner is sharded from. --test-shards alone is refused, because the recipe would skip cmake.test() for every testing configuration and nothing would then run the suite. The generated CI passes neither flag: xmsconan job build reads [ci].test_shards from build.toml and supplies the artifacts directory itself (§5.6, §10.4).
--skip-dependency-libs Do not stage the Conan cache's shared libraries into <wheel-dir>/libs. They exist only so the repair tools can resolve imports, so this is for a build whose wheel is not repaired. No generated job passes it: xmsconan job build reads [ci].windows_wheel_repair itself and skips the staging when it is off (§12.1).
--version VERSION The version --upload publishes and --wheel-dir stages. Defaults to $XMS_VERSION when that is set, and otherwise to what the CI environment says (§10): the tag on a tag pipeline, 0.0.0 on any other CI job, and setuptools-scm's reading of the checkout's tags outside CI. No generated job passes the flag or sets the variable any more; both remain as the override for building one release under a chosen version.
--skip-build --upload After a successful build, push the matrix to the Conan remote. Refuses to run without a release version -- 0.0.0, which is what an untagged pipeline and a checkout with no tags both resolve to, or anything containing *, the glob the flag once defaulted to: it matched every version of the library in the local cache, and whatever it matched was on the shared remote afterwards. Pass --version, or run from a tag pipeline.

xmsconan gen and xmsconan profiles also remove any .txt profile in conan_profiles/ that the current matrix does not produce, so narrowing [matrix] does not leave stale profiles behind for an IDE to pick. CMakePresets.json is a single rewritten file and was always clean; the two now agree. Non-.txt files in that directory are left alone.

9.2 Configure a single profile (for an IDE)

build.py runs conan create for every config. To get a buildable IDE configuration for one profile (no conan create, just install + configure):

xmsconan build \
    --cmake_dir . \
    --build_dir ../builds/xmscore \
    --profile VS2022_TESTING \
    --generator vs2022

Available profile names live under xmsconan/build_tools/profiles/{debug,release}/, and --profile matches the exact filename — a name that isn't a file there fails outright rather than falling back. Examples that exist today:

  • GCC13, GCC13_TESTING, GCC13_PYBIND, GCC13_TESTING_D
  • CLANG17_PYBIND, CLANG16_TESTING_D
  • VS2019, VS2019_TESTING, VS2019_TESTING_DYNAMIC, VS2019_TESTING_DYNAMIC_D
  • VS2022_TESTING, VS2022_TESTING_D

The full list is §20. You can also pass any explicit profile path with --profile /path/to/profile.

9.3 Useful build flags

  • --allow-missing-test-files — Build even when ./test_files/ doesn't exist.
  • -DXMS_GTEST_DISCOVER_TESTS=ON — Register every gtest case as its own ctest test. Off by default: gtest_discover_tests makes ctest spawn one process per TEST_F, and for a suite of a few hundred cases the process starts cost more than the tests. The default registers the runner as a single ctest entry that runs the whole suite in one process; parallelism comes from --test-shards instead. Turn it on when you want ctest -R to select an individual case by name.
  • --dry-run — Print the Conan and CMake commands without running them.
  • -v / -q — Verbose / quiet output; -v also turns a one-line failure into a traceback (§4.2).

10. Generating CI

xmsconan ci build.toml

Emits .github/workflows/<Lib>-CI.yaml (when ci_type = "github") or .gitlab-ci.yml (when ci_type = "gitlab"). Commit the result — CI runs against the committed file.

Unlike the build files, the CI file is committed, so a stale one is a diff someone could notice. xmsconan ci --check (§4.1) is what notices it: it exits 1 with a diff when the committed pipeline no longer matches what build.toml and the current xmsconan would render — which is the state a repository ends up in after a [ci] edit, or after an xmsconan upgrade nobody regenerated against.

The generated jobs follow the pattern:

  1. Install the toolchain (pip install --upgrade "xmsconan[ci]>=…" -- conan, cmake, gcovr and the flake8 plugins arrive with it, §10.3)
  2. Run the job -- one xmsconan job <kind> call (§10.4). On GitLab that is the whole of the script: job build sets up Conan, generates the build files, builds this job's leg and stages its wheel; job test, job package, job deploy, job lint and job coverage --pages are the rest. Each GitHub platform job is the same list, one step per command, with the credentials on the steps that reach a remote — see §10.1. The wheel upload is the one step that runs two lines: it echoes which of the variable, the secret, or the built-in default supplied AQUAPI_URL before it uploads, because that is the only unmasked evidence of where the wheels are going. job test is the one it has no use for: the runner that built the package is the one that tests it, so the suite runs inside job build.
  3. On tag pushes: xmsconan job deploy on both -- the wheel upload, and the Conan upload of every tarball the build jobs left in .export/ on GitLab, or of what this runner's own cache already holds on GitHub (--from-cache, §10.4)

The deploy jobs are deliberately not collapsed the same way: they publish, and what a publish targets is worth reading in the pipeline file rather than inferring from a subcommand name.

No step names a version. Every xmsconan command, and the build.py it generates, resolves one the same way when given none:

  1. --version on the command line (XMS_VERSION in the environment, for build.py, §9.3).
  2. The tag: CI_COMMIT_TAG on GitLab, or GITHUB_REF_NAME on GitHub Actions when GITHUB_REF_TYPE is tag -- a branch run has a ref name too, and it is not a version.
  3. 0.0.0 on any other CI job (GITLAB_CI or GITHUB_ACTIONS set). Every untagged pipeline has always built 0.0.0, and setuptools-scm cannot reproduce that from a runner's checkout: a shallow clone makes it invent 0.1.devN, and a full one would stamp a dev version onto packages the pipeline then downloads, restores and names under the fallback.
  4. setuptools-scm, from the checkout's git tags -- a developer's machine.
  5. 0.0.0.

So the tag pipeline and the branch pipeline are one file, differing only in what the host exports: no job passes the version from one step to the next, the tarballs between stages take theirs from a {version} placeholder (§14), and the deploy steps upload whatever the tag says. xmsconan ci still accepts --version, but the generated file no longer carries a library version anywhere. The GitLab Lint job is the one place a version is still pinned rather than resolved, and it is xmsconan job lint that pins it: linting is not publishing and nothing there reaches a package name, so a resolved version would make a tag pipeline's lint job differ from the branch pipeline's that just passed. It renders .flake8 and builds nothing.

10.1 GitHub specifics

  • Mac matrix: build_type × python-version=ci_mac_python_versions.
  • Linux / Linux-ARM matrices: build_type × python-version=ci_linux_python_versions.
  • Job name: carries the version on any platform that fans out (GCC-13 (Release, 3.14, Linux)). GitHub uses an explicit name: verbatim and only auto-appends matrix values when none is given, so without this two legs would share one status-check name — ambiguous in the checks list and in branch-protection matching. A single-version platform keeps its original name, so existing required checks keep matching.
  • Windows matrix: build_type × compiler-version × python-version=ci_python_versions.
  • The build_type axis defaults to [Release, Debug] and shrinks to whatever the [filter] table leaves buildable (§5.8) — a build_type pin, or any filter that empties a whole build type. The wheel repair / upload / deploy steps drop per platform when that platform builds no wheel. An os/arch/compiler pin is not applied — those are separate job blocks, so xmsconan ci warns about the jobs it would empty instead.
  • Wheel artifacts carry -py${{ matrix.python-version }} on any platform that fans out; a single-version platform keeps its bare wheel-${{ runner.os }} name.
  • Linux containers resolve to conan-gcc13-py${{ matrix.python-version }}:latest.
  • No step asks for a wheel; the build reads whether it made one. The build line used to append --wheel-dir wheelhouse behind matrix.build_type == 'Release', because build.py exits 1 when --wheel-dir yields no complete set of wheels (§9.1) and [matrix].pybind_build_types defaults to Release only (§5.4.1), so a Debug leg had nothing to extract. That gate was a proxy for a question xmsconan job build now asks the configurations directly — did this leg build a pybind configuration whose wheel the recipe produced — and the proxy was wrong for the one library the question is interesting for: a library naming Debug in [matrix].pybind_build_types gets no wheel from its Windows Debug build by design (§7.5) and a perfectly good one everywhere else, which a build-type gate cannot say. Two consequences: that library's Windows Debug leg no longer fails at "no complete set of wheels", and on Linux and macOS its Debug leg now stages a wheel that is built and discarded — seconds, and what GitLab's flagless build has always done. The steps that consume the wheel keep their matrix.build_type == 'Release' guards, so nothing publishes a Debug wheel.
  • flake deliberately stays on a single hardcoded interpreter — linting is ABI-independent, and pinning it keeps lint results identical across repos.
  • A tag needs a published GitHub Release. On a tag, the last step of every platform job, Upload Zipped Conan Packages, attaches <MATRIX_NAME>.tar.gz to the tag's GitHub Release. A tag with no published Release — a bare git push of the tag, or a Release still in draft — fails that step on every leg, after the Conan packages and wheels are already up. Create the Release against the existing tag (gh release create <tag> --verify-tag) and re-run the failed jobs. A re-run repeats each job whole — the build, the Conan upload and the wheel upload, then the attach. The Conan upload accepts the repeat, and so does the index the wheel goes to by default: Aquaveo's dev index is a volatile devpi stage, which takes the re-built wheel in place of the first. An index that refuses to replace a file it already holds — a non-volatile stage, or PyPI — fails the re-run at Upload wheel to Aquapi instead. An asset of the same name already on the Release fails the step too, rather than being replaced.
  • Third-party actions are referenced by commit SHA, with the tag in a trailing comment (uses: microsoft/setup-msbuild@30375c6… # v3). A tag is a movable ref in a repository Aquaveo does not control, and the action runs in a job whose steps hold the Conan login and the deploy secrets, and can write GITHUB_ENV for every step after it. actions/* stays on tags — a compromise of GitHub's own namespace is a compromise of the runner regardless. To move a pin, resolve the new tag with gh api repos/<owner>/<repo>/commits/<tag> --jq .sha and edit the template. test_github_ci_pins_third_party_actions_to_a_sha fails on any third-party action added by tag.
  • Secrets are set on the steps that use them, never on the job. No job-level env: value references secrets.. Job-level env is inherited by every step — the pinned actions, the artifact uploads, the wheel publish — and log masking only hides a secret that is printed. Note what this does and does not buy for the Conan pair specifically: Build the Conan Packages is the step that runs conan create, which builds a venv, installs the test dependencies from PyPI and runs the library's own suite, so that suite sees the credentials either way — resolving the dependencies is what they are for. What scoping keeps them away from is every step with no Conan business. CONAN2_USER_SECRET / CONAN2_PASSWORD_SECRET reach Build the Conan Packages and Upload Releases to Conan — the two steps that reach the remote, one resolving this library's dependencies and one publishing its packages. There is no conan remote login step ahead of them any more: both commands run the Conan setup themselves and neither logs in, because Conan reads the pair from the environment when it needs it, and conan remote login with no credentials to hand it prompts — on a runner, a job that hangs rather than one that says what is missing. The coverage workflow is the same rule one step later: xmsconan conan-setup writes the remote into the runner's Conan home and reaches nothing, so the pair sits on Run Coverage. AQUAPI_USERNAME_SECRET / AQUAPI_PASSWORD_SECRET reach Upload wheel to Aquapi alone, and AQUAVEO_GITHUB_TOKEN reaches Upload Zipped Conan Packages alone — the tag-gated actions/github-script step that finds the tag's GitHub Release and attaches the packaged Conan cache to it. It holds the token as step env, like every other credential here, rather than handing it to the action's github-token input, and the script builds its client from that variable (getOctokit(process.env.RELEASE_TOKEN)). Seven tests pin the scoping, all of them rendered with [ci].linux_arm both off and on so the opt-in fourth build job is covered: test_github_workflows_keep_secrets_off_the_job_environment (nothing on the workflow, job, container or services env — GitHub hands container.env to every step exactly as it hands job.env), test_github_ci_gives_the_conan_login_to_every_build_step, test_github_coverage_gives_the_login_to_the_run_and_not_to_the_setup, test_github_ci_gives_the_conan_login_to_every_package_upload, test_github_ci_gives_the_index_credentials_to_the_wheel_upload, test_github_ci_attaches_the_archive_with_one_first_party_step (the token, and the one variable the script reads it from), and test_github_workflows_hand_secrets_to_exactly_the_documented_steps, which asserts equality with the documented holder set per job — an allow-list alone is satisfied by a workflow that lost every credential, and a workflow-wide set is satisfied by three build jobs out of four, which is exactly the copy-drift the four near-identical jobs make possible. The first and last of the seven also render Coverage.yaml, and the third is the coverage workflow's own; the rest are XmsCore-CI.yaml only, because the coverage workflow publishes neither a package nor a wheel. The generated file's header repeats the rule.
  • The wheel index URL resolves variable, then secret, then a built-in default. The expression is ${{ vars.AQUAPI_URL_DEV || secrets.AQUAPI_URL_DEV || 'https://public.aquapi.aquaveo.com/aquaveo/dev/' }} — the repository or organisation variable AQUAPI_URL_DEV, then the secret of that name, then the public dev index. Both names are read on purpose: the secret is where this value lived before the workflow was ever generated, so a maintainer who opens Settings → Secrets and variables → Actions → Secrets and edits it there still moves the upload target, which is what anyone who knows this repository already expects. || yields the first truthy operand and the empty string is falsy, so a non-empty variable wins and an empty one falls through to the secret. That is the one way to get this wrong — once a non-empty variable exists, editing the secret changes nothing and nothing says so — which is why the deploy step logs the phrase AQUAPI_URL supplied by the AQUAPI_URL_DEV variable / ... secret / ... built-in default before it uploads. Read that line before editing either place. Masking follows the secret's value, not the name it was read under: while the secret holds this URL the job log shows *** for the target even when the variable supplied it, so the source label — which matches no secret and is never masked — is the only thing in the log that distinguishes the three. Keep the value in one place; once the variable is authoritative, delete the secret, and the upload target reads in clear.
  • The flake job installs xmsconan[ci] and runs xmsconan job lint, which renders .flake8 at the pinned fallback version and then runs plain flake8 _package — the same one command GitLab's Lint job runs. It deliberately does not pass flake8 settings on the command line: that duplicates .flake8.jinja, and the two copies drift apart silently (CI once used a different ignore list and a stale conf.py exclude, so a clean local run did not imply a clean CI run). Change lint settings in .flake8.jinja only. The plugins come with the [ci] extra (§10.3) rather than from a list on the install line, because such a list is where flake8-tidy-imports went missing: .flake8.jinja sets banned-modules, an option only that plugin registers, and flake8 accepts config options no installed plugin claims and enforces nothing, so the osgeo.* ban reported the same green as a run that had checked it, while GitLab (which installed the plugin) linted to a stricter rule.

10.2 GitLab specifics

The bullets below describe the default pipeline shape, whose Linux work is one Conan Build job looping every configuration and a Coverage Build job compiling the two instrumented ones. [matrix].wheel_only = true (§5.4.1) replaces both with one build job per configuration, running concurrently, with the instrumented ones measuring their own coverage in place — see §11.5. Everything here about filters, the Windows jobs, wheels, and [ci].split_tests applies to either shape.

  • Build jobs name at most their own leg, so a [filter] table in build.toml (§5.8) applies with no template change — GitLab has no build_type matrix axis to narrow, and xmsconan job build applies [filter] and the leg selector in that order itself. A filter that leaves a platform with no pybind configuration drops that platform's wheel work: the Repair Wheel and Wheel Deploy jobs and the staged Linux wheel, and Wheel Deploy - Windows plus the in-place repair step for Windows. The two are independent — "compiler.runtime" = "static" takes out only the Windows side. The Conan Deploy jobs publish packages rather than wheels and are unaffected.
  • Conan Build: parallel:matrix over PYTHON_TARGET_VERSION from linux_python_versions, each leg saving its own -py<version> export tarball. With a single version it stays a plain job with no parallel:matrix, exactly as before. Conan Deploy - Linux never fans out — it globs .export/ and uploads once, whatever the build produced.
  • Repair Wheel and Wheel Deploy stay single jobs: they collect every leg's wheels and act on all of them at once.
  • Coverage Build pins both its container image and PYTHON_TARGET_VERSION from the resolved coverage ABI ([coverage].python_version, §5.7), so the interpreter inside the image and the ABI the pybind build targets are the same value. Under an explicit [ci].docker_image only the pin is resolved — the image is whatever the repo named, so keeping that image's interpreter in step is the repo's job. xmsconan coverage exports the variable itself, so the job is correct without it; it is declared here as well so the pairing is visible in one place, and so a regression in the tool cannot pass unnoticed on GitLab specifically. Unlike Conan Build, it never fans out — coverage commits to a single ABI by design. The Coverage job that follows it needs no pin at all: the report phase reads JSON, so it runs in a plain python:<coverage ABI> image and builds nothing.
  • [ci].split_tests cannot be combined with a multi-version linux_python_versions — the C++ test jobs take the build job's artifacts by name, so a multi-ABI build would leave them testing an indeterminate one. xmsconan ci rejects that combination at generation time.
  • [ci].split_tests is also rejected when the [filter] table leaves no Linux testing configuration (options.testing = false, or a pybind = true pin). Splitting makes the build job set XMS_SKIP_CXX_TESTS=1, so with no runner staged there is nowhere left for the suite to run — the pipeline would pass having tested nothing. Note that [ci].build_types is not the axis here: a pybind = true pin keeps Release configurations while staging no runner at all.
  • One test job per testing configuration, each naming its own. [ci].split_tests emits a Run C++ Tests - <label> job for every testing configuration the Linux build stages — Release-testing and Debug-testing on the default matrix — and each passes its own --label. They are peers in the Test stage off the single Conan Build, and each uploads only its own test_artifacts/<label>/ (the needs: artifacts: true download hands every job the whole staged tree, so an unscoped path would have each one re-upload the others' runners too). Every job writes TEST-cxxtest.xml; GitLab keys the pipeline test report on the job name, so the configurations stay distinguishable in the MR widget. The labels are computed by the generator with the same config_label() the build names the directories with, so the two cannot drift — a template concatenating <build_type>-testing would be a second implementation free to. This is what a single job got wrong: it found its directory by falling back to the first of Debug-testing, Release-testing that existed, so a matrix building both compiled the Release runner on every pipeline and never executed it, with nothing red to show for it. xmsconan test-shards keeps that fallback for interactive use after a local build.py, but no generated pipeline relies on it any more — xmsconan job test passes the label the job was generated with.
  • Each job shards inside one container. The job downloads the build job's test_artifacts/, then runs xmsconan job test, which reads the shard count, the artifacts directory and [ci].xvfb from build.toml and hands them to the shard runner. That restores the executable bit on the runner (a GitLab artifact round-trip drops it), relinks test_files/ next to the runner, and forks [ci].test_shards runner processes with GTEST_TOTAL_SHARDS/GTEST_SHARD_INDEX set. Each shard writes its own gtest XML, which the tool merges by suite name into TEST-cxxtest.xml and declares as artifacts:reports:junit, so a failing case shows up in the MR widget by name. Per-shard console output is held and replayed as one contiguous block rather than interleaved. A shard that dies before writing XML — a timeout, a segfault — contributes a synthetic error suite to the merged report, so a red job never shows an all-green test tab. Without test_shards the same script runs a single shard: one code path, so the artifact discovery and test_files relinking do not have to exist twice. With [ci].xvfb = true each shard gets its own X display, because xvfb-run -a picks a free server number and N simultaneous starts race for it.
  • Conan Build - Windows runs a parallel:matrix over PYTHON_TARGET_VERSION from python_versions; Conan Deploy - Windows does not, and pins the newest of them in variables: instead. Restoring and uploading is ABI-independent, and needs: artifacts: true hands every instance the whole fan-out's tarballs, so a matrix there was N jobs restoring the same set and racing to upload one Conan reference; the pinned version is only what uv venv --python builds the venv on. Both run on image: GLR-UV with tags: [WinVM]image: selects the VM template, the tag routes to the Windows fleet. One image serves every ABI: the job establishes the interpreter itself with uv venv --python ${PYTHON_TARGET_VERSION} .venv, activates it, and installs xmsconan[ci] into it with uv pip install. That venv is load-bearing, not hygiene — the recipe points CMake at the interpreter running conan and the generated CMakeLists.txt requires that exact version, which is what the retired per-ABI GLR-py310 / GLR-py313 images existed to provide. uv --version runs first so a runner without uv fails on a line that says so, and nothing reads python before the venv is active. cmake (>=3.21) comes into that venv with the extra, and xmsconan job build prints its version before it builds anything: the retired per-ABI images carried cmake on PATH and GLR-UV does not, and conan shells out to cmake for every configuration, dependencies built from source included. The deploy jobs get it too and never invoke it -- they restore a cache tarball and upload it -- which is the price of one extra carrying the whole toolchain (§10.3). The job command also sets UV_PYTHON from PYTHON_TARGET_VERSION before it builds: uv build given no interpreter picks the newest one it can find, which on a multi-ABI image is the wrong one for every leg but the highest (§11.3).
  • Conan Build - Windows VS2019, Conan Deploy - Windows VS2019 (only with [ci].windows_vs2019, §5.6): the second Windows toolchain, msvc 192. Same runner, same ABI fan-out and same tag-gated export/restore split as the msvc 194 pair; four things differ, and all four are load-bearing.
    • xmsconan job build --platform windows_vs2019 selects the msvc 192 matrix, and is the only thing the job spells: the flag carries the other five differences with it. The recipe forks its third-party stack on compiler.version (boost 1.74.0.3 + zlib 1.2.11 rather than 1.86.0 + 1.3.1), so the boost option defaults are dropped for this platform because the legacy recipe does not declare them, and missing dependencies are built from source because the legacy graph is not fully prebuilt.
    • The aquaveo-vs2019 remote is added, appended rather than inserted first: on a shared runner it must not become the first stop for every conan install. It is where the legacy dependencies resolve from as well as where the results go. The CI remote is still configured alongside it — the recipe's own dependencies resolve from there even on the legacy toolchain.
    • Every step that publishes is restricted to compiler.version=192. The Conan cache on a runner is per machine, not per job, so both conan cache save and conan upload — which match by reference — would otherwise carry a concurrent msvc 194 job's binaries onto the remote whose only purpose is to keep the two toolchains apart, and exit 0 having done it. The mitigation is symmetric: the msvc 194 jobs restrict their own save and upload to compiler.version=194, unconditionally, whether or not [ci].windows_vs2019 is set. The mirror leak is the worse of the two — it puts msvc 192 binaries on the production aquaveo remote — and a guard present only in the configuration that needs it is one refactor from being dropped from the one that does. Neither version is rendered into the CI file: job build derives its save's query from the configurations it actually built (and declines to claim one when they disagree), and job deploy reads its upload's query from the packager matrix row --platform selects. A literal here would not fail loudly when it fell behind a toolchain bump — conan upload -p compiler.version=194 after a move to 195 matches nothing and the job goes green having published no binaries at all.
    • No wheels. A wheel's tags (cp310-cp310-win_amd64) say nothing about which MSVC built it, so an msvc 192 wheel and an msvc 194 wheel are the same devpi filename and publishing both would have them overwrite each other by upload order. The VS2019 wheels the desktop products consume are still produced deliberately, by hand (§16.8).
  • Wheel-repair always runs cp313-cp313's xmsconan job package inside the manylinux container; auditwheel itself doesn't care about the host Python.
  • Required CI variables for every pipeline: CONAN_LOGIN_USERNAME and CONAN_PASSWORD. No generated job sets them — Conan reads them from the environment itself, the same way conan remote login does on a workstation (§16.2) — so they come from the project's or group's CI/CD variables, Masked and Protected. Unlike the deploy variables below they are not tag-only: every build resolves dependencies from the aquaveo remote, so a pipeline missing them fails at conan install, not at release time. The generated file's header lists them ungated for that reason.
  • Required CI variables: AQUAPI_URL, AQUAPI_USERNAME, AQUAPI_PASSWORD (for wheel deploy). Set the username and password Masked and Protected in the project's CI/CD variables: masked keeps the value out of the job log when a command echoes it, protected keeps it off branch pipelines, where anyone who can push a branch can print it. Neither survives CI_DEBUG_TRACE=true, which prints every variable the job holds, masked or not -- enable it only on a pipeline with no deploy jobs, and rotate anything it printed. They are read from the process environment by the wheel half of xmsconan job deploy, which hands them to uv publish through its environment (§13), and are deliberately not placed in any profile's [buildenv]: conan echoes the profile it is handed to stdout under "Input profiles" before every conan create, so a credential put there is printed to the job log in cleartext. Nothing in the recipe or the generated CMake reads them, so a build is unaffected. This is why they are absent from the [filter.buildenv] names in §5.8 -- they are not names the profiles set. The absence is enforced, not merely maintained: the single function that serializes a profile refuses to write a [buildenv] name outside the allow-list, so a credential that reached a configuration by accident fails the build at that point rather than being quietly dropped from one profile and printed by another. Adding a genuinely new build variable therefore means adding its name to PUBLIC_BUILDENV_KEYS -- which is the moment to ask whether it is safe to print in a job log.
  • Concurrent conan remote add on one runner. Each Windows job adds its remotes at the start, and several jobs share a machine, so they write one ~/.conan2/remotes.json. Conan writes it as remotes.json.tmp plus an os.replace, so the file cannot be torn — but there is no lock around the read-modify-write, so two jobs adding remotes in the same instant can lose one of the two additions. The failure is loud (the job that lost its remote fails resolving dependencies; it does not publish to the wrong place) and the fix — a per-job CONAN_HOME — would give up the shared package cache these builds depend on, so it is left alone deliberately. Retry the job.
  • Required CI variables with [ci].windows_vs2019: CONAN_LOGIN_USERNAME_AQUAVEO_VS2019 and CONAN_PASSWORD_AQUAVEO_VS2019, for the upload to aquaveo-vs2019. Conan derives those names by uppercasing the remote name and replacing hyphens with underscores, and reads them from the environment — the generated job adds the remote but never logs in explicitly, exactly as it does for aquaveo. Without them the build jobs still pass and only the tag-time deploy fails, so add the variables before the first release rather than after. Set both Masked and Protected, as above.
  • [ci].linux = false yields a Windows-only pipeline: Conan Build - Windows, Lint, and (on tags) Wheel Deploy - Windows and Conan Deploy - Windows. Two combinations are rejected at generation time rather than producing a pipeline that fails opaquely later — linux = false together with windows = false (nothing would build), and linux = false with coverage = true (coverage compiles with --coverage under gcc, and the generated CMakeLists.txt rejects MSVC when XMS_COVERAGE is set).
  • Each platform stages and publishes its own wheel. Linux builds one, repairs it in the Package-stage Repair Wheel job, and uploads it from Wheel Deploy. Windows passes --wheel-dir wheelhouse and then repairs in place inside Conan Build - Windows, because delvewheel reads the DLL imports of a win_amd64 .pyd and only runs on a Windows host — the manylinux container that repairs the Linux wheel cannot stand in. Wheel Deploy - Windows uploads the result on tags from a plain python:3.13 image, since a devpi upload needs no MSVC toolchain. A Windows-only pipeline therefore publishes wheels, and the manual VS2019 track (§16.8) is no longer the only route. [ci].windows_wheel_repair = false turns that in-place repair off for a library that has nothing to vendor — the step renders either way and xmsconan job build decides inside it; see §12.1. The deploy job is unaffected either way.
  • The Windows cache snapshot is gone. Conan Deploy - Windows used to cp -r ${HOME}/.conan2/p/* into a conan_packages/ artifact for debugging, after the upload had already succeeded. It was an unqueried copy of a shared runner's cache, so on a machine also running the msvc 192 matrix it snapshotted the other toolchain's binaries beside this job's — unusable for the one question it existed to answer — and nothing consumed it. xmsconan job deploy --cache-archive PATH writes the queried equivalent (conan cache save, restricted to this platform's compiler.version) for a job that wants one. The GitHub platform jobs are what pass it: each one attaches its archive to the release as <MATRIX_NAME>.tar.gz, and it is queried, so a shared runner's other toolchain does not ride along.
  • The Windows wheel jobs fan out with the build matrix. Conan Build - Windows runs once per PYTHON_TARGET_VERSION, and each instance stages its own wheel into wheelhouse/. Each wheel's own ABI tag (cp310 / cp313 / cp314) keeps the files distinct, so the merged artifact holds every wheel and one Wheel Deploy - Windows job uploads the set.

10.3 Toolchain versions

Generated jobs install one distribution, xmsconan[ci], and everything else they run arrives as its dependencies. The ci extra in xmsconan's pyproject.toml carries conan, cmake, gcovr, flake8 and the plugins the generated .flake8 relies on, so each version lives in one place rather than on every install line of every template:

Tool Spec Why
xmsconan[ci] >=<version that generated the file>,<<next major>, always with --upgrade An xmsconan fix reaches a repo on its next CI run, rather than requiring a regenerate-and-commit pass across the whole suite. The floor rules out resolving a version older than the templates were generated against; the cap keeps the next major -- the one kind of release allowed to change what a generated file expects of the tool -- from reaching a workflow that was not regenerated against it.
conan ~=2.31.0 (patch series) Conan computes package_ids and runs the compatibility plugin; a minor bump can silently detach a build from binaries already on the remote. Every job takes the same pin, the coverage workflow included -- it used to install conan unpinned, so the run that reports coverage could resolve different package ids than the run that builds.
cmake >=3.21 What the generated CMakeLists.txt requires. The Linux images and the GitHub runners carry a system cmake, and the one from the extra now sits ahead of it on PATH; the Conan Build jobs print cmake --version after the install so the log shows which one configured, while the instrumented coverage jobs, which build through xmsconan coverage, do not. The GitLab Windows image carries none (§10.2).
gcovr >=7,<9 Its JSON and Cobertura output is what xmsconan coverage parses (§11).
flake8 and its plugins unpinned flake8-bugbear, flake8-docstrings, flake8-import-order, flake8-tidy-imports, pep8-naming -- the set the generated .flake8 is written against. flake8 accepts an option no installed plugin claims and enforces nothing, so test_ci_extra_carries_the_plugin_behind_each_generated_flake8_option pairs each plugin option .flake8.jinja sets with the plugin that registers it. flake8-aquaveo is deliberately not among them: the GitLab Lint job installs it from the internal index as before, the GitHub flake job has never run the AQU rules, and whether both hosts should is a decision for the lint command rather than for the extra.

Bumping any of these is an xmsconan release, which every generated job then takes on its next run. pip install "xmsconan[ci]" gives a workstation the same toolchain (§2).

--upgrade is not optional on the xmsconan install. pip treats an already-satisfied constraint as a no-op, so on a runner image with xmsconan baked in, the floor alone installs nothing and the job silently runs whatever version the image happens to carry. test_xmsconan_installs_float_and_upgrade asserts both halves — the >= and the --upgrade — across every CI option combination for both templates.

Two consequences worth knowing:

  • An xmsconan release reaches every repo on its next CI run. That is the point of the floor, but it cuts both ways: a bad release is suite-wide immediately, and the committed CI file no longer records which xmsconan actually ran. The sharpest edge is that xmsconan owns the Conan profiles — a release that changes a profile's compiler.version changes every package_id, for every repo, without any repo changing a line. Treat profile changes with the same care as a breaking API change.
  • Generate CI from a released xmsconan. A workflow generated from an unreleased working copy writes a floor that nothing on devpi satisfies yet (e.g. >=2.16.1.dev1,<3 while the newest release is 2.16.0), and CI fails at pip install. Unlike the old == pin, this one self-heals: the same file starts working once that version ships. A checkout that is not installed at all reports itself as 0.0.0, and the >=0.0.0,<1 it writes is satisfied by nothing on the index: that job fails at pip install, where the bare floor used to float to the newest release.

10.4 xmsconan job — the command a generated job runs

Every generated job's script is now an install line and xmsconan job calls -- the whole script on GitLab, one per step on GitHub. The six subcommands are the six things a pipeline does:

Command Job it runs What it derives
xmsconan job build Conan Build, the per-leg Linux jobs, both Windows jobs; every GitHub platform job's Build the Conan Packages Conan setup, xmsconan gen at the resolved version, the configuration matrix and its filters, the wheel, the export tarball
xmsconan job test Run C++ Tests - <label> Shard count, artifacts directory, [ci].xvfb, the JUnit report name
xmsconan job package Repair Wheel; GitHub's Repair wheel, on the platforms that have one The wheel directory to repair
xmsconan job deploy Wheel Deploy, Wheel Deploy - Windows, all three Conan Deploy jobs; GitHub's Upload wheel to Aquapi and Upload Releases to Conan Which tarballs .export/ holds, the remote and package query for this platform, the wheel directory, the devpi credentials
xmsconan job lint Lint; GitHub's Run Flake xmsconan gen at the pinned fallback version, then flake8 _package
xmsconan job coverage --pages pages Which coverage-html-* reports exist, the public/ tree, the landing page's links and title

job build takes the flags that name this job and nothing else:

Flag Default What it does
--leg {library,testing,pybind} every configuration this platform's matrix produces Which configuration kind this job builds. A Linux fan-out job names one; the Windows jobs, which loop a whole matrix, name none.
--platform windows_vs2019 detect from the running machine Build the msvc 192 matrix. Carries the legacy remote, --build-missing, the dropped boost defaults, the suppressed wheel and the -vs2019- tarball segment with it (§10.2).
--export off Save a Conan cache tarball under .export/ for the deploy job to restore. Passed only by the jobs a tag pipeline publishes from — a branch pipeline's tarball is never restored, and uploading one costs artifact storage on every push.
--release-skips-testing off On a release version, drop the testing configurations. This is the GitLab Windows job's old tag rule, and, under [matrix].wheel_only, every GitHub platform job's: nothing installs a test runner and no tag pipeline runs one. On GitHub it also replaced a JSON --filter the template selected through a step-level env: expression, and on the Windows leg through a second if:-gated step -- shell: cmd expands %VAR% before the argument is parsed, so that leg could not hold the filter in a variable. The tool tests the same tag the expressions did, from the version it already resolved (§10). A flag rather than an unconditional rule, because the jobs that loop a whole matrix publish from that same build and their tarball would lose binaries a release ships.
--defer-cxx-tests off A separate job in this pipeline runs the C++ suite this build compiles, so set XMS_SKIP_CXX_TESTS rather than also running it inline. Inert unless [ci].split_tests is on. A flag rather than something the tool infers from --leg: the Linux and Windows builds both run a flagless xmsconan job build, and only the generator knows that the Linux one has Run C++ Tests jobs downstream and the Windows one has none.
--build-missing off (implied by --platform windows_vs2019) Build missing dependencies from source.
--version / --toml resolved (§10), build.toml As everywhere else.

job deploy publishes both halves by default; each flag narrows it:

Flag Default What it does
--conan-only / --wheels-only publish both Publish only the Conan packages, or only the wheels. Mutually exclusive — asking for neither half is a deploy that publishes nothing, and the argument parser is a better place to learn that than a green job with an empty log. GitLab passes one or the other because its wheel and package publishes are separate jobs on separate runners; a GitHub job that does both passes neither.
--platform windows_vs2019 detect from the running machine Publish the msvc 192 matrix: append the aquaveo-vs2019 remote and upload there instead. The compiler.version query comes with it (§10.2).
--from-cache restore .export/*.tar.gz first Publish what this runner's Conan cache already holds, restoring nothing. GitLab's build and deploy are different jobs on different runners, so the tarball is how the binaries cross; GitHub's are the same job on one runner, where the cache is still warm and a restore would be a round-trip through a file the job itself just wrote. Stated by the caller rather than inferred from an empty .export/, which is also what a build that failed to save leaves behind. Read only while the Conan half publishes, so --wheels-only refuses it rather than accepting a flag it would never consult.
--cache-archive PATH off After uploading, write conan cache save to PATH, restricted to the same query as the upload. For a job that wants the published binaries as a file — every GitHub platform job attaches one to the release — or a hand-run investigation. Written from the same query as the Conan upload, so like --from-cache it is refused beside --wheels-only.
--version / --toml resolved (§10), build.toml Resolved as in §10, and then checked: a deploy refuses a version no release names. 0.0.0 — what an untagged run resolves to — and a glob are both rejected before either half publishes, because an upload is the one step nothing takes back (§10.5).

The tarballs are globbed, not named. job deploy restores every .export/*.tar.gz its dependencies: handed it and uploads once at the end. The template used to spell each tarball's name for the restore while job build composed the same name for the save — two derivations of one string on opposite sides of a layer, and a rename on either side restored a file that was not there, uploaded nothing, and exited 0. It also means the deploy jobs no longer fan out over PYTHON_TARGET_VERSION: the per-instance --restore name was the only thing keeping a matrix's instances apart, and with a glob they would each restore the whole set and race to upload one Conan reference.

Everything else is read, not passed. The wheel directory, the artifacts directory, ctest's parallelism, UV_PYTHON, the skip-tests variable, the shard count, [ci].xvfb, [ci].windows_wheel_repair, and the export tarball's name were all decisions the template made and then had to describe to a command that could not check them. They are facts about build.toml and the job's environment, and reading them where they are used is what keeps the two from drifting — a repository that turns sharding on no longer has to regenerate its CI before the shards run.

Three environment variables the templates used to set themselves -- export lines on GitLab, job-level env: on GitHub -- are set by job build instead, each only when the environment has not already chosen a value — so a bigger runner can still raise a level without regenerating the CI:

Variable Set to When
CTEST_PARALLEL_LEVEL 8 always
UV_PYTHON $PYTHON_TARGET_VERSION only when that variable is set — off CI nothing names an ABI, and pinning uv to whichever interpreter happens to be first on PATH is not the same decision. Every generated job sets it -- GitLab's testing and library legs included, and each GitHub platform job from its matrix.python-version -- so in a pipeline this is always on.
XMS_SKIP_CXX_TESTS 1 under [ci].split_tests, on the jobs the generator marks --defer-cxx-tests — the ones whose compiled runner a separate Run C++ Tests job consumes

The two output directories are the exception that proves it: test_artifacts/ and wheelhouse/ are fixed by the tool, and the pipeline's only remaining say in them is which ones it collects as artifacts. A rename on either side uploads nothing, which is why the test asserting the pairing reads both from the same constants.

Output is folded into sections — section_start/section_end markers on GitLab, ::group:: on GitHub, plain ==> … headers elsewhere — so a failing build's compiler errors are still the last thing in the log rather than being pushed off the end by setup noise.

10.5 Replaying a CI job locally

A red job is reproducible without reading the pipeline file, because the job's script is now the command you would type:

# What "Python Build - py3.13" ran
PYTHON_TARGET_VERSION=3.13 xmsconan job build --leg pybind

# What "Run C++ Tests - Debug-testing" ran, after a local build
xmsconan job test --label Debug-testing

# The same test job, narrowed to one case
xmsconan job test --label Debug-testing -- --gtest_filter=MyTest.*

# What "Lint" ran
xmsconan job lint

# What "Conan Deploy - Windows VS2019" ran, given a .export/ to publish
xmsconan job deploy --conan-only --platform windows_vs2019

# What "pages" ran, given the coverage-html-* trees Coverage Build produced
xmsconan job coverage --pages

Three things to know when you do:

  • The environment is part of the job. PYTHON_TARGET_VERSION selects the ABI the pybind fan-out builds and names the export tarball; BUILD_TYPE narrows the matrix the same way the fan-out jobs' variables: block does. Both come from the job's variables: in the generated file — copy them across, or you will build a different matrix than the runner did.
  • A version is resolved, not assumed. Off CI, xmsconan job build takes the version from setuptools-scm; on CI it takes the tag, or 0.0.0 when there is none (§10). Pass --version to pin it.
  • --export is safe to omit and usually should be. It writes a Conan cache tarball under .export/ that only a deploy job reads.
  • job deploy publishes for real. It is the one job kind with an outward-facing effect: it uploads to the Conan remote and to devpi, from whatever is in .export/ and wheelhouse/ at the time. There is no dry-run flag. Replay it only when you mean to publish, and remember that --version is what names the reference — a resolved 0.0.0 is refused, but a wrong pin is not.
  • --defer-cxx-tests is worth omitting too. Under [ci].split_tests GitLab's Linux build passes it so the compiled runner is exercised by the Run C++ Tests jobs instead; replaying the build without it runs the suite inline, which is usually what you want from one local command. Pass it when you are reproducing what the runner did rather than testing the code.

A job that builds nothing now says so and fails: when the [filter] table and the leg selector cancel each other out, xmsconan job build prints which filters it applied and exits non-zero, rather than looping an empty list and exiting 0. That case used to read as a passing build that produced no packages.


11. Coverage (xmsconan coverage)

xmsconan coverage (also available as the legacy xmsconan_coverage script) runs two instrumented Conan builds and produces both C++ and Python coverage reports. Configured via the [coverage] table in build.toml (§5.7). The run splits into a collect phase (the builds and every rendered report) and a report phase (the thresholds and the gate); --phase selects one and the default runs both, see §11.4.

xmsconan coverage build.toml   # --version is optional; without it the version resolves as in §10

What it does — steps 1-4 are the collect phase, step 5 the report phase (§11.4):

  1. Generates conanfile.py, build.py, and CMakeLists.txt from build.toml (via xmsconan gen).

  2. Sets XMS_COVERAGE=1 and invokes build.py twice — sequentially by default, see [coverage].parallel (§5.7):

    • First: --filter '{"build_type":"Debug","options":{"testing":true,"pybind":false}}' — Conan create that builds the library + CxxTest runner under --coverage, then runs the runner. Produces the .gcda set gcovr reads.
    • Second: --filter '{"build_type":"Release","options":{"pybind":true,"testing":false,"python_version":"3.13"}}' — Conan create that builds the library + pybind wheel, then runs pytest-cov against the wheel. Instrumented like the first, so the binding layer -- C++ that only a Python test can reach -- produces its own .gcda. Release is not a problem for line data: the XMS_COVERAGE block appends -O0 -g after CMake's -O3, so the build is unoptimized either way, while still resolving against Release dependencies. Produces cov-py.xml, cov-py-summary.json, and coverage-html-py/ inside the build folder.

    The two coverage builds pin the inverse of each other, and they pin different build types — Debug for the CxxTest half, Release for the pybind half — so a [filter] table (§5.8) conflicts by requiring testing/pybind as much as by excluding it, and any build_type pin cancels one of the two outright. Either way one of the builds is left with no configurations and build.py exits non-zero. A python_version pin other than [coverage].python_version cancels the Python build the same way. xmsconan ci warns about each conflict at generation time.

    XMS_COVERAGE=1 in the environment tells the packager to set the recipe's coverage=True option on every configuration — the option, not a [buildenv] entry, is what reaches the build, so an instrumented build carries its own package_id and can never satisfy --build=missing for a production build (or vice versa). The recipe forwards the option to CMake as -DXMS_COVERAGE, which adds --coverage. Coverage no longer changes which configurations exist: the testing-only Debug variant is part of the standard packager fan-out either way, and the Python half reuses whatever pybind configuration [matrix].pybind_build_types already names. Pybind builds get the option too: the binding layer is C++ that only a Python test can reach, and gcovr merges that build's .gcda into the C++ report. The same option keys pytest-cov in the recipe, which is why Python coverage is collected from the same build. Requiring a Debug pybind build used to cost every dependency a Debug+pybind binary — the one combination the xms libraries do not publish — and would need a debug interpreter on Windows. python_version on the pybind build is pinned to one ABI (highest of [ci].linux_python_versions by default, or [coverage].python_version when set) so multi-version fan-outs cannot non-deterministically pick whichever pybind config finished last.

  3. Locates the two build folders in the local Conan cache and runs gcovr against both. Each folder is read separately with its own --root -- one invocation cannot span two roots, and --root has to match the absolute paths embedded in that folder's .gcno files -- writing a JSON tracefile each. A final gcovr run combines them with --add-tracefile into cov-cpp.xml, cov-cpp-summary.json, and coverage-html-cpp/. Tracefile paths are relative to --root, so the two folders' copies of the same source line up and hit counts sum: a line the CxxTest runner never reaches still counts as covered when a Python test reaches it through the bindings. The pytest-cov artifacts are copied out of the pybind build folder as before. The two legs write into separate Conan build folders, but they share the local Conan cache -- which is not safe for concurrent writes -- and contend for CPU when overlapped (an identical shard measured 1.57× slower beside a second leg), which is why running them together is opt-in (§5.7). Their console output is captured per leg and replayed whole once that leg finishes rather than interleaved line by line, which would otherwise split every compiler diagnostic away from the file name printed above it. A leg that fails — or cannot start at all — is recorded and reported after both have finished, so the other leg's reports are still produced.

  4. Writes cov-cpp.xml, cov-py.xml, cov-cpp-summary.json, cov-py-summary.json, coverage-html-cpp/, and coverage-html-py/ into --output_dir.

  5. Re-reads the thresholds from build.toml, compares the line-coverage percent for each layer against [coverage].cpp_threshold / [coverage].python_threshold, and exits non-zero on regression — with a distinct code for the gate, see §11.6. The tool also exits non-zero if build.py reported a test failure in either layer — but only after still producing gcovr reports, copying artifacts, and (if $GITHUB_STEP_SUMMARY is set) appending the markdown summary, so the coverage data and the failing-test signal are both visible in the same run. If a coverage summary file is present but missing its expected keys (gcovr/pytest-cov schema drift, truncated write), the tool raises rather than reporting a misleading 0%. A cov-cpp-summary.json or coverage-status.json that is absent is a tool failure rather than a gate miss (exit 1): the collect phase always writes both, so their absence means the run that should have produced them did not. A cov-py-summary.json that is absent fails the Python layer outright: the pybind build in step 2 always runs with coverage=True, so there is no configuration in which pytest-cov legitimately writes nothing, and python_threshold defaults to 0 — treating the absent file as 0% would report PASS for a layer that was never measured.

11.1 What this requires of the recipe

  • python_namespaced_dir must be set on the recipe (it is by default — xmsconan gen derives it from library_name). With coverage=True the recipe targets pytest-cov at exactly xms.<python_namespaced_dir> so coverage doesn't leak across xms_dependencies installed in the build venv. The recipe's configure() raises at install time if coverage=True is set without python_namespaced_dir, so the run fails fast before a full instrumented build is wasted.
  • The compiler must support --coverage (GCC/Clang). MSVC is rejected by the generated CMakeLists.txt when XMS_COVERAGE is non-empty.
  • gcovr must be on PATH (the xmsconan[ci] extra carries it, §10.3).

11.2 [ci].xvfb = true

If the library needs an X server to run its tests (VTK, GUI libs), xmsconan coverage re-execs itself under xvfb-run -a -s "-screen 0 1280x1024x24" automatically. A missing xvfb-run is logged as a warning, not a fatal error, so the test failures surface clearly.

11.3 Environment variables consumed

Variable Set by Effect
XMS_COVERAGE xmsconan coverage Packager-side switch. The packager translates it into the recipe's coverage=True option on every configuration; the option then enables --coverage in CMakeLists.txt (via -DXMS_COVERAGE), installs pytest-cov, and passes --cov=xms.<python_namespaced_dir> to pytest. The recipe itself no longer reads this variable, and neither does the generated CMakeLists.txt — for raw cmake --preset builds (where the recipe's build() never runs), setting it at generation time instead bakes XMS_COVERAGE=1 into every generated configure preset's cacheVariables, so regenerating with it set is how a local consumer coverage script instruments its build.
XMS_COVERAGE_PIP_INDEX you Optional extra --extra-index-url for the build venv's pip install when coverage is enabled (useful when pytest-cov lives on a private index).
UV_PYTHON xmsconan job build, from PYTHON_TARGET_VERSION (§10.4) Read by uv itself, not by xmsconan. It pins the interpreter every uv build in the job uses, so the wheel carries the ABI the build was compiled against. Without it uv build discovers an interpreter and takes the newest available — on a runner serving several ABIs that is the wrong one for every leg but the highest, and the resulting wheel will not install into the build's own venv. Set it yourself for a local pybind build on a machine with more than one Python; a workstation with a single interpreter never notices. Because uv reads it, it also applies to dependencies conan builds from source, whose recipes came from whatever xmsconan version published them — which a flag on the current recipe cannot reach.
PIP_INDEX_URL / PIP_EXTRA_INDEX_URL you Read by pip itself, not by xmsconan. The wheel install at the end of a pybind build resolves xms_python_dependencies from whatever index pip is configured with; set these when any of those requirements live on a private index.
CI_COMMIT_TAG GitLab, on a tag pipeline The version every xmsconan command resolves when given none (§10). xmsconan coverage resolves it once, generates the build files with the result and passes it to the build.py it runs, so the three cannot disagree.
GITHUB_REF_NAME / GITHUB_REF_TYPE GitHub Actions, on every run The same on GitHub. The ref name counts only when the type is tag: a branch run has a ref name too. The packager reads the type as well: on a tag it sets RELEASE_PYTHON=True in every profile, as CI_COMMIT_TAG does on GitLab, where the generated workflow used to derive it with a third-party action.
GITLAB_CI / GITHUB_ACTIONS the host, on every job Marks a CI job. Untagged, it resolves 0.0.0 -- which --upload refuses (§9.3, §14) -- rather than asking setuptools-scm, whose answer from a runner's checkout is a dev version no other job in the pipeline would agree on.
XMS_VERSION you Read by build.py ahead of everything above (§9.3). The generated jobs no longer set it; it is the override for building one release under a chosen version without editing the pipeline.
GITHUB_STEP_SUMMARY GitHub Actions When set, a coverage summary table is appended.

11.4 Phases (--phase)

--phase splits the run at the boundary between the part that needs a build tree and the part that only needs the numbers. The default, all, does both in one process — which is what a local run wants.

Phase Does Needs
collect Steps 1-4: both instrumented Conan builds, gcovr, and every rendered artifact. Also writes coverage-status.json. A toolchain, the Conan cache, and the build folders. Minutes.
measure Steps 1-4 for one leg, named by --leg. Builds that leg instrumented, runs its tests, and writes a gcovr tracefile plus that leg's slice of the status file. Renders no report — merging needs both legs, and no single job has both. A toolchain, the Conan cache, and one build folder. Minutes.
report Step 5: merges any per-leg tracefiles, re-reads the thresholds from build.toml, reads the summaries, logs the table and the $GITHUB_STEP_SUMMARY rows, and applies the gate. cov-cpp-summary.json or cov-cpp-tracefile-*.json, plus cov-py-summary.json and coverage-status.json. Seconds, in a bare Python image with gcovr.
all collect and then report. A collect that fails returns its own exit code without reaching the gate. Both.

--leg selects which instrumented build measure runs: cpp is the Debug testing configuration, python the Release pybind one. It is required by measure and ignored by every other phase; measure without it is a usage error rather than a default, because there is no leg that is the obvious one to mean. An unknown leg raises.

measure is what makes the two instrumented builds concurrent jobs rather than two sequential compiles in one. Each writes its artifacts under names carrying its leg — cov-cpp-tracefile-<leg>.json, coverage-status-<leg>.json — so two jobs uploading into one artifact space cannot overwrite each other, and report unions the status files rather than trusting whichever landed last. A leg whose build leaves no package in the Conan cache records itself unmeasured and exits 1; it does not write an empty tracefile, which the merge would read as a layer that is genuinely 0% covered.

The split lands where it does because _run_gcovr passes --root <build folder> and --html-details: the rendering has to happen beside the build folder holding the compiler's copy of the sources. Rendering inside collect means only JSON and finished HTML ever cross a job boundary — no .gcda to relocate, no GCOV_PREFIX/GCOV_PREFIX_STRIP.

coverage-status.json carries exactly what the gate needs and no report file records: whether either build leg reported a test failure, and whether each layer was measured at all. Thresholds are deliberately not in it — report re-reads build.toml, so raising a threshold re-gates an existing set of artifacts without rebuilding anything.

[ci].xvfb applies to collect only: the re-exec under xvfb-run happens where the tests run, so the report phase needs no display and no xvfb-run on PATH.

Beyond the CI scheduling this buys testability. The gate used to be reachable only by driving two real Conan builds, which made the logic deciding pass-vs-fail the least-tested part of the coverage tool; the report phase needs no toolchain, no Conan cache, and no build folder. After a collect run it is genuinely JSON in, exit code out. After measure legs it also shells out to gcovr once, to merge the per-leg tracefiles — still no compiler, but gcovr has to be on PATH in the report job too.

11.5 GitLab vs GitHub

  • GitLab, [matrix].wheel_only = true: there is no Coverage Build. The two instrumented builds are ordinary Build-stage jobs in the per-configuration fan-out wheel_only produces (§5.4.1), each running xmsconan coverage --phase measure --leg <cpp|python>, and the Coverage stage holds a single Coverage job running --phase report — the merge and the gate. That removes the duplicate compile the old shape paid for: Coverage Build recompiled the Debug testing and pybind configurations because instrumentation is part of the package_id, so no production binary could satisfy it, and 503 of the 1334 objects a pipeline compiled were that second copy. The Coverage job needs: the instrumented build jobs with artifacts: true, which is how the tracefiles reach it; the build folders never move. allow_failure: exit_codes: [3] under [ci].split_tests applies here the same way it does below.
  • GitLab, otherwise: xmsconan ci emits a Coverage stage holding two jobs, both invoking xmsconan coverage. Coverage Build runs --phase collect — the two instrumented builds and gcovr — and declares needs: [], so it starts at t=0 instead of queueing behind Conan Build. Coverage runs --phase report off its artifacts and exposes cov-cpp.xml as the cobertura coverage report. needs: [] rather than physically moving the collect job into the Build stage: a Build-stage job would make every Test-stage job wait behind the instrumented compile, which is the opposite of the intent. The coverage: regex matches the Coverage total: <pct>% line the report phase prints rather than gcovr's TOTAL row, because gcovr now runs in the other job and GitLab scrapes only the job it is declared on. The pages stage takes the rendered HTML from Coverage Build — the job that produced it — and runs xmsconan job coverage --pages, which writes a small landing page at the Pages root linking the cpp/ and python/ reports. Only the reports that exist are copied and linked, and the two decisions are made together. The template rendered this as inline echo HTML on two branches, both of which copied coverage-html-cpp and linked it unconditionally while guarding the Python half with a [ -d ], and that split two ways: with no C++ directory the cp failed and took the job with it, so a run whose Python layer measured fine published nothing; with the directory there but empty the cp succeeded and the first link 404'd, green. "Exists" here means the directory holds an index.html — the file the link resolves to — because a directory gcovr created and wrote nothing into is that same 404 in its quiet form. A run that produced no report at all fails the job rather than publishing an index that links nothing, which reads as a measurement of zero. The report job deliberately does not re-upload that tree: needs: artifacts: true already hands it the whole thing, so listing it again would store a second copy of every gcovr detail page. Depending on the collect job also means the report publishes on a gate miss, which is the run you most want to read. Under [ci].split_tests the Coverage job carries allow_failure: exit_codes: [3] — the tests gate the pipeline from their own job, so a coverage threshold miss is advisory here, but the tool failing is not. Coverage Build carries no allow_failure at all: the collect phase cannot produce a 3, so anything it fails on is a real breakage. This was allow_failure: true, which forgave crashes too; see §11.6.
  • GitHub: xmsconan ci emits a separate Coverage.yaml workflow that runs on push and pull_request, uploads coverage-html-*/ and cov-*.xml as artifacts, and appends a summary table to the run page. The workflow runs directly on ubuntu-latest (no docker container), which is what lets the --upgrade "xmsconan[ci]>=…" install of §10.3 actually take effect here — the image this job used to run in baked xmsconan in, so the install no-op'd and the canary was locked to whatever the image carried. Setting [ci].xvfb = true apt-installs xvfb as an extra step; [ci].docker_image is honored by the build/deploy workflows but not by Coverage.

11.6 Exit codes

xmsconan coverage distinguishes the coverage gate failing from the tool failing, because the generated CI treats them differently. The codes are the shared vocabulary of §4.2; this is the one command that produces 3.

Code Meaning
0 Both layers cleared their thresholds and every build leg completed.
1 The tool failed: an unhandled exception, or neither build leg left a package in the local Conan cache, so there was no .gcda to report on.
2 argparse usage error (a bad flag). Not produced by the coverage run itself.
3 The coverage gate failed: a layer below its threshold, an unmeasured layer (either one — a single build leg that left no package cannot fully measure its layer), or build.py reporting a test failure. Reports and artifacts were still produced.

Only 3 is forgiven by the generated GitLab job's allow_failure (§11.5), and only when [ci].split_tests puts the tests in a job of their own. The gate is advisory there because the tests already gate the pipeline elsewhere — but a crash, neither build leg leaving a package, or gcovr falling over must still fail the job. Collapsing both onto 1 under a blanket allow_failure: true is what let a Coverage stage report green while producing no report at all.

A single failed leg is not an error: the surviving leg's .gcda still produces a partial report, and the gate then fails as a gate (3) because the missing layer cannot be measured. Only losing both legs is exit 1.

The codes are the same per phase (§11.4), with the split moving where each one can arise. collect can only return 0 or 1 — it renders reports and records the test-failure and measured flags rather than acting on them, so it never gates. report returns 3 for the gate and 1 when coverage-status.json or cov-cpp-summary.json is missing or malformed: the collect phase always writes both, so by then their absence is a broken pipeline, not a coverage regression, and reporting it as a gate miss would let allow_failure forgive it.


12. Wheel repair (xmsconan wheel-repair)

Conan-built wheels reference shared libraries from the build environment that won't exist on consumer machines. Wheel repair bundles them in.

# Auto-detect the platform from sys.platform
xmsconan wheel-repair --wheel-dir wheelhouse

# Or be explicit
xmsconan wheel-repair --wheel-dir wheelhouse --platform linux

What runs per platform:

Platform Tool How libs are found
Linux auditwheel repair (with patchelf) LD_LIBRARY_PATH={wheel_dir}/libs
macOS delocate-wheel DYLD_LIBRARY_PATH={wheel_dir}/libs
Windows delvewheel repair --namespace-pkg xms --add-path {wheel_dir}/libs

Each tool is installed into the interpreter running xmsconan_wheel_repair and then invoked by the absolute path it was installed to, resolved against that interpreter's script directory. Installing xmsconan as a uv tool leaves that directory off PATH, so a bare-name invocation would fail with FileNotFoundError even though the install had just succeeded.

build.py already populates wheelhouse/libs/ for you when --wheel-dir is set — collect_dependency_libs copies every .dll / .so / .dylib anywhere in the Conan cache, which is routinely several hundred files. After repair, the original wheelhouse/ is replaced with the repaired version (the libs/ directory is removed).

12.1 Opting out on Windows ([ci].windows_wheel_repair = false)

Windows repair is the one that can be turned off, because it can make a wheel worse.

delvewheel's default ignore lists excuse vcruntime140.dll and vcruntime140_1.dll for a CPython wheel (they ship with the interpreter), and python3xx.dll / api-ms-* by regex. msvcp140.dll is on none of them. Any .pyd built with MSVC imports it, so delvewheel treats it as a needed dependency and vendors it — mangled to msvcp140-<hash>.dll, with the _delvewheel_patch hook rewriting the module's import table to point at the wheel's private copy. It will always find one: --add-path {wheel_dir}/libs holds everything from the Conan cache, and System32 is on PATH besides.

For a library whose .pyd statically links everything and imports nothing third-party there is nothing legitimate to bundle, so the entire outcome of repair is a second C++ runtime inside the process. That matters where the host application supplies the runtime itself — GMS/SMS/WMS scrub PATH in dmGetScriptEnvironment and point it at their own shipped ms_redist_* DLLs precisely so one CRT is in play.

The default is derived from ci_type, not hardcoded, because ci_type is a proxy for who installs the wheel:

ci_type Default Why
github true Those wheels are published for installation into arbitrary Python environments, which have no XMS runtime on PATH. The DLLs a module needs must travel with it.
gitlab false Those wheels are internal, and the only thing that loads them supplies the C++ runtime itself. Repairing them vendors a private mangled copy of the runtime the host is deliberately controlling.

A flat default would be wrong in one direction or the other: true everywhere would start the GitLab repos repairing wheels they previously never even staged, and false everywhere would stop the GitHub repos bundling DLLs their users need. Set the key explicitly to override either way:

[ci]
windows_wheel_repair = false

Every reader — xmsconan job build, xmsconan publish, and the xmsconan vs2019 driver — resolves this through one function. The CI generator is no longer among them: the Windows repair happens inside the build job either way (only a Windows host can run delvewheel), so the key changes what that job does rather than which steps are rendered, and the generated workflow is identical at both settings, and the whole [ci] table is validated against a key allowlist with per-key types at generation time. A misspelled key (windows_repair_wheel) or a quoted boolean ("false") is rejected rather than falling back to the default, because for a switch that turns work off the default means the work keeps happening and the only symptom is the harm the switch exists to prevent.

That skips the in-place repair inside xmsconan job build on both forges, skips it in xmsconan publish when it runs on Windows and in the xmsconan vs2019 driver, and in each case leaves the hundreds of cache DLLs unstaged, since they exist only so the repair tools can resolve imports. No step disappears from either generated pipeline: the Windows job runs the same command at both settings and reads the key when it gets there. The wheel is still built, still uploaded as an artifact, and still deployed — unrepaired, which for such a library is what it already was in substance.

There is no equivalent switch for Linux or macOS. A Linux wheel has to be repaired to carry a manylinux platform tag and to bundle libstdc++.so.6; skipping it would publish something pip cannot install portably.

Confirm what a given wheel actually got by unzipping the repaired output: a vendored CRT shows up as xms/<subdir>/*.libs/msvcp140-<hash>.dll alongside a _delvewheel_patch entry in the package __init__.py.


13. Wheel deploy (xmsconan wheel-deploy)

Uploads wheelhouse/*.whl to a devpi index with uv publish.

xmsconan wheel-deploy --wheel-dir wheelhouse

Credential resolution order (first non-empty wins):

  1. CLI flags: --url, --username, --password-file
  2. Environment: AQUAPI_URL, AQUAPI_USERNAME, AQUAPI_PASSWORD
  3. ~/.xmsconan.toml [aquapi] section

AQUAPI_URL is the index itself (https://public.aquapi.aquaveo.com/aquaveo/dev/), which is also devpi's upload endpoint — not the +simple/ page pip reads from it. A generated GitHub workflow sets all three on the Upload wheel to Aquapi step and nowhere else (§10.1); a generated GitLab pipeline reads them from the project's CI/CD variables (§10.2).

The password never touches a command line, in either direction. There is no --password flag — --password-file reads it from a file, as conan-setup and vs2019 setup do (§17) — because a process's argv lands in your shell history, ps output, and, on a managed Windows box, the Event 4688 / Sysmon record that gets shipped to the SIEM in cleartext. And the upload runs as uv publish --publish-url <url> <wheels> with the username and password in the child's environment as UV_PUBLISH_USERNAME and UV_PUBLISH_PASSWORD, the same way conan remote login gets CONAN_PASSWORD_<REMOTE>. The uv that runs is the one the uv package — a dependency of xmsconan — installed next to it (uv.find_uv_bin()), not whatever uv is first on PATH, so it is present wherever xmsconan is installed, a uv tool install or pipx layout included; a generated job needs nothing extra.

--client devpi keeps the previous devpi use / devpi login --password / devpi upload sequence for one release. It is the last place xmsconan passes a password on a subprocess's command line, it prints a warning saying so, and it is removed in the release after this one.

An empty wheel directory is an error, not a no-op: uv publish given no files would fall back to dist/*, and a deploy job that uploads nothing must not go green.


14. Conan deploy (xmsconan conan-deploy)

Used to ship Conan binaries between CI stages or to upload them at the end. The three modes:

# Save the cached package(s) to a tarball
xmsconan conan-deploy xmscore 7.0.0 --save xmscore-linux-7.0.0.tar.gz

# Restore from a tarball (e.g. produced by an earlier CI stage)
xmsconan conan-deploy xmscore 7.0.0 --restore xmscore-linux-7.0.0.tar.gz

# Upload the cached package(s) to the aquaveo remote
xmsconan conan-deploy xmscore 7.0.0 --upload

# Or, end-to-end in one shot
xmsconan conan-deploy xmscore 7.0.0 --restore xmscore-linux-7.0.0.tar.gz --upload

At least one of --save / --restore / --upload is required.

The version is optional. Without it the tool resolves one the way every other command does (§10) -- the tag on a tag pipeline, 0.0.0 on any other CI job, setuptools-scm from a checkout -- and a {version} in a --save or --restore path is replaced with the result. That is how the generated jobs name their tarballs without the version being known when the pipeline was generated:

xmsconan conan-deploy xmscore --save xmscore-linux-{version}.tar.gz
xmsconan conan-deploy xmscore --restore xmscore-linux-{version}.tar.gz --upload

--upload refuses 0.0.0 for the same reason build.py --upload does (§9.3): it is what an untagged job resolves, and nothing that resolves it is a release.


15. Full release pipeline (xmsconan publish)

Wraps the entire flow — useful for CI and for one-off local releases inside a Docker container.

# --version is optional (resolved as in §10); reads creds from ~/.xmsconan.toml or env
xmsconan publish --version 7.0.0

# Build only, no upload
xmsconan publish --version 7.0.0 --no-deploy

# Skip the wheel half / Conan half independently
xmsconan publish --version 7.0.0 --no-wheel    # Conan-only release
xmsconan publish --version 7.0.0 --no-conan    # wheel-only release

# Restrict the matrix
xmsconan publish --version 7.0.0 --filter '{"build_type": "Release"}'

What it runs (with --no-deploy=false). Every step except build.py runs the named command's code in this process rather than launching that command; the tools those commands launch themselves, such as conan, still run as their own processes:

  1. xmsconan conan-setup --login
  2. xmsconan gen --version <ver> build.toml (anything the generator raises stops the run here, before anything is built: one line, generating build files failed: <reason>, with the traceback under -v, and exit 1)
  3. python build.py --version <ver> --wheel-dir <dir> (wrapped in xvfb-run if [ci].xvfb=true and there is no $DISPLAY on Linux)
  4. xmsconan wheel-repair --wheel-dir <dir> (skipped on Windows when [ci].windows_wheel_repair is false)
  5. xmsconan wheel-deploy --wheel-dir <dir> (skipped with --no-wheel)
  6. xmsconan conan-deploy <library> <version> --upload (skipped with --no-conan)

--version is optional and resolves as in §10, but publish refuses 0.0.0 and any glob before it builds anything: a release that cannot name itself has nothing to publish.


16. VS2019 / msvc 192 packages (xmsconan vs2019)

The msvc 192 (Visual Studio 2019) binaries that the Aquaveo desktop products (GMS/SMS/WMS) consume are published to a separate Conan remote, aquaveo-vs2019, so they never mix with the CI-published binaries. xmsconan vs2019 drives that build by hand, on a developer workstation that has VS2019 installed.

GitLab CI can now build this matrix too. The GitLab GLR-UV runner carries VS2019 alongside VS2022, so a repository can set [ci].windows_vs2019 = true (§5.6) and get Conan Build - Windows VS2019 on every pipeline, publishing to the same aquaveo-vs2019 remote — reproducibly, rather than whenever someone remembers to run this by hand. GitHub cannot: it retired the windows-2019 image and has no replacement.

This track stays, and is still the only route for two things: wheels, which CI deliberately does not publish for msvc 192 (a wheel's tags do not record which MSVC built it, so it would collide with the msvc 194 wheel on devpi — see §10.2 and §16.8), and libraries whose repository has not opted in. It is also what you want for a one-off local rebuild.

If you are looking for the CI matrix, see §10.

Available both as xmsconan vs2019 <subcommand> and as the console script xmsconan_vs2019 <subcommand>. Three subcommands: setup, build, upload.

The recipe side of the fork — legacy boost, and the optional per-library [vs2019_dependency_overrides] table for libraries whose sister dependencies are pinned differently on VS2019 — is described in §7.4. Nothing below configures it; this section is only about driving the build.

16.1 End to end

# 1. One time: add + log in to the aquaveo-vs2019 remote, then preflight the machine
xmsconan_vs2019 setup --password-file <path to your conan password file>

# 2. See what would be built — prints the matrix per library and exits
xmsconan_vs2019 build --root E:\code\xms\migration --preview

# 3. Build it. Hours, not minutes; per-configuration logs land in .\vs2019-logs\
xmsconan_vs2019 build --root E:\code\xms\migration --log-dir .\vs2019-logs --version 7.0.0

# 4. Read the summary table, then publish
xmsconan_vs2019 upload --library xmscore --version 7.0.0

# 5. Python wheels are a separate pass, one per Python version — see §16.8

build and upload are separate verbs on purpose: a build never uploads as a side effect. The run takes hours and its output lands on a remote other people build against, so a human looks at the summary table before anything is published. There is no --upload flag on build. Wheels follow the same rule: build --wheel-dir stages them, and xmsconan_wheel_repair / xmsconan_wheel_deploy are the commands that repair and publish (§16.8).

16.2 setup

Adds the remote, logs in, then runs the preflight checks.

Flag Default Effect
--password-file PATH File holding the remote password on its own. Trailing whitespace (the editor's newline) is stripped. A path that doesn't exist, a file that can't be read, and a file that holds nothing but whitespace are all errors (exit 2) — see the resolution table below.
--username NAME resolved (see below) Remote username.
--remote-url URL https://conan2.aquaveo.com/artifactory/api/conan/aquaveo-vs2019 Artifactory URL backing the remote.
--remote-name NAME aquaveo-vs2019 Conan remote name to add. Pair it with --remote-url when pointing at a different Artifactory repo.
-v / -q Verbose / quiet, after the verb (xmsconan vs2019 setup -v). -q silences xmsconan's own progress lines and keeps its errors and the preflight table; conan's own output is unaffected. -v adds debug detail (§4).

Credential resolution order (first non-empty wins), resolved together so ~/.xmsconan.toml is read at most once:

Username Password
1 --username --password-file — a path that doesn't exist, or a file that exists but holds no password, is an error, not a fall-through to the next source. A typo (or a secret that never landed in the file) must not silently log you in with a different password.
2 $CONAN_LOGIN_USERNAME $CONAN_PASSWORD
3 [conan] username in ~/.xmsconan.toml (§17) [conan] password in the same file
4 aquaveo — Conan prompts interactively

Both halves fall back independently, so a personal Artifactory account configured in ~/.xmsconan.toml is used as your username with your password. (The username used to be pinned to aquaveo before the config file was ever consulted, which meant a personal password was sent with the shared username and the login failed with no hint why.) If no source supplies a password, setup hands Conan nothing and Conan prompts for both username and password itself — the file is not consulted a second time to second-guess that, because it has already been consulted here.

The password is never a command-line argument. setup hands it to Conan in the child process's environment (CONAN_LOGIN_USERNAME_AQUAVEO_VS2019 / CONAN_PASSWORD_AQUAVEO_VS2019 — Conan derives those names by uppercasing the remote name and replacing hyphens with underscores) and runs a bare conan remote login aquaveo-vs2019. On a managed workstation, process-creation auditing (Windows Event 4688 with command-line capture, or Sysmon Event ID 1) copies the full argv of every process into the event log and ships it to the SIEM in cleartext, where it outlives and out-reads the NTFS ACLs on your password file.

Where the remote lands in the list. setup appends aquaveo-vs2019 after the remotes already configured — it does not insert it at index 0 the way xmsconan conan-setup does for the CI remote. Conan resolves a version range such as xmscore/[>=7.0.0 <8.0.0] across every remote in list order, so a VS2019 remote at the front would be the first stop for every conan install and conan create --build=missing on the machine, including ordinary msvc 194 work, and a version present only on the VS2019 remote would win. That is the exact mixing the separate remote exists to prevent. setup prints (appended) when it adds the remote; run conan remote list to see the resulting order, and conan remote update aquaveo-vs2019 --index N if you ever need to change it deliberately.

setup exits 0 only when every preflight check (§16.3) also passed, so a fresh machine gets one pass/fail answer on whether it can build the matrix. A failed preflight exits 2 — the same code build uses for the same condition; an unusable --password-file exits 2; conan not being on PATH exits 2; any other failing conan command propagates its own exit code.

16.3 Preflight

Run at the end of setup, and again at the start of every build — a failure exits 2 either way, aborting the build before anything is compiled. Both verbs take --remote-name, and the third check follows it.

Check Passes when Fix
Visual Studio 2019 vswhere.exe finds a 16.x install that carries the C++ toolset (-requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64) and has a Microsoft.VCToolsVersion.default.txt. Reports the install path and the default MSVC toolset version. Install VS2019 with the Desktop development with C++ workload. msvc 192 packages can only be built on a machine that has it.
conan client conan --version is on the pinned ~=2.31.0 series. pip install "conan~=2.31.0", or pip install "xmsconan[ci]", which carries the pin (§10.3)
conan remote aquaveo-vs2019 The remote appears in conan remote list and is enabled. conan remote list keeps printing disabled remotes as [… Enabled: False], so the name alone is not enough. xmsconan_vs2019 setup --password-file <path>, or conan remote enable aquaveo-vs2019 if it is merely disabled.
python interpreter (build only) The matrix that is actually about to be built contains no pybind configuration, or every pybind configuration targets the Python version this shell is running. Re-run from a virtual environment of the requested version, or pass --python-versions <the version you are running>. Full explanation in §16.8.

The interpreter check is the one check that depends on the request rather than the machine, so it runs after the matrix has been generated and --filter applied, and prints into the same block. That scoping is deliberate: all but the pybind configurations have pybind=False and don't care which interpreter is running, so building only those from whatever virtualenv you happen to be in keeps working. setup doesn't run it — there is no matrix at that point.

A conan version outside the pinned series is a failure, not a warning: a minor bump can change package_id computation and silently detach these hand-built packages from the binaries already on the remote. Same reasoning as the CI pin in §10.3.

A VS2019 carrying only, say, the .NET workload used to pass this check and then die on the first conan create, hours into the run — hence both the -requires filter and the toolset-file check.

16.4 build

Flag Default Effect
--root DIR . Directory holding the library checkouts (one subdirectory per library). A --root that doesn't exist is an error (exit 2), not eight silent skips.
--only LIB Build only this library; repeatable. Overrides the enabled flag, so a mid-migration library can be exercised.
--from LIB Resume the stack at this library, skipping the ones before it. For picking up after a mid-stack failure.
--preview off Print the configuration matrix per library and exit. Nothing is built; preflight is not run and --root is not checked, because the matrix is computed from the library list and doesn't read the checkouts.
--continue-on-error off Attempt the next library after one fails.
--no-generate off Skip generating the build files and use the conanfile.py already in the checkout.
--log-dir DIR Redirect each configuration's conan create output to <DIR>/<library>-<config label>.log and print a one-line pointer instead. A wall of interleaved compiler output is unreadable on a 14-configuration run. The label carries the whole configuration, runtime includedxmscore-Release-static-testing.log, xmscore-Debug-dynamic-wchar_typedef.log, xmscore-Release-dynamic-pybind-py3.13.log. Without the runtime the 14 msvc configurations would collapse onto 8 filenames and each static build would overwrite the dynamic build's log.
--python-versions X.Y [X.Y …] 3.10 3.13 Python versions the pybind variants fan out across. A pybind configuration only builds when this matches the Python running conan, so wheels are built one version per run — §16.8.
--wheel-dir DIR After the build, copy each pybind package's .whl into DIR and fill DIR/libs with the shared libraries the repair step needs. Repair and publish stay separate commands (§16.8). A run that asked for a wheel and got none exits 1.
--filter JSON Restrict the matrix, same shape as build.py --filter: '{"build_type": "Release"}'. Nested keys are spelled out — '{"options": {"pybind": true}}' selects the pybind configurations.
--version V Stamped into each library's generated build files, and exported as XMS_VERSION so it reaches each profile's [buildenv] ahead of anything the environment would resolve (§9.3). Without it, each library's version is resolved from its own checkout, as xmsconan gen run inside it would (§10).
--remote-name NAME aquaveo-vs2019 The remote the preflight check requires. Match it to the --remote-name you gave setup; a machine set up against a different Artifactory repo otherwise fails preflight (exit 2) on a remote it was never meant to have.
-v / -q Verbose / quiet, after the verb (xmsconan vs2019 build -v). -q silences xmsconan's own progress lines and keeps its errors and the report tables (preflight, --preview, wheel summary, ==> Summary); the packager's per-configuration status lines and conan's output are unaffected. -v adds debug detail (§4).

Per library, in dependency order, build:

  1. Skips the library (not a failure) when <root>/<library> or its build.toml is missing, so a partially migrated stack still builds what it can. A library whose matrix is emptied by --filter is likewise reported as skipped, with no configurations matched --filter in the notes column.
  2. Writes the checkout's build files — what xmsconan gen [--version V] build.toml would write there — unless --no-generate. Anything that stops the generator — a build.toml it rejects, a file it cannot write, a fault in the generator itself — fails that library, and is logged as it happens, with its traceback under -v.
  3. Generates the windows_vs2019 matrix, applies --filter, and runs conan create per configuration.

A single failing configuration does not stop the library — the packager runs the rest and reports the count. A failing library stops the run unless --continue-on-error is passed. Either way a summary table (attempted / succeeded / failed / elapsed per library) is printed at the end.

Re-running is safe for the logs. A configuration's log always lands at the canonical <library>-<label>.log; if one is already there from an earlier run it is renamed to <library>-<label>.<timestamp>.log first (with a .1/.2 counter if two runs collide inside the same second). Re-running a failed matrix preserves the previous evidence rather than truncating it.

build regenerates in place, and a partial run leaves that behind. Step 2 regenerates inside your checkout, overwriting conanfile.py, CMakeLists.txt, and build.py and stamping them with the --version you passed. If the run fails, is interrupted, or you stop it after the summary, those regenerated files stay in the working tree — with the VS2019 version baked in. Check git status in each library before committing anything, or re-run xmsconan gen with the version you actually want. --no-generate skips this step entirely and builds whatever conanfile.py is already there.

Exit codes:

Code Meaning
0 At least one library built and none failed.
1 A library failed, or --wheel-dir was given and no complete set of wheels came out (§16.8). The second case reuses code 1 rather than adding a fourth: the run was asked for an artifact, it ran, and the artifact isn't there — and the xmsconan_wheel_repair you'd run next would be handed an empty directory.
2 The request or the machine was wrong: an unknown --only/--from name, a --filter that isn't a JSON object, a --python-versions entry the packager rejects, a --root that doesn't exist, a selection that matches no library at all (--only xmscore --from xmsgrid, or a --from past the last enabled library), a failed preflight — including the interpreter check in §16.8 — conan not on PATH, or a file the run needed that could not be read or written. The last two are told apart in the message: only an executable that would not start is reported as a PATH problem. A build.toml step 2 rejects, or a build file it cannot write, is not among them: it fails that library, which is 1.
4 The run completed but nothing was built — every selected library was skipped (no checkout, no build.toml, or --filter matched nothing). Not success: a typo in --root used to print a table of skips and exit 0, which any && chain or wrapper script read as "the stack is built". Not 3 either, since that is the coverage gate the generated CI forgives (§4.2); this track exited 3 for it before the vocabulary was shared.

16.5 The library list

The driver holds the XMS stack in dependency order — each library builds against the packages produced by the ones before it — with an enabled flag per entry:

xmscore → xmsgrid → xmsinterp → xmsmesher → xmsextractor
→ xmsstamper → xmsconstraint → xmsgridtrace → xmssnap

Every entry now has a Conan 2 recipe, so all of them are enabled. The flag remains so a library can be dropped from a plain build run without deleting its row; --only <lib> ignores the flag entirely, so a library can still be built while it is off.

xmssnap sits last because nothing builds against it: Python is its only consumer, and the pybind wheel is the only consumable artifact. xmsgridtrace has a Conan 2 recipe but no msvc 192 packages published yet.

A name that is not in this tuple cannot be built by the driver at all — --only/--from validate against it and exit 2 with unknown library. Adding a newly migrated library here is a one-line change in xmsconan/build_tools/vs2019_build.py.

16.6 The matrix — 14 configurations per library by default

With the default two Python versions, windows_vs2019 (msvc 192, x86_64, cppstd 17) produces:

Group Count Shape
base 4 Release/Debug × static/dynamic runtime
wchar_t=typedef 4 the same four, with the MSVC /Zc:wchar_t- toggle
testing 4 the same four, with testing=True
pybind 2 Release + dynamic runtime only, one per --python-versions entry

A library's [matrix] table (§5.4.1) narrows this the same way it narrows the CI matrix — the driver reads the build.toml in each checkout, so --preview and the build agree. pybind_build_types = ["Release", "Debug"] is how this track produces the Debug module the desktop products link.

windows_vs2019 is a normal platform key of XmsConanPackager.generate_configurations(), so it can be driven directly too (the valid keys are darwin, linux, windows, windows_vs2019; anything else raises ValueError naming the unknown key and listing the valid ones).

Two things the platform key does not imply, which the driver passes explicitly:

  • XmsConanPackager(..., apply_boost_defaults=False) — the boost/*:without_stacktrace and without_locale profile defaults name conan-center boost 1.86 options, and Conan fails a build when a profile sets an option no involved recipe declares. The legacy boost/1.74.0.3 may not declare them. Pass one of them through profile_options if you need it individually.
  • upload(version, remote='aquaveo-vs2019', package_query='compiler.version=192')XmsConanPackager.upload still defaults to aquaveo, and without a package_query conan upload matches by reference only: every binary of that version sitting in the local cache is published. On a workstation that is both the VS2019 build box and a normal msvc 194 dev machine, that quietly pushes msvc 194 binaries onto the VS2019 remote and exits 0. package_query is passed through to conan upload -p.

16.7 upload

xmsconan_vs2019 upload --library xmscore --version 7.0.0
Flag Default Effect
--library NAME required Library / Conan package name.
--version V required Package version. Every <library>/<version>* package in the local cache whose compiler.version is 192 is uploaded.
--remote NAME aquaveo-vs2019 Conan remote to upload to. Any other value is refused unless --allow-other-remote is also passed.
--allow-other-remote off Permit a --remote other than aquaveo-vs2019.
-v / -q Verbose / quiet, after the verb (xmsconan vs2019 upload -v). -q silences xmsconan's own progress lines; the packager's upload status lines, including its failure line, print regardless, as does conan's output. -v adds debug detail (§4).

Both --library and --version are required and there is deliberately no * default — a shared remote is the wrong place to discover that a wildcard matched more than you meant.

Two guards keep msvc 192 binaries off the CI remote, because the failure mode is silent and the cleanup is somebody else's afternoon:

  • The upload is restricted to compiler.version=192 (conan upload -p). Without it, conan upload matches by reference alone and takes the whole local cache — msvc 194 binaries included — along for the ride.
  • --remote aquaveo is one word away from --remote aquaveo-vs2019 and is refused (exit 2) with a message naming the right remote. Pass --allow-other-remote if you genuinely mean a different remote, e.g. a scratch repo.

Exit code 0 means the packages actually landed. A failing conan upload prints the reference, the remote, and conan's exit status, and the subcommand exits 1. XmsConanPackager.upload() returns 0/1 for this reason; the generated build.py --upload (§6) exits 1 on the same signal. upload runs no preflight, so conan missing from PATH surfaces here as exit 2 with a message rather than a traceback.

16.8 Python wheels — one run per Python version

The msvc 192 wheels published to Aquaveo's devpi index are built the same way as the Conan packages: by hand, on the VS2019 box. The whole sequence, for Python 3.10:

# from a Python 3.10 virtual environment with conan + xmsconan installed
xmsconan vs2019 build --root <root> --version 7.0.0 --python-versions 3.10 \
    --filter '{"options": {"pybind": true}}' --wheel-dir wheelhouse
xmsconan_wheel_repair --wheel-dir wheelhouse --platform windows
xmsconan_wheel_deploy --wheel-dir wheelhouse

Then repeat the whole thing from a 3.13 virtual environment, into a different --wheel-dir, if you need the 3.13 wheel too.

PowerShell mangles --filter. It strips the inner double quotes, so conan sees {options: {pybind: true}} and the driver rejects it as invalid JSON (exit 2). Run these commands from Git Bash, where they work exactly as written. In PowerShell you'd have to write --filter '{\"options\": {\"pybind\": true}}'; using Git Bash is the supported path.

Why one run per version. XmsConan2File hands CMake Python3_EXECUTABLE = sys.executable — the interpreter running conan — while the generated CMakeLists.txt requires find_package(Python3 ${PYTHON_TARGET_VERSION} EXACT REQUIRED). The recipe therefore assumes the interpreter running conan is the target Python. CI satisfies that implicitly: actions/setup-python installs the matrix version and conan runs under it. A workstation doesn't — nothing installs a matching interpreter for you — so from a 3.12 venv a --python-versions 3.10 run dies at configure with

Could NOT find Python3: Found unsuitable version "3.12.0",
but required is exact version "3.10" (found .../python.exe)

on every pybind configuration, after the non-pybind ones that don't care have already built. build catches this before compiling anything (§16.3) and exits 2, naming the version you're running, the version(s) the matrix wants, and both ways out. From a 3.10 venv everything lines up with no recipe change: the EXACT check passes, the wheel comes out tagged cp310, and the recipe's Python test venv is 3.10 as well.

What --wheel-dir does. After a clean build, per library:

  1. XmsConanPackager.extract_wheel(DIR, version=<--version or '*'>) copies each pybind package's .whl out of the Conan cache.
  2. XmsConanPackager.collect_dependency_libs(DIR/libs) gathers the shared libraries next to it — the same call, in the same place, that the generated build.py makes in CI. It is not optional on Windows: xmsconan_wheel_repair --platform windows passes delvewheel repair --add-path <DIR>/libs unconditionally, and a repair that finds nothing there yields a wheel that imports on the build box and fails everywhere else.

Then it prints what is staged and what comes next:

==> Wheels: 1 in E:\code\xms\migration\wheelhouse: xmscore-7.0.0-cp310-cp310-win_amd64.whl
    Next: xmsconan_wheel_repair --wheel-dir wheelhouse --platform windows, then xmsconan_wheel_deploy --wheel-dir wheelhouse

Points worth knowing:

  • Extraction is skipped when a configuration failed. The cache may still hold a wheel from an earlier run, and staging that for xmsconan_wheel_deploy is worse than staging none. The run already exits 1 on the failure itself.
  • A matrix with no pybind configuration is a failure, not a no-op. extract_wheel searches the whole local cache, so without that guard it would find last week's wheel and report success. If you pass --wheel-dir, select the pybind configurations (--filter '{"options": {"pybind": true}}') or build the full matrix.
  • A partial fan-out is a failure too. extract_wheel returns False when it finds wheels for only some of the --python-versions entries, which is why --python-versions 3.10 (exactly the version you're running) belongs in the command above: leaving the default 3.10 3.13 there would fail the interpreter check first, and a --filter narrowed to one python_version while --python-versions still lists two reports a missing wheel at the end.
  • The summary lists the whole directory, not just this run's copies — that directory is what repair and deploy act on next, so a wheel left over from an earlier run or a different version is about to be published too. Use a fresh --wheel-dir per version.
  • collect_dependency_libs walks the entire Conan cache, so on a box that is also a normal msvc 194 dev machine it stages DLLs from those packages as well. delvewheel only vendors libraries the wheel actually imports, but if you have same-named DLLs from both toolchains, repair from a clean cache or check the delvewheel output.
  • No devpi upload path lives in this driver. xmsconan_wheel_deploy is a separate command for the same reason upload is separate from build — see §13 for its credential resolution (AQUAPI_URL / AQUAPI_USERNAME / AQUAPI_PASSWORD, or ~/.xmsconan.toml).

17. Credentials (~/.xmsconan.toml)

Avoid passing credentials on every command:

[aquapi]
url      = "https://public.aquapi.aquaveo.com/aquaveo/dev/"
username = "your_username"
password = "your_password"

[conan]
username = "your_username"
password = "your_password"

Always overridden by CLI flags / env vars when present. The [conan] section is the last fallback for xmsconan conan-setup --login and for xmsconan vs2019 setup (§16.2), and [aquapi] for xmsconan wheel-deploy (§13) — all three resolve --password-file, then the environment ($CONAN_PASSWORD / $AQUAPI_PASSWORD), then this file, and none has a --password flag to put the secret on a command line. Don't commit this file. It's read-only as far as xmsconan is concerned.

A missing file is fine — credentials can come from flags or the environment instead. A file that exists but is not valid TOML raises Could not parse <path> rather than being treated as absent, so a mistyped config reports itself instead of surfacing later as "No devpi URL provided (--url, $AQUAPI_URL, or ~/.xmsconan.toml)" — advice to configure the very file that failed to parse.


18. Consuming an XMS library from another project

Once a release has been pushed to the Aquaveo Conan remote, downstream Conan consumers depend on it like any other Conan 2 package:

# downstream conanfile.py
class MyApp(ConanFile):
    settings = "os", "compiler", "build_type", "arch"

    def requirements(self):
        # C++-only consumer — no python_version, no pybind
        self.requires("xmscore/7.0.0")

    def configure(self):
        # If you DO want the Python bindings, set both:
        self.options["xmscore"].pybind = True
        self.options["xmscore"].python_version = "3.13"   # or "3.10"

Or with explicit options on the install:

conan install . \
    -s build_type=Release \
    -o "xmscore/*:pybind=True" \
    -o "xmscore/*:python_version=3.10"

The xms_dependencies field in your build.toml handles the same wiring automatically for sister XMS libraries.

18.1 Consuming the wheel (Python-only)

pip install xmscore -i https://public.aquapi.aquaveo.com/aquaveo/dev/+simple

Wheels are tagged cp310-cp310-... or cp313-cp313-...; pip picks the right one based on the active interpreter.


19. Troubleshooting

  • ImportError: libFoo.so: cannot open shared object file, or DLL load failed while importing _<name>, during the Python tests. The module links a dependency built as a shared library and cannot find it. The recipe handles this (§7.6) through conanrun off Windows and a generated sitecustomize.py on it; if it still happens, check that the dependency declares the directory holding its shared library in bindirs or libdirs.
  • auditwheel/delocate/delvewheel missing libraries. Run build.py --wheel-dir wheelhouse before repair — that step populates wheelhouse/libs/. Repairing without it produces a wheel that loads fine on the build host and crashes everywhere else.
  • PYTHON_TARGET_VERSION mismatch in CMake. The recipe sets it from the python_version Conan option. If you're poking CMake directly, pass -DPYTHON_TARGET_VERSION=3.13.
  • No pybind package found to extract. Means build.py ran but no pybind config was built. Check build.py --preview to see the matrix; common causes are --filter or the [filter] table in build.toml (§5.8) excluding the pybind variant, or every pybind variant having failed.
  • No configurations match the requested filters. The [filter] table and the --filter on the command line narrow the matrix together, and nothing survived both. The message lists what was applied; drop the conflicting --filter, loosen [filter] in build.toml, or pass --ignore-build-filter for a one-off build.
  • Dual wheel uploads colliding on devpi. With python_versions=["3.10","3.13"], wheels carry distinct cp3XY tags, so devpi treats them as separate uploads of the same release. No special config required.
  • Generated CI references a runner or image that doesn't exist. A GitLab Windows opt-in needs the GLR-UV image on a runner tagged WinVM; a Linux opt-in needs the matching conan-gcc13-py<version> container. If one isn't available, drop that version from the relevant list until it is — the platforms are independent, so Windows can carry a version Linux cannot.
  • Windows job dies on uv: command not found. The job is on a runner image that predates the uv migration (the per-ABI GLR-py310 / GLR-py313 VMs). The generated pipeline requires GLR-UV; check that image: GLR-UV resolves on the WinVM fleet rather than pinning the old images back.
  • Windows job fails with 'cmake' is not recognized as an internal or external command. Every configuration dies the same way, right after conan prints RUN: cmake -G "Visual Studio 17 2022" .... The GLR-py310 / GLR-py313 images carried cmake on PATH; GLR-UV does not. Pipelines generated by xmsconan 2.29.0 hit this; later versions install cmake into the job's own venv -- by name at first, and as part of the xmsconan[ci] extra since (§10.3). Regenerate .gitlab-ci.yml against the newer xmsconan rather than installing cmake on the runner snapshot — the venv install is what makes the job independent of how the VM was provisioned.
  • A wheel is built for the wrong ABI, then error: Failed to determine installation plan. The log shows something like Successfully built xmsfoo-0.0.0-cp314-cp314-win_amd64.whl in a job whose PYTHON_TARGET_VERSION is 3.10, and the install into the build's own venv then fails. uv build was given no interpreter and picked the newest one on the machine. Export UV_PYTHON (§11.3); xmsconan job build does it for you from PYTHON_TARGET_VERSION. A related symptom from the same cause is the wheel build failing outright with IndexError: list index out of range in setuptools/_distutils/compilers/C/msvc.py — see the entry below.
  • Wheel build dies with Call to setuptools.build_meta.build_wheel failed, and IndexError: list index out of range in the MSVC linker. The generated _package/pyproject.toml declares the module in ext-modules with sources = [], purely so the wheel gets a platform tag; the .pyd itself is built by CMake. setuptools normally skips that empty extension because build_py has already copied the CMake-built .pyd to the path build_ext would write, so it judges the extension up to date. When the interpreter building the wheel is not the one the .pyd was compiled for, the filename does not match, and setuptools links zero object files instead. optional = true does not catch it: setuptools only swallows CCompilerError / DistutilsError for an optional extension. The fix is the same — pin UV_PYTHON — and note this can surface while compiling a dependency from source, whose recipe came from an older xmsconan; the environment variable reaches those too.
  • Windows pybind leg fails find_package(Python3 ... EXACT REQUIRED). The venv was not active when build.py ran, so conan is running under the machine's interpreter rather than the matrix ABI. uv venv --python ${PYTHON_TARGET_VERSION} .venv and source .venv/Scripts/activate must both precede it; GitLab runs a job's script: lines as one shell, so the activation carries — but a step inserted above it does not get it.
  • Linux job fails pulling its container. The generated container: block pulls without credentials, which only works for a public image. On GHCR conan-gcc13-py3.13 is public but conan-gcc13-py3.14 is not, so a 3.14 Linux leg fails at pull until the package visibility is changed or a credentials: stanza is added.
  • xmsconan vs2019 build stops at preflight with "outside the pinned ~=2.31.0 series". Intentional (§16.3). Install the pinned client — pip install "conan~=2.31.0" — rather than working around the check; a different minor can change package_ids and detach your build from what's already published.
  • VS2019 build fails on a boost option Conan says no recipe defines. The legacy boost/1.74.0.3 doesn't declare the conan-center 1.86 options. The driver already passes apply_boost_defaults=False; if you're constructing XmsConanPackager yourself for msvc 192, pass it too (§16.6).
  • Conan Deploy - Windows VS2019 fails authenticating to aquaveo-vs2019. The job adds the remote but never logs in explicitly; Conan reads CONAN_LOGIN_USERNAME_AQUAVEO_VS2019 / CONAN_PASSWORD_AQUAVEO_VS2019 from the environment. Add both as GitLab CI variables. Only the tag-time deploy needs them, so the branch pipelines pass without them and the gap surfaces at the first release.
  • A deploy job went green but no binaries appeared on the remote. Check the job's -------- Upload summary --------: if it lists the recipe revision with an empty packages section, the recipe was published alone. A bare pkg/version is a recipe-only pattern, so conan upload pkg/version -p <query> filters an empty set of packages and exits 0; the pattern needs :*. Fixed in xmsconan 2.29.2 — versions 2.29.0 and 2.29.1 publish no binaries from any deploy step that passes --package-query, which is every Windows deploy. Recovery does not need a new tag: the deploy jobs install xmsconan[ci]>=, so retrying them once a fixed version is on the index republishes from the same build artifacts, as long as those have not expired (expire_in: 1d).
  • Binaries from the wrong toolchain turned up on a remote. Something published without its --package-query compiler.version=.... Both conan cache save and conan upload match by reference, and a runner's Conan cache is shared across the jobs on that machine, so an unqueried publish from the msvc 192 job carries whatever the msvc 194 job left behind. xmsconan job deploy applies the matching query on every Windows publish, in both directions, reading it from the packager matrix; a hand-run xmsconan conan-deploy without --package-query, or a build.py --upload whose --platform does not match the binaries in the cache, does not.
  • A tag pipeline went green but the remote has the recipe and no packages (No packages found for this revision). Fixed in the release carrying this note; if you are on an older one, republish by hand. xmsconan_conan_deploy --save --package-query ... resolved its package list with conan list "<ref>:*", which reports each binary as an info block with no package revision. conan cache save archives a package revision folder, so it saved the recipe and skipped every binary -- exit 0, tarball written, nothing in it -- and the deploy stage faithfully uploaded a lone recipe, also exit 0. The list now asks for revisions ("<ref>:*#*"). Two things made it hard to notice: only the queried path was affected, so the unqueried Linux deploys published correctly, and branch pipelines never upload, so it could only ever surface on a tag. Check a release with conan list "<ref>#<rrev>:*" -r aquaveo rather than trusting a green pipeline.
  • VS2019 packages don't show up for consumers. They go to aquaveo-vs2019, not aquaveo — the consuming machine needs that remote configured too. Note that xmsconan_vs2019 setup appends the remote rather than putting it first (§16.2), so it does not shadow aquaveo for your other work.
  • xmsconan vs2019 build exits 4 with a table of skipped rows. Nothing was built. Almost always a typo in --root (each library is looked for at <root>/<library>), a --filter that matches no configuration, or a library that has no build.toml yet. See the exit-code table in §16.4.
  • Could NOT find Python3: Found unsuitable version "3.12.0", but required is exact version "3.10". The interpreter running conan is the target Python for a pybind build (§16.8). Re-run from a 3.10 virtual environment, or build the version you're running with --python-versions 3.12. xmsconan vs2019 build now catches this at preflight and exits 2 before compiling anything — but only when the filtered matrix actually contains a pybind configuration.
  • --filter is not valid JSON in PowerShell. PowerShell strips the inner double quotes from '{"options": {"pybind": true}}'. Run it from Git Bash, or escape them: '{\"options\": {\"pybind\": true}}'.
  • xmsconan vs2019 build exits 1 with "no complete set of wheels". --wheel-dir was given but the run produced no wheel, or only part of the --python-versions fan-out. Usual causes: the matrix had no pybind configuration (add --filter '{"options": {"pybind": true}}'), or --python-versions listed a version the run never built. See §16.8.
  • xmsconan vs2019 upload refuses the remote. --remote is restricted to aquaveo-vs2019; anything else needs --allow-other-remote (§16.7). This is deliberate — --remote aquaveo would publish msvc 192 binaries into the remote every CI consumer resolves against.

20. Reference: shipped Conan profiles

Located under xmsconan/build_tools/profiles/{debug,release}/. Use the basename with xmsconan build --profile. The match is on the exact filename, so only the names that actually exist work — the combinations below are the complete list, not a pattern to extrapolate from:

Family Release Debug
GCC GCC5, GCC6, GCC7, GCC13, each also _TESTING and _PYBIND GCC5_D, GCC6_D, GCC7_D, GCC13_D, each also _TESTING_D
Clang CLANG9, CLANG9_TESTING, CLANG9_PYBIND, CLANG16_TESTING, CLANG16_PYBIND, CLANG17_PYBIND CLANG9_D, CLANG9_TESTING_D, CLANG16_D, CLANG16_TESTING_D
MSVC 192 (VS2019) VS2019, VS2019_TESTING, VS2019_TESTING_DYNAMIC, VS2019_PYBIND VS2019_D, VS2019_TESTING_D, VS2019_TESTING_DYNAMIC_D
MSVC 194 (VS2022) VS2022_TESTING VS2022_TESTING_D

There is no bare VS2022 profile, and no VS2022_TESTING_DYNAMIC in either flavor. xmsconan build --profile VS2022 fails with A valid --profile is required. Available profiles: [...] listing everything that does exist. (The full matrix builds — build.py, xmsconan vs2019 build — don't use these files at all: XmsConanPackager writes a temporary profile per configuration. These are for configuring a single build tree by hand, §9.2.)

profiles/base/ also sits on that walk, so its fragments (vs_2019, release, x64, pybind, …) are accepted as --profile values too. They are includes, not complete profiles; don't build with them directly.

Each suffix means:

  • _D — Debug build
  • _TESTING — Testing-enabled build (cxxtest/gtest runner)
  • _PYBIND — Pybind-enabled build (Release only)
  • _DYNAMIC (MSVC) — Dynamic CRT (MD/MDd) instead of static (MT/MTd)

For a custom mix, write your own profile that include()s entries from xmsconan/build_tools/profiles/base/ and pass it via --profile /path/to/profile.