Future: single shared Botan build per cargo graph (mixed rnp[vendored] + botan[vendored])

[ronaldtse]

Summary

Future enhancement — not blocking users today. When a build graph contains both rnp[vendored] and botan[vendored] (e.g. a tool needing OpenPGP plus raw Botan primitives), Botan is compiled twice and the final link binds whichever libbotan-3 archive comes first in search-path order.

What happens today

• botan-sys (via botan[vendored]) compiles Botan in its OUT_DIR; rnp-src compiles its own in another. botan_src::build() lands the build in whichever crate's OUT_DIR calls it — there is no shared cache.
• Cargo cannot unify 0.x minor versions, so with version skew the graph carries two different Botan versions. Both emit -lbotan-3 with different search paths; one archive silently wins.
• The loser's consumer breaks: either missing-symbol link errors, or — worse — a silent C++ ABI/layout mismatch (Botan's C++ ABI is unstable across minors, per its sonames). Nothing warns.

Mitigations already shipped

• Version alignment (feat(rnp-src): Botan 3.13 alignment + mixed-graph duplicate-Botan detection #75): rnp-src pins the same botan-src as botan-sys (0.31300.0 = Botan 3.13.0), so mixed graphs compile the same version twice — wasteful (~10 min), but correct.
• Detection (feat(rnp-src): Botan 3.13 alignment + mixed-graph duplicate-Botan detection #75, opt-in botan-sys-detect feature): loud cargo:warning when another vendored Botan is present in the graph.
• README guidance: prefer system Botan for both crates (one shared libbotan, zero duplication), then aligned-vendored.

The actual goal

Exactly one Botan compilation per cargo build graph, shared by every consumer. Real fixes live upstream in botan-rs:

1. A shared build cache inside botan-src, keyed by version/target/configuration — universal, but pushes cache-key correctness, locking, and staleness onto upstream.
2. Links metadata (lib_dir/include_dir) from botan-sys's vendored path — small, but insufficient in general: build scripts cannot retroactively configure a dependency's already-finished build (e.g. our BOTAN_CONFIGURE_ENABLE_MODULES for PQC runs before our own Botan build, which ordering can't work when botan-sys's build already ran), and it would expose botan-sys's build-tree layout as a quasi-API.

A meek, constraint-framed issue draft for botan-rs exists and will be filed when we prioritize this. Until then, the mitigations above keep mixed graphs correct-if-wasteful.

[ronaldtse]

Upstream conversation opened: randombit/botan-rs#199 (sharing one vendored Botan build across crates that compile Botan — question / use-case report).

Status recap since this issue was filed:

• The "aligned pins" mitigation noted above is now fully in effect again after fix(rnp-src): backport RSA short-MPI padding fix, unpin Botan to 3.13.0 #93: the vendored path builds Botan 3.13.0 with the RSA short-MPI backport applied, so mixed rnp[vendored] + botan[vendored] graphs compile the same Botan version twice — correct, still wasteful.
• The detection warning (botan-sys-detect) and the README guidance (prefer system Botan for both) remain the day-to-day mitigations.

The real fix still lives upstream; both candidate mechanisms (links metadata vs. a shared content-addressed cache in botan-src) are analyzed in randombit/botan-rs#199 with their constraints (notably: module-set configuration can't be retrofitted onto an already-built dependency, which matters for our PQC module set). Keeping this issue open as the tracking item; closing criteria = one of the upstream mechanisms lands and rnp-src adopts it.