Skip to content

fix(build): repair the macOS conda build and the CUDA wheel toolchain - #1185

Merged
IvanaGyro merged 4 commits into
masterfrom
claude/dazzling-ritchie-zc681l
Sep 18, 2026
Merged

IvanaGyro merged 4 commits into
masterfrom
claude/dazzling-ritchie-zc681l

Conversation

@IvanaGyro

@IvanaGyro IvanaGyro commented Sep 17, 2026

Copy link
Copy Markdown
Member

Two independent build breakages, both latent problems that surfaced when an external toolchain moved. Neither is caused by a source change in this repository.

1. Missing <iterator> breaks the macOS conda build

Problem

src/utils/vec_intersect.cpp calls std::back_inserter in vec2d_intersect, vec_intersect and both vec_intersect_ overloads, and std::distance in vec_intersect_, but includes only <algorithm> and <vector>. Both names are declared in <iterator>.

libstdc++ reaches them anyway, because <vector> and <algorithm> pull in <bits/stl_iterator.h> — so the missing include is invisible on Linux and ci-cmake_tests (Ubuntu/GCC) stays green. libc++ drops those transitive includes in C++20 mode, which is the mode this project builds in (CMAKE_CXX_STANDARD 20), so the file does not compile on macOS:

src/utils/vec_intersect.cpp:18:76: error: no member named 'back_inserter' in namespace 'std'
   18 |     std::set_intersection(v1.begin(), v1.end(), v2.begin(), v2.end(), std::back_inserter(out));
      |                                                                            ^~~~~~~~~~~~~
4 errors generated.

Both macOS jobs of Conda Build (Test build) die there at 56%, taking the wheel build with them; ubuntu-latest passes. master is affected too. The file last changed on 2026-06-30 in 43fd1fd, and the same code still built on master as recently as run 32725171357 (2026-08-24, 32d5d82); the conda-forge macOS toolchain has since moved to a libc++ that no longer supplies the transitive include.

Fix

Add #include <iterator>.

Scope of that claim, narrowed after review: it holds for back_inserter — the only other two users, include/utils/vec_clone.hpp and src/backend/linalg_internal_cpu/Gemm_Batch_internal.hpp, include <iterator> directly. It does not hold for std::distance, which nine files name without a direct include: src/utils/vec_unique.cpp, src/utils/vec_where.cpp, src/Bond.cpp, src/BlockUniTensor.cpp, src/BlockFermionicUniTensor.cpp, src/UniTensor_base.cpp, src/DenseUniTensor.cpp, include/UniTensor.hpp and include/LinOp.hpp. All of them compile today: seven reach <iterator> through the chain utils.hppvec_clone.hpp (which includes it at line 5), and vec_unique.cpp / vec_where.cpp only because libc++'s <algorithm> still supplies std::distance. That last pair is the same fragility class as the bug fixed here, but is not a current breakage and is not addressed by this PR.

2. Unpinned nvcc siblings break the CUDA wheel build

Problem

CUDA_BUILD_TOOLCHAIN in tools/prepare_cuda_release.py pins nvidia-cuda-nvcc, which supplies ptxas, but not nvidia-nvvm, which supplies cicc. nvcc splits the device compile between them: cicc emits the PTX and ptxas assembles it, and ptxas rejects a PTX ISA newer than its own.

Neither nvidia-nvvm nor nvidia-cuda-crt is named in the list. They arrive through nvidia-cuda-nvcc, whose requires_dist is ["nvidia-nvvm", "nvidia-cuda-runtime", "nvidia-cuda-crt"] with no version bounds, so pip resolves them to the newest CUDA minor version while ptxas stays pinned. Resolving the current list shows the split:

nvidia-cuda-nvcc  13.3.73     <- ptxas
nvidia-nvvm       13.4.92     <- cicc
nvidia-cuda-crt   13.4.92

A 13.4 cicc emits PTX ISA 9.4, which the 13.3 ptxas cannot assemble, so the build fails at enable_language(CUDA) (CMakeLists.txt:270) before any project source is compiled:

ptxas tmp/CMakeCUDACompilerId.ptx, line 9; fatal : Unsupported .version 9.4; current version is '9.3'
ptxas fatal : Ptx assembly aborted due to errors

nvidia-nvvm 13.4.59 was published 2026-09-09 at 18:02 UTC, under four hours after the last green Release CUDA wheels run. The build has no lockfile, so every run since re-resolves to 13.4.

Fix

Pin nvidia-nvvm and nvidia-cuda-crt to nvcc's own version so the whole compiler comes from one CUDA minor release. Re-resolving with the pins puts all three on 13.3.73.

pixi.toml's [target.linux-64.pypi-dependencies] block states that its ranges must match CUDA_BUILD_TOOLCHAIN, and it pinned nvidia-cuda-nvcc only, so the same two constraints are added there. pixi.lock already held all three at 13.3.73, so no environment was broken in practice and the lock is unchanged — pixi lock reports it already up to date. The exposure was a future re-resolve.

This keeps the toolchain on 13.3 rather than bumping to 13.4; it does not change what the published wheel links against or the driver it requires.

Testing

Neither failure reproduces on Linux/GCC, so both fixes were verified by CI on this PR, on the toolchains that actually rejected the code:

  • BuildAndTest-macos-latest and BuildAndTest-macos-15-intel (Conda Build) — both pass. These are the jobs that were dying at 56% on back_inserter.
  • BuildWheel-ubuntu-24.04 (Release CUDA wheels) — passes, taking 2h02m to build; before the pin it failed after about 2 minutes at CMake's CUDA compiler-ID probe.
  • All other checks green: Formatting Check, BuildAndTest (ctest + pytest), DownstreamFindPackage, check (version consistency), every CPU wheel leg, Codecov.

Local runs, which confirm no regression rather than demonstrating either fix:

  • debug-openblas-cpu — build + ctest: 1839/1839 passed
  • debug-mkl-cpu — build + ctest: 1873/1873 passed
  • pre-commit on all changed files: clean (clang-format v14; .clang-format sets SortIncludes: false, so the new include stays where it is written)
  • pip resolution of the CUDA specs: without the pins nvidia-nvvm/nvidia-cuda-crt come back 13.4.92 against a 13.3.73 ptxas; with them all three resolve to 13.3.73

Remaining scope note: the CUDA wheel job builds on a GPU-less runner and only asserts cytnx.Device.Ngpus == 0, so it proves the toolchain compiles and links, not that device code runs correctly.

vec_intersect.cpp calls std::back_inserter in vec2d_intersect,
vec_intersect and both vec_intersect_ overloads, and std::distance in
vec_intersect_, but includes only <algorithm> and <vector>. Both names
are declared in <iterator>.

libstdc++ reaches them anyway because <vector> and <algorithm> pull in
<bits/stl_iterator.h>, so the missing include is invisible on Linux.
libc++ drops those transitive includes in C++20 mode, which is the mode
this project builds in (CMAKE_CXX_STANDARD 20), so the file does not
compile on macOS:

    src/utils/vec_intersect.cpp:18:76: error: no member named
    'back_inserter' in namespace 'std'

Include <iterator> directly. This is the only file under src/ or
include/ that names back_inserter or std::distance without it.

Co-Authored-By: Claude <noreply@anthropic.com>
@IvanaGyro IvanaGyro changed the title fix(utils): include &lt;iterator&gt; in vec_intersect.cpp fix(utils): include <iterator> in vec_intersect.cpp Sep 17, 2026
@IvanaGyro IvanaGyro changed the title fix(utils): include <iterator> in vec_intersect.cpp fix(utils): add the missing iterator include in vec_intersect.cpp Sep 17, 2026
@codecov

codecov Bot commented Sep 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.89%. Comparing base (32d5d82) to head (194a42f).
⚠️ Report is 5 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1185   +/-   ##
=======================================
  Coverage   68.89%   68.89%           
=======================================
  Files         208      208           
  Lines       22411    22411           
  Branches       72       72           
=======================================
  Hits        15439    15439           
  Misses       6950     6950           
  Partials       22       22           
Flag Coverage Δ
cpp 68.97% <ø> (ø)
python 63.96% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
C++ backend 68.97% <ø> (ø)
Python bindings ∅ <ø> (∅)
Python package 63.96% <ø> (ø)
Files with missing lines Coverage Δ
src/utils/vec_intersect.cpp 57.14% <ø> (ø)

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 32d5d82...194a42f. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

CUDA_BUILD_TOOLCHAIN in tools/prepare_cuda_release.py pins
nvidia-cuda-nvcc, which supplies ptxas, but not nvidia-nvvm, which
supplies cicc. nvcc splits the device compile between them: cicc emits
the PTX and ptxas assembles it, and ptxas rejects a PTX ISA newer than
its own.

Neither nvidia-nvvm nor nvidia-cuda-crt is named in the list today. They
reach the toolchain through nvidia-cuda-nvcc, whose requires_dist is
["nvidia-nvvm", "nvidia-cuda-runtime", "nvidia-cuda-crt"] with no
version bounds, so pip resolves them to the newest CUDA minor version
available while ptxas stays on the pinned one. Resolving the current
list shows the split:

    nvidia-cuda-nvcc  13.3.73     <- ptxas
    nvidia-nvvm       13.4.92     <- cicc
    nvidia-cuda-crt   13.4.92

A 13.4 cicc emits PTX ISA 9.4, which the 13.3 ptxas cannot assemble, so
the CUDA wheel build fails at enable_language(CUDA) in
CMakeLists.txt:270, before any project source is compiled:

    ptxas tmp/CMakeCUDACompilerId.ptx, line 9; fatal : Unsupported
    .version 9.4; current version is '9.3'

Pin both to nvcc's own version so the whole compiler comes from one
CUDA minor release. Re-resolving with the pins puts nvidia-cuda-nvcc,
nvidia-nvvm and nvidia-cuda-crt all on 13.3.73.

Co-Authored-By: Claude <noreply@anthropic.com>
@IvanaGyro IvanaGyro changed the title fix(utils): add the missing iterator include in vec_intersect.cpp fix(build): repair the macOS conda build and the CUDA wheel toolchain Sep 17, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 71402fc6e7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tools/prepare_cuda_release.py
Comment thread tools/prepare_cuda_release.py
pixi.toml's [target.linux-64.pypi-dependencies] block states that its
version ranges must match CUDA_BUILD_TOOLCHAIN and
CUDA_RUNTIME_DEPENDENCIES in tools/prepare_cuda_release.py. It pinned
nvidia-cuda-nvcc but not nvidia-nvvm or nvidia-cuda-crt, which
nvidia-cuda-nvcc requires without version bounds.

pixi.lock currently holds all three at 13.3.73, so existing environments
are unaffected. A re-resolve, however, was free to pick a 13.4 cicc
(nvidia-nvvm) against the pinned 13.3 ptxas (nvidia-cuda-nvcc), which
reproduces locally the unsupported-PTX-ISA failure the release path now
guards against.

Add the two constraints so both paths hold the whole compiler to one
CUDA minor release. `pixi lock` reports the lock file already up to
date -- the locked versions satisfy the new ranges -- so pixi.lock is
unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>
@manuschneider

Copy link
Copy Markdown
Collaborator

Add #include . It is the only file under src/ or include/ that names back_inserter or std::distance without including it.

The "only file" claim is wrong for std::distance, and residual risk remains. The PR states vec_intersect.cpp "is the only file under src/ or include/ that names back_inserter or std::distance without including it." That's true for back_inserter (the other two users include <iterator> directly), but src/utils/vec_unique.cpp and src/utils/vec_where.cpp use std::distance with no <iterator> anywhere in their include closure (only <vector>, <algorithm>, <cstring>, Type.hpp). They compile on macOS today only because libc++'s <algorithm> still supplies std::distance transitively — exactly the fragility class this PR fixes. Additionally, UniTensor.hpp, Bond.cpp, LinOp.hpp and others get <iterator> only via the chain utils.hppvec_clone.hpp. Not a current breakage, but worth two more one-line includes, either here or as a follow-up.

Comment thread tools/prepare_cuda_release.py Outdated
# dropped here since this only ever runs inside the Linux manylinux container.
CUDA_BUILD_TOOLCHAIN = [
"nvidia-cuda-nvcc ~=13.3.73",
# nvidia-nvvm and nvidia-cuda-crt are not listed here because the build

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This contradicts the code it sits on. "nvidia-nvvm and nvidia-cuda-crt are not listed here because the build calls them directly" is claimed directly above lines that do list them. The intended meaning (listed because of unpinned resolution, not direct use) only emerges on a third read. The pixi.toml version of the same comment ("Named although nvidia-cuda-nvcc already depends on them: …") is clear; the script comment should be reworded to match.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — reworded in 194a42f. The sentence parsed first as "these are not listed here", two lines above the entries that list them; the intended reading (direct use is not the reason they appear) only arrived on re-reading. It now matches the pixi.toml wording:

    # Named although nvidia-cuda-nvcc already depends on them: it does so
    # without version bounds, which lets pip resolve a newer CUDA minor
    # version for cicc (nvidia-nvvm) than for ptxas (nvidia-cuda-nvcc), and
    # ptxas rejects a PTX ISA newer than its own. Must match the
    # nvidia-cuda-nvcc version above.

Comment text only; CUDA_BUILD_TOOLCHAIN is unchanged, and both CPU presets still build and pass (1839/1839 and 1873/1873).


Generated by Claude Code

The comment opened "nvidia-nvvm and nvidia-cuda-crt are not listed here
because the build calls them directly", two lines above the entries that
list them. The intended reading is that direct use is not the reason they
appear, but the sentence parses first as a claim that they are absent.

Restate it the way the equivalent comment in pixi.toml already does --
named despite nvidia-cuda-nvcc depending on them, because that dependency
carries no version bound -- so both copies read the same way.

Comment text only; CUDA_BUILD_TOOLCHAIN is unchanged.

Co-Authored-By: Claude <noreply@anthropic.com>

Copy link
Copy Markdown
Member Author

You're right, and the scope is wider than two files. Checked against the tree:

back_inserter — the claim holds. Three users, and the other two (include/utils/vec_clone.hpp, src/backend/linalg_internal_cpu/Gemm_Batch_internal.hpp) include <iterator> directly.

std::distance — the claim is wrong. Nine files name it with no direct <iterator>:

file reaches <iterator> via
src/utils/vec_unique.cpp nothing — libc++'s <algorithm> only
src/utils/vec_where.cpp nothing — libc++'s <algorithm> only
src/Bond.cpp, src/BlockUniTensor.cpp, src/BlockFermionicUniTensor.cpp, src/UniTensor_base.cpp, src/DenseUniTensor.cpp, include/UniTensor.hpp, include/LinOp.hpp utils.hppvec_clone.hpp (which includes it at line 5)

So vec_unique.cpp and vec_where.cpp are the genuinely exposed pair, exactly as you describe; the other seven lean on a header chain that could be broken by an unrelated include cleanup rather than by a libc++ change. Same fragility class either way, none of it a current breakage.

The PR description has been corrected — it no longer claims std::distance coverage, and it names all nine files and how each one resolves.

On where to fix it: leaving it out of this PR for now. This one is two build repairs that are both red on master, and the author would rather decide the include-hygiene sweep separately than widen it a third time. Not declining the substance — just not bundling it here.


Generated by Claude Code

@IvanaGyro
IvanaGyro merged commit 59eacea into master Sep 18, 2026
24 checks passed
@IvanaGyro
IvanaGyro deleted the claude/dazzling-ritchie-zc681l branch September 18, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants