Skip to content

[ci] Use UBTI for nightly#10200

Merged
seldridge merged 3 commits intomainfrom
dev/seldridge/ci-ubti-nightly
Apr 14, 2026
Merged

[ci] Use UBTI for nightly#10200
seldridge merged 3 commits intomainfrom
dev/seldridge/ci-ubti-nightly

Conversation

@seldridge
Copy link
Copy Markdown
Member

@seldridge seldridge commented Apr 13, 2026

Switch the nightly regression from using a circt/images Docker container
and a custom build/test to using the UBTI build. This is intended to be a
one-to-one change where all jobs running previously will run the same as
before (just on different infrastructure).

This is done for three reasons: (1) the combination of docker images
running on hosted runners has empirically been broken for some time (both
on the external MSFT-hosted runners as well as in at least one internal
hosted GitHub Enterprise instance), (2) moving onto a single build flow
for everything greatly simplifies how things are built and tested (there
is exactly one build/test flow), and (3) the UBTI flow is more easily
modifiable as it is self-documenting as opposed to being split over the
circt/images and llvm/circt repositories.

Assisted-by: Claude:claude-sonnet-4-6

This is stacked on #10188.

I'm not going to try a full nightly until #10188 lands (to get the Verilator and Yosys caching). However, I've started and cancelled a workflow dispatch of this here: https://github.com/llvm/circt/actions/runs/24372135856 This is showing all 17 jobs being launched. A benefit about this is that we now have more flexibility in launching nightlies over different runners. (The integration tests won't run fully for anything other than Linux right now, but this can be made to work.)

Full nightly is running here: https://github.com/llvm/circt/actions/runs/24407900984

@fabianschuiki
Copy link
Copy Markdown
Contributor

Results of circt-tests run for b2295df compared to results for 40af603: no change to test results.

@uenoku
Copy link
Copy Markdown
Member

uenoku commented Apr 14, 2026

Is a normal runner able to build Debug? I thought the primary reason to use MSFT's runner was to be able to build debug which normal runners were not able to.

@seldridge
Copy link
Copy Markdown
Member Author

Is a normal runner able to build Debug? I thought the primary reason to use MSFT's runner was to be able to build debug which normal runners were not able to.

We're going to find out! https://github.com/llvm/circt/actions/runs/24377364005

I thought I was getting debug builds through normal tests (check-circt) with the pre-merge CI flow no problem. The issue was just that the caches were too large. I'll see what the fully nightly build does and see where we're at. Valgrind could be a problem, too.

Base automatically changed from dev/seldridge/ci-ubti-integration-tests to main April 14, 2026 02:34
@uenoku
Copy link
Copy Markdown
Member

uenoku commented Apr 14, 2026

Looks like Debug was built successfully! But https://github.com/llvm/circt/blob/main/integration_test/circt-synth/comb-lowering-lec.mlir caused timeout 🙃 Certainly debug caused either Z3 or JIT execution very slow. I'll reduce problem sizes or split files.

@uenoku
Copy link
Copy Markdown
Member

uenoku commented Apr 14, 2026

#10202 should fix this hopefully.

Switch the nightly regression from using a `circt/images` Docker container
and a custom build/test to using the UBTI build.  This is intended to be a
one-to-one change where all jobs running previously will run the same as
before (just on different infrastructure).

This is done for three reasons: (1) the combination of docker images
running on hosted runners has empirically been broken for some time (both
on the external MSFT-hosted runners as well as in at least one internal
hosted GitHub Enterprise instance), (2) moving onto a single build flow
for everything greatly simplifies how things are built and tested (there
is exactly one build/test flow), and (3) the UBTI flow is more easily
modifiable as it is self-documenting as opposed to being split over the
`circt/images` and `llvm/circt` repositories.

Assisted-by: Claude:claude-sonnet-4-6
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
@seldridge seldridge force-pushed the dev/seldridge/ci-ubti-nightly branch from b2295df to 7884b61 Compare April 14, 2026 15:27
@fabianschuiki
Copy link
Copy Markdown
Contributor

Results of circt-tests run for 7884b61 compared to results for ed3f87d: no change to test results.

Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
@seldridge
Copy link
Copy Markdown
Member Author

8403175 fixes a bug in sccache keys which are case-sensitive. Somewhat interestingly, it seems like GitHub actions conditional syntax is case insensitive which would have made this more obvious.

Each Release sccache is 100--200MiB which should enough for us to fir the ~10 of these that a nightly build kicks off. We might even be able to get hits from nightly-to-nightly (as opposed to just from pre-merge-to-nightly).

I'm going to do one more end-to-end run to make sure this is good and then merge.

@fabianschuiki
Copy link
Copy Markdown
Contributor

Results of circt-tests run for 8403175 compared to results for 7884b61: no change to test results.

Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
@fabianschuiki
Copy link
Copy Markdown
Contributor

Results of circt-tests run for 1047a95 compared to results for 8403175: no change to test results.

@seldridge seldridge marked this pull request as ready for review April 14, 2026 21:39
@seldridge seldridge requested a review from teqdruid as a code owner April 14, 2026 21:39
@seldridge
Copy link
Copy Markdown
Member Author

I'm seeing this working for everything except the known valgrind nightly failure: #9914. I'll merge this and treat the valgrind failure as follow-up work.

@seldridge seldridge merged commit baaf71e into main Apr 14, 2026
6 of 8 checks passed
@seldridge seldridge deleted the dev/seldridge/ci-ubti-nightly branch April 14, 2026 21:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants