Skip to content

test(chain): run each scenario as its own file on a stack of its own - #701

Merged
TheGreatAxios merged 3 commits into
mainfrom
cl-11274-fast-chain
Oct 28, 2026
Merged

TheGreatAxios merged 3 commits into
mainfrom
cl-11274-fast-chain

Conversation

@TheGreatAxios

Copy link
Copy Markdown
Contributor

The chain test's scenarios now run at the same time, each on a stack of its own, so CI waits for the slowest scenario instead of all of them in a row. This replaces #697, which closed when its base branch (#696) merged.

  • tests/chain/chain.test.ts is split into one file per scenario: deterministic, judged, waiting, pushed, held, woken, unwoken and disconnected.
  • Their shared setup and helpers move to tests/chain/chain.ts. Each file calls chainStack() at its top level, which starts its own hub, database, fake GitHub and stub model for that file. A file gets its own stack however the files are run.
  • The only code removed is after(), which made each scenario wait for the one before it so they could share one stack. Every test name and every assertion is the same as before (checked line by line), and the run still has the same 12 tests.
  • bun run test:chain runs two files at a time. CI runs the chain in five jobs (--shard=k/5), two files per job.

Times

Before, main run 40568541009: the chain step alone took 18m29s, and the whole check job took 22m50s.

After, run 40621672007 on this change before chainStack(): 4m09s wall from first job start to last job end. The chain shards took 1m44s, 2m44s, 4m09s, 2m28s and 2m20s; the slowest holds unwoken, which waits out the listener's two-minute key wait.

Locally with chainStack(): bun run test:chain passed all 12 tests, and a plain bun test of two chain files in one process passed both, each on its own stack.

Overlap with #695

#695 edits tests/chain/chain.test.ts, which this pull request splits. Whichever lands second moves its scenario edits into the matching per-scenario file, and #695's new reread scenario becomes a file of its own. tests/chain/stack.ts is not touched here.

The manager's and listener's production timers stay as they are, and scenarios keep a stack each.

Closes CL-11274

Each file starts its own hub on its own database, so scenarios run at once
and none counts another's runs. CI shards the files across five jobs, two
files at a time in each.
Each test file calls it at its top level, so a file gets its own stack
whether or not files share a module registry.
@linear-code

linear-code Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

CL-11274

CL-9988

CL-11609

@TheGreatAxios

Copy link
Copy Markdown
Contributor Author

Independent review of 78510a6.

No findings.

What I checked:

  • Every test name and every expect line in main's tests/chain/chain.test.ts appears unchanged in exactly one of the eight scenario files: 10 test names and 56 expect lines, diffed as sorted lists with no differences and no duplicates. Diffing the whole scenario bodies the same way, the only changes are the removed after() calls, hub and workspace read through chain.hub and chain.workspace, and the imports and file headers. The shared setup in tests/chain/chain.ts matches main's line for line apart from being wrapped in chainStack().
  • No scenario loses a precondition. In main, judged, waiting, pushed and held waited on the scenario before them, and woken, unwoken and disconnected all waited on held, so those three already ran at the same time. None of them read the earlier scenario's results: each counts its own managers, model decisions and GitHub misses from a baseline it takes itself. The one assertion that relied on running first, expect(model.decisions).toEqual([]) in deterministic, still holds because each file has a fresh stub model. unwoken's key trigger, woken's model.hold() and disconnected's SIGSTOP of the listener's sidecar now only affect that file's own stack, which is a tighter setup than before.
  • Each stack is separate: a random database name (corbits_chain_<hex>), a new hub port, new mkdtemp directories for the portal build and the hub's data, and port 0 for the fake GitHub and the stub model. chain.ts has no mutable module-level state, because everything lives inside chainStack().
  • Shards: test:chain points at the tests/chain directory, and --shard=k/5 assigns the sorted files round-robin. I checked this with ten dummy files on Bun 1.4.4, and the CI logs of this run show each of the 10 chain files in exactly one shard, 12 tests in total. A file added later is picked up by the directory path and lands in a shard automatically, so it cannot be skipped silently. An eleventh file would give one shard three files.
  • No production code, vendor/ or tooling/ changed. Only ci.yml, package.json, docs/DEV.md and tests/chain/ did.
  • The docs agree with the change. The chain.ts header and the docs/DEV.md text describe what now happens, and the tests/chain/ line in AGENTS.md already says CI runs the chain as its own job.

How I verified it: oxlint on tests/chain is clean. bun check -p tests/chain on Bun 1.4.4 reports no type errors. bun --conditions=intx-src test tests/chain/deterministic.test.ts tests/chain/disconnected.test.ts passed 3 of 3 locally in one process, each file on its own stack, in 222s. All five CI chain shards are green.

@TheGreatAxios
TheGreatAxios merged commit 1c1919c into main Oct 28, 2026
17 checks passed
@TheGreatAxios
TheGreatAxios deleted the cl-11274-fast-chain branch October 28, 2026 03:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant