feat(parallel): per-VU iteration counter for customParallelIterations - #1555
Open
scriptonist wants to merge 9 commits into
Open
feat(parallel): per-VU iteration counter for customParallelIterations#1555scriptonist wants to merge 9 commits into
scriptonist wants to merge 9 commits into
Conversation
- Add Run.isCustomParallelIterations as the single source of truth for the mode gate (Change 0) - Initialize Partition.loopIteration=0 and Partition.stopped=false (Change 1, 6c prep) - Raise cursor.cycles to MAX_SAFE_INTEGER in custom mode so seek doesn't trip the bounds guard (Change 2) - Increment partition.cursor.iteration on each runSinglePartition call in custom mode (Change 3) - Expose Partition.resetVariables() public helper for the full-fresh- on-reuse contract landing in a later change (Change 6b prep) Tests: 12 new unit cases in test/unit/custom-parallel-iterations.test.js covering construction-time invariants, the Run gating helper, and multi-loop counter monotonicity. All 449 unit tests pass.
- Change 4: gate the end-of-partition early-return at parallel.command:103 on !isCustomParallelIterations. Without this, loop 2+ would short- circuit (coords.iteration === startIndex + partitionCycles is true every loop in custom mode because startIndex=0, partitionCycles=1) and the runtime would skip every item. - Change 5: in the coords.cr block, after firing iteration trigger, early-return next() in custom mode. The host (perftest) drives the next loop via startParallelIteration; the runtime must not auto-loop in parallel. beforeIteration also no longer fires here in custom mode — startParallelIteration's own parallel-command invocation carries start:true and fires it. Tests: 4 new unit cases on the parallel processor (with mocked context) — Change 4 in custom + runtime-managed modes, Change 5 in custom + runtime-managed modes. 453 passing (was 449).
- Change 6: stopSinglePartition resets partition.loopIteration to 0, re-clones partition.variables via the resetVariables() helper, and sets partition.stopped=true. Recycled partitions behave as brand-new VUs — pm.info.iteration === 0 with a fresh pm.variables scope. Gated on isCustomParallelIterations so runtime-managed mode is untouched. - Change 6c: updatePartitionVariables drops writes when partition.stopped is true. Guards the late-write race where a script in flight at stopSinglePartition time finishes after the variables re-clone, otherwise leaking the dead VU's pm.variables mutations into the next VU's fresh scope. Tests: 9 new unit cases — loopIteration reset, stopped flag toggle, variables re-clone, flag clear on next runSinglePartition, full-cycle regression (stop→start→counter=0), runtime-managed-mode non-mutation, late-write drop happy + race + missing-partition paths. 462 unit tests passing (was 453).
…7, 9)
- Change 7: at the host.execute call site in event.command.js,
transform cursor.cycles to -1 when isCustomParallelIterations is
true. -1 is the wire sentinel that survives JSON/structured-clone
encoding (Infinity would JSON.stringify to null,
Number.MAX_SAFE_INTEGER would leak to pm.info.iterationCount).
Sandbox-side rendering of -1 -> Infinity is the matching change in
postman-sandbox/lib/sandbox/pmapi.js. Helper factored out as
applySandboxCursorSentinel and exposed for unit tests.
- Change 9: in the eof branch of parallel.command, fire the iteration
trigger with payload.coords ('loop just completed') in custom mode
rather than the post-rollover snapshot. Gated to custom mode to
preserve existing Newman/desktop semantics — flipping the global
contract risks silent downstream regressions we can't audit from
inside postman-runtime.
Tests: 5 new unit cases — sentinel transform happy + non-mutation +
no-op paths, eof trigger custom + runtime-managed payloads.
467 unit tests passing (was 462).
…ions Self-contained driver that exercises the perftest invocation pattern: runner.run() -> run.start() -> startParallelIteration() loop until maxLoops -> abort. Verifies the full chain: runtime drives the loop -> host.execute hands the (sentinel-transformed) cursor to the sandbox -> script reads pm.info.iteration + pm.info.iterationCount. Tests cover: - T1: first loop sees iteration=0 (via runtime trigger cursor) - T3: monotonic iteration across 3 loops ([0, 1, 2]) - T2/loop-2-runs-all-items: no skip-everything regression - T4/T5 trigger counts: iteration + beforeIteration fire once per loop - T6/T6b: stop+restart resets cursor.iteration to 0 AND re-clones pm.variables (verified via in-script marker) - T10 regression: maxConcurrency=2 mode still completes normally - T12: pm.info.iterationCount === Infinity end-to-end (cross-repo wire contract verified) Note: T12 requires the matching postman-sandbox change (cycles===-1 rendered as Infinity in pmapi.js). Until the sandbox version pinned in package.json is bumped, T12 fails locally — run 'node npm/cache.js' in the local postman-sandbox repo and copy .cache/bootcode.js into node_modules/postman-sandbox/.cache/ to verify locally before release.
Lint: use object spread instead of Object.assign, repo noop convention for empty test callbacks, and wrap/format long lines per style rules. The per-partition cookie jar unit test mocked only the raw customParallelIterations option, so after stopSinglePartition migrated to the derived isCustomParallelIterations flag the custom-mode branch never executed. Mock now mirrors real Run construction; assertions unchanged.
Custom-mode runs completed via triggerStopAction -> triggers(null) without settling the stored _process completion callback, leaving the 3-minute global timeout armed, which later re-fired the host callback with a spurious timeout error. triggerStopAction now routes completion through the stored process callback (clearing it first) so the timeout-settling wrapper runs; runtime-managed mode is unaffected. Integration tests: gate the pm.info.iterationCount assertion on sandbox capability - exact -1 (sentinel passthrough) with current postman-sandbox, exact Infinity once the sandbox-side transform ships; add once-guards around mocha callbacks. Add unit coverage for the custom-mode completion callback path.
Temporarily bundle postman-sandbox 6.7.2-per-vu.0 (branch feat/per-vu-variables-parallel-iterations, commit 75d4caf) as a file: dependency so CI exercises the cycles -1 -> pm.info.iterationCount Infinity transform before it ships upstream. The capability-gated integration assertion now runs its Infinity branch. Re-point to the published postman-sandbox release and drop vendor/ once the sandbox change ships (see vendor/README.md).
vendor/ is npm-ignored so the temporary vendored postman-sandbox tarball can never ship in a published runtime, and the repository system test exempts only the exact vendored file: spec from the exact-semver dependency rule. Both changes are temporary and listed in the vendor/README.md removal checklist for when the sandbox change ships upstream.
scriptonist
marked this pull request as ready for review
July 22, 2026 13:56
Codecov Report❌ Patch coverage is
❌ Your patch check has failed because the patch coverage (94.73%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## develop #1555 +/- ##
===========================================
+ Coverage 76.19% 76.38% +0.18%
===========================================
Files 50 50
Lines 4020 4052 +32
Branches 1172 1181 +9
===========================================
+ Hits 3063 3095 +32
Misses 721 721
Partials 236 236
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Per-partition (per-VU) iteration counter for
customParallelIterationsmode, sopm.info.iteration/pm.info.iterationCountandpm.variablesbehave correctly when the host (perftest) drives the parallel-iteration loop itself.Previously every host-driven loop re-ran at cursor iteration 0, so scripts always saw
pm.info.iteration === 0. Now the counter increments per loop for each VU and resets on VU recycle.Changes
Run.isCustomParallelIterationsas the mode gate;Partition.loopIteration/stoppedfields; cursorcyclesraised toMAX_SAFE_INTEGERin custom mode;runSinglePartitionseedscursor.iterationfrom the per-VU counter.coords.crauto-loop are gated on!isCustomParallelIterations, so the host (not the runtime) drives the next loop.stopSinglePartitionresetsloopIteration, re-clonesvariables, resets the cookie jar, and setsstopped = true;updatePartitionVariablesdrops late writes from in-flight scripts so a dead VU's mutations don't leak into the next VU on that slot.cycles: -1; the sandbox renders that aspm.info.iterationCount === Infinity(total is host-driven and unknowable). The eofiterationtrigger payload carries pre-rollover coords in custom mode so the host attributes completed work to the correct iteration; runtime-managed mode is unchanged.Temporary: vendored postman-sandbox
The
-1 → Infinitytransform is not yet in a published postman-sandbox release, so this PR vendors a prebuilt tarball (vendor/postman-sandbox-6.7.2-per-vu-variables.tgz, built from postman-sandboxfeat/per-vu-variables-parallel-iterations@75d4caf) as afile:dependency.vendor/is npm-ignored — the tarball cannot ship in a published runtime.test/system/repository.test.jsexempts only this exactfile:spec from the exact-semver rule.package.jsonto the published version and revert per the checklist invendor/README.md.Tests
test/unit/custom-parallel-iterations.test.js— construction invariants, mode gate, counter monotonicity, reset/late-write behavior, sandbox cursor sentinel.test/integration/runner-spec/customParallelIterations.test.js— end-to-endrun.start()→startParallelIteration()/stopParallelIteration()loops against postman-echo, including stop/restart variable isolation and a runtime-managed regression test. Thepm.info.iterationCountassertion is capability-gated: exactInfinitywith the vendored sandbox, exact-1(sentinel passthrough) with a published one — no re-edit needed when the pin changes.