feat(parallel): per-VU iteration data for customParallelIterations - #1559
Open
saialekhya-001 wants to merge 15 commits into
Open
feat(parallel): per-VU iteration data for customParallelIterations#1559saialekhya-001 wants to merge 15 commits into
saialekhya-001 wants to merge 15 commits into
Conversation
- Add Run.isCustomParallelIterations as the single source of truth for the mode gate (Change 0) - Initialize Partition.loopIteration=0 and Partition.stopped=false (Change 1, 6c prep) - Raise cursor.cycles to MAX_SAFE_INTEGER in custom mode so seek doesn't trip the bounds guard (Change 2) - Increment partition.cursor.iteration on each runSinglePartition call in custom mode (Change 3) - Expose Partition.resetVariables() public helper for the full-fresh- on-reuse contract landing in a later change (Change 6b prep) Tests: 12 new unit cases in test/unit/custom-parallel-iterations.test.js covering construction-time invariants, the Run gating helper, and multi-loop counter monotonicity. All 449 unit tests pass.
- Change 4: gate the end-of-partition early-return at parallel.command:103 on !isCustomParallelIterations. Without this, loop 2+ would short- circuit (coords.iteration === startIndex + partitionCycles is true every loop in custom mode because startIndex=0, partitionCycles=1) and the runtime would skip every item. - Change 5: in the coords.cr block, after firing iteration trigger, early-return next() in custom mode. The host (perftest) drives the next loop via startParallelIteration; the runtime must not auto-loop in parallel. beforeIteration also no longer fires here in custom mode — startParallelIteration's own parallel-command invocation carries start:true and fires it. Tests: 4 new unit cases on the parallel processor (with mocked context) — Change 4 in custom + runtime-managed modes, Change 5 in custom + runtime-managed modes. 453 passing (was 449).
- Change 6: stopSinglePartition resets partition.loopIteration to 0, re-clones partition.variables via the resetVariables() helper, and sets partition.stopped=true. Recycled partitions behave as brand-new VUs — pm.info.iteration === 0 with a fresh pm.variables scope. Gated on isCustomParallelIterations so runtime-managed mode is untouched. - Change 6c: updatePartitionVariables drops writes when partition.stopped is true. Guards the late-write race where a script in flight at stopSinglePartition time finishes after the variables re-clone, otherwise leaking the dead VU's pm.variables mutations into the next VU's fresh scope. Tests: 9 new unit cases — loopIteration reset, stopped flag toggle, variables re-clone, flag clear on next runSinglePartition, full-cycle regression (stop→start→counter=0), runtime-managed-mode non-mutation, late-write drop happy + race + missing-partition paths. 462 unit tests passing (was 453).
…7, 9)
- Change 7: at the host.execute call site in event.command.js,
transform cursor.cycles to -1 when isCustomParallelIterations is
true. -1 is the wire sentinel that survives JSON/structured-clone
encoding (Infinity would JSON.stringify to null,
Number.MAX_SAFE_INTEGER would leak to pm.info.iterationCount).
Sandbox-side rendering of -1 -> Infinity is the matching change in
postman-sandbox/lib/sandbox/pmapi.js. Helper factored out as
applySandboxCursorSentinel and exposed for unit tests.
- Change 9: in the eof branch of parallel.command, fire the iteration
trigger with payload.coords ('loop just completed') in custom mode
rather than the post-rollover snapshot. Gated to custom mode to
preserve existing Newman/desktop semantics — flipping the global
contract risks silent downstream regressions we can't audit from
inside postman-runtime.
Tests: 5 new unit cases — sentinel transform happy + non-mutation +
no-op paths, eof trigger custom + runtime-managed payloads.
467 unit tests passing (was 462).
…ions Self-contained driver that exercises the perftest invocation pattern: runner.run() -> run.start() -> startParallelIteration() loop until maxLoops -> abort. Verifies the full chain: runtime drives the loop -> host.execute hands the (sentinel-transformed) cursor to the sandbox -> script reads pm.info.iteration + pm.info.iterationCount. Tests cover: - T1: first loop sees iteration=0 (via runtime trigger cursor) - T3: monotonic iteration across 3 loops ([0, 1, 2]) - T2/loop-2-runs-all-items: no skip-everything regression - T4/T5 trigger counts: iteration + beforeIteration fire once per loop - T6/T6b: stop+restart resets cursor.iteration to 0 AND re-clones pm.variables (verified via in-script marker) - T10 regression: maxConcurrency=2 mode still completes normally - T12: pm.info.iterationCount === Infinity end-to-end (cross-repo wire contract verified) Note: T12 requires the matching postman-sandbox change (cycles===-1 rendered as Infinity in pmapi.js). Until the sandbox version pinned in package.json is bumped, T12 fails locally — run 'node npm/cache.js' in the local postman-sandbox repo and copy .cache/bootcode.js into node_modules/postman-sandbox/.cache/ to verify locally before release.
Lint: use object spread instead of Object.assign, repo noop convention for empty test callbacks, and wrap/format long lines per style rules. The per-partition cookie jar unit test mocked only the raw customParallelIterations option, so after stopSinglePartition migrated to the derived isCustomParallelIterations flag the custom-mode branch never executed. Mock now mirrors real Run construction; assertions unchanged.
Custom-mode runs completed via triggerStopAction -> triggers(null) without settling the stored _process completion callback, leaving the 3-minute global timeout armed, which later re-fired the host callback with a spurious timeout error. triggerStopAction now routes completion through the stored process callback (clearing it first) so the timeout-settling wrapper runs; runtime-managed mode is unaffected. Integration tests: gate the pm.info.iterationCount assertion on sandbox capability - exact -1 (sentinel passthrough) with current postman-sandbox, exact Infinity once the sandbox-side transform ships; add once-guards around mocha callbacks. Add unit coverage for the custom-mode completion callback path.
Temporarily bundle postman-sandbox 6.7.2-per-vu.0 (branch feat/per-vu-variables-parallel-iterations, commit 75d4caf) as a file: dependency so CI exercises the cycles -1 -> pm.info.iterationCount Infinity transform before it ships upstream. The capability-gated integration assertion now runs its Infinity branch. Re-point to the published postman-sandbox release and drop vendor/ once the sandbox change ships (see vendor/README.md).
vendor/ is npm-ignored so the temporary vendored postman-sandbox tarball can never ship in a published runtime, and the repository system test exempts only the exact vendored file: spec from the exact-semver dependency rule. Both changes are temporary and listed in the vendor/README.md removal checklist for when the sandbox change ships upstream.
…vu fork postman-sandbox 6.7.3 now ships both the per-VU sentinel (cycles -1 -> pm.info.iterationCount: Infinity, commit 75d4caf) and the pm.datasets streaming pull-protocol, so the temporary vendored `6.7.2-per-vu.0` tarball is obsolete. It also lacked the streaming support, which caused the 4 datasets.test.js failures once this branch picked up develop's streaming tests (#1553). - point postman-sandbox at published 6.7.3, regenerate lockfile - drop vendor/ (tgz + README) and the .npmignore vendor/ exclusion - revert the repository.test.js exact-semver exemption for the file: spec Validated locally: datasets.test.js 22 passing, customParallelIterations integration 8 passing (+30 unit), system + lint green. Co-Authored-By: Claude <noreply@anthropic.com>
In customParallelIterations mode the host injects each VU's data row via
startParallelIteration(index, row). That row was assigned to
partition.variables._variables and sent over the sandbox boundary as
`_variables`, so it surfaced only as pm.variables / {{key}} — pm.iterationData
(which the sandbox builds from the `data` field) stayed empty.
Route the per-VU row into the `data` payload field: partition-manager stores it
as partition.iterationData in custom mode (leaving _variables as the pristine
per-VU script scope), and parallel.command sources the item payload `data` from
partition.iterationData. The sandbox layers iterationData into _variables, so
{{key}} and pm.variables.get() keep resolving and pm.iterationData.get() now
works. No sandbox change required.
This is the routing the @postman/performance-test lib depends on (it was only
ever in the vendored pr1555.0 tarball, never committed); required for datasets/
datafiles as iteration data in parallel perf runs.
Co-Authored-By: Claude <noreply@anthropic.com>
The previous commit routed the per-VU payload to iterationData INSTEAD of
_variables in customParallelIterations mode, which broke perPartitionCookieJar
(the per-partition {{marker}} no longer resolved — 7 failing). Seed both: keep
_variables so {{key}} / pm.variables.get() and per-partition markers resolve
(the original, proven behavior), and additionally set iterationData so the row
surfaces as pm.iterationData.
Co-Authored-By: Claude <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #1559 +/- ##
===========================================
+ Coverage 77.54% 77.83% +0.28%
===========================================
Files 51 51
Lines 4338 4380 +42
Branches 1223 1238 +15
===========================================
+ Hits 3364 3409 +45
+ Misses 731 729 -2
+ Partials 243 242 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
scriptonist
reviewed
Aug 5, 2026
…ard late writes Addresses two correctness issues in customParallelIterations reuse: 1. Variable leak across VU reuse. Completed scripts wrote their _variables back to the run-global state._variables unconditionally; Partition#_cloneVariables clones from that scope on reuse, so a recycled VU inherited the previous occupant's mutations. Skip the run-global write in parallel mode (event.command.js) — the per-VU scope is persisted via updatePartitionVariables, keeping state._variables the pristine baseline new/recycled VUs clone from. 2. Completion race on immediate slot reuse. `stopped` is cleared before the new VU's work, so an in-flight completion from the previous occupant could overwrite the new occupant. Tag each partition with a generation (bumped on stopSinglePartition), stamp it onto the queued execution (coords.partitionGeneration), and drop writes in updatePartitionVariables whose generation no longer matches. Tests: unit coverage for the generation guard, custom-mode reuse (iterationData + _variables routing, cursor.iteration bump), and the poolFinished abort/error branches; an integration regression proving a recycled VU sees a pristine scope (fails without fix 1). Full lint + unit green; targeted integration green. Co-Authored-By: Claude <noreply@anthropic.com>
Lint-only follow-up to the per-VU reuse fix (CI Lint job):
- parallel.command.js: use `{ ...coords, partitionGeneration }` instead of Object.assign
- partition-manager.js: document the new `generation` param on updatePartitionVariables
Co-Authored-By: Claude <noreply@anthropic.com>
Author
CI note:
|
Author
Behaviour in
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds per-VU iteration data + counter for the
customParallelIterationsrun mode, so each virtual user (VU) sees its own iteration index and its own row viapm.iterationDatawhen iterations run in parallel across partitions. This is the routing the@postman/performance-testlibrary depends on for datasets/datafiles as iteration data in parallel perf runs.Changes
partition-manager.js,partition.js,run.js).pm.iterationData— in custom mode the host-injected per-VU row is seeded into BOTH the VU's_variables(so{{key}}/pm.variables.get()and per-partition markers resolve) ANDpartition.iterationData, whichparallel.commandsources the item payloaddatafrom →pm.iterationData.get()works. No sandbox change required.6.7.3(has thecycles -1 → iterationCount: Infinitysentinel +pm.datasetsstreaming); the temporary vendored6.7.2-per-vufork was dropped.custom-parallel-iterations,partition-manager,per-partition-cookie-jar) + integration (customParallelIterations.test.js,datasets.test.js). Full unit + integration suite green locally.Generated with Claude Code