perf(producer): stage frames via a junction before copying on Windows

[sidorovanthon]

What

Rebased onto current main and re-scoped. When linkOrCopyFrameDir can't create a plain symlink for an extracted-frames dir, it now tries a junction before falling back to the full recursive copy.

Why

main already degrades a rejected symlink (EPERM/EACCES/UNKNOWN — Windows without Developer Mode/Administrator) to cpSync(recursive). That copy is correct but, as the code comment itself notes, measurably slower — every frame of every video is physically copied into the compiled dir.

A Windows junction is a directory link that needs neither Developer Mode nor Administrator. So the common no-privilege case can stage the frame dir at link speed instead of copying it. The copy stays exactly where it was, but now only as the last resort for mounts that support no links at all (SMB/NFS/exFAT).

How

linkOrCopyFrameDir becomes three tiers:

1. Plain symlink — unchanged cheap default (POSIX, Windows with Developer Mode).
2. Junction — attempted only after a plain symlink is rejected with EPERM/EACCES/UNKNOWN. On POSIX Node ignores the "junction" type, and we only reach this tier on a privilege error POSIX never raises, so POSIX behavior is unchanged. A junction needs a local absolute target and doesn't exist on SMB/NFS/exFAT, so a capability error here (EPERM/EACCES/UNKNOWN/EINVAL/ENOSYS/EOPNOTSUPP/ENOTSUP, see isSymlinkCapabilityError) drops to tier 3. A non-capability junction error (e.g. ENOSPC) still propagates.
3. Recursive copy — the existing last resort, with the one-time degrade notice.

MaterializeFileSystem.symlinkSync gains an optional type?: "junction" parameter; the default fs-backed implementation already forwards it, and the only other staging path (materializeSymlinks, distributed plan()) is untouched.

Test plan

renderOrchestrator.test.ts — materializeExtractedFramesForCompiledDir suite:

• junction fast-path: plain symlink throws EPERM, junction succeeds → cpSync is not called, frames remapped exactly as in the symlink happy path.

• copy only when both fail: plain symlink and junction both throw EPERM → plain then junction attempted (asserted via the recorded type argument), then cpSync(recursive).

• UNKNOWN variant unchanged (both link attempts fail → copy).

• non-capability junction error (ENOSPC from the junction attempt) → rethrown, no copy.

• Unit tests added/updated (renderOrchestrator.test.ts: 145 passed / 0 failed via vitest)

• bunx oxlint / bunx oxfmt clean; tsc --noEmit clean; lefthook pre-commit (lint/format/fallow/typecheck) green

• Documentation updated (if applicable) — behavior/perf-only change, no docs affected

🤖 Generated with Claude Code

[miguel-heygen]

@sidorovanthon rebase main pls, after that I'll review again

[sidorovanthon]

@miguel-heygen rebased onto current main and re-scoped the PR.

The original diff predated your linkOrCopyFrameDir symlink→copy fallback and largely duplicated it. On a fresh main that fallback is already there, so I dropped the duplication and kept only the part main doesn't cover: a junction attempt between the symlink and the copy.

A Windows junction needs neither Developer Mode nor Administrator, so the common no-privilege case now stages the frame dir at link speed instead of copying every frame (your own comment flags the copy as "measurably slower"). The copy stays as the last resort for mounts with no link support at all (SMB/NFS/exFAT), where even a junction fails with a capability error; a non-capability junction error (e.g. ENOSPC) still propagates. POSIX behavior is unchanged — that tier is only reached on a privilege error POSIX never raises.

Title/description updated to match. Ready for another look.

[miguel-heygen]

The junction implementation is currently bypassed by the exact local Windows path this PR is intended to optimize. renderOrchestrator.ts passes materializeSymlinks: shouldCopyExtractedFrames(process.platform), and shouldCopyExtractedFrames("win32") is true; stageExtractedFrameDirOnce(..., true) calls cpSync(recursive) directly and never reaches linkOrCopyFrameDir, so neither the plain-symlink nor junction tier runs. A fresh 0.7.58 Windows field report reproduced the consequence with one 305.453s video: all 9,164 JPEGs were extracted, then the process exited while materializing the cache into compiledDir. Please close the class in this PR by routing local Windows staging through the link/junction/copy ladder (while keeping distributed-plan materialization as a real copy), and add an orchestrator-level regression proving the actual win32 local-render call does not request eager copy. The helper-only tests currently pass while the production caller still selects the old path.

[sidorovanthon]

@miguel-heygen thanks - you were exactly right, the junction ladder was dead code on the very platform it targeted. Rebased onto current main and closed the class in this push (33e0789b0).

Root cause. The local render invoked the extract stage with materializeSymlinks: shouldCopyExtractedFrames(process.platform), and shouldCopyExtractedFrames("win32") is true. stageExtractedFrameDirOnce(..., true) calls cpSync(recursive) directly, so neither the plain-symlink nor the junction tier in linkOrCopyFrameDir ever ran. The helper tests passed while the production caller still selected the old path - precisely as you noted.

Fix:

1. renderOrchestrator.ts - the local in-process render now passes materializeSymlinks: false, routing win32 through the symlink → junction → copy ladder. A junction needs neither Developer Mode nor admin, so the common no-privilege case stages at link speed; the recursive copy stays as the ladder's last resort (SMB/NFS/exFAT).
2. Deleted shouldCopyExtractedFrames - the platform → copy coupling is exactly what bypassed the ladder, and the junction tier now owns the win32 privilege case. Removing it makes the bypass structurally impossible to reintroduce; its JSDoc on materializeSymlinks now spells out "distributed plan() only — do not recouple to process.platform."
3. plan.ts unchanged - distributed staging keeps materializeSymlinks: true (a planDir must be self-contained; symlinks/junctions don't survive S3/GCS).
4. Orchestrator-level regression - added at runExtractVideosStage (the stage the orchestrator actually calls, not a helper): the local path does not request an eager copy (flag false or omitted → materializeExtractedFramesForCompiledDir reaches the ladder), and only the distributed path does. Existing junction fast-path helper test already proves false + EPERM → junction, no copy - so the full chain is covered.

Behaviorally a strict improvement: the ladder's tier 3 is the same cpSync(recursive), so the win32 worst case is identical to today, best case is a privilege-free junction.

Verification (Windows): producer unit lane 306 passed; renderOrchestrator.test.ts 143/143 (junction ladder intact); extractVideosStage.test.ts 7/7 incl. 3 new; tsc --noEmit, oxlint, oxfmt, test-classification all clean. (One unrelated pre-existing failure in hyperframeRuntimeLoader.test.ts - a Windows path-separator assertion - reproduces without this change.)

Ready for another look.