fix(workspace): single-flight workspace updates - #1834
Conversation
Workspace updates were only single-flighted per member repo: the .update.lock guard stops two updates racing on the same repo's index, but it does not stop the post-commit hook from firing one full update_workspace pass per rebase commit. A rebase that rewrites N commits spawned N redundant whole-workspace re-indexes. Add a workspace-level lock at <workspace>/.repowise-workspace/.update.lock, acquired by update_workspace() before the parallel pass. A concurrent workspace update defers (skipped_reason=in_flight) and records a .update.pending marker per stale member so the in-flight pass rolls forward to the latest HEAD; the lock is released in a finally. Refactor update_lock.py to share the exclusive-create/liveness logic between the per-repo and workspace guards. Add regression tests for the workspace lock and the deferral/release behaviour.
|
✅ Health of changed files: 4.5 → 4.7 (+0.2) 📋 At a glance Files & modules (2)
✅ Health gate: passed 📌 Before you merge
🔎 More signals (3)🗺️ Change map flowchart LR
subgraph PR ["Changed in this PR (2 with dependents)"]
f_packages_core_src_repowise_core_update_lock_py[".../core/update_lock.py 🔥"]:::changed
f_packages_core_src_repowise_core_workspace_update_py[".../workspace/update.py 🔥"]:::changed
end
f_packages_cli_src_repowise_cli_commands_update_cmd_command_py[".../update_cmd/command.py"]
f_packages_core_src_repowise_core_update_lock_py --> f_packages_cli_src_repowise_cli_commands_update_cmd_command_py
f_packages_cli_src_repowise_cli_commands_update_cmd_workspace_py[".../update_cmd/workspace.py"]
f_packages_core_src_repowise_core_update_lock_py --> f_packages_cli_src_repowise_cli_commands_update_cmd_workspace_py
f_packages_cli_src_repowise_cli_commands_upgrade_flow_py[".../commands/upgrade_flow.py"]
f_packages_core_src_repowise_core_update_lock_py --> f_packages_cli_src_repowise_cli_commands_upgrade_flow_py
f_packages_cli_src_repowise_cli_commands_watch_cmd_py[".../commands/watch_cmd.py"]
f_packages_core_src_repowise_core_update_lock_py --> f_packages_cli_src_repowise_cli_commands_watch_cmd_py
f_packages_cli_src_repowise_cli_commands_doctor_cmd_workspace_checks_py[".../doctor_cmd/workspace_checks.py"]
f_packages_core_src_repowise_core_workspace_update_py --> f_packages_cli_src_repowise_cli_commands_doctor_cmd_workspace_checks_py
f_packages_cli_src_repowise_cli_commands_init_cmd_workspace_py[".../init_cmd/workspace.py"]
f_packages_core_src_repowise_core_workspace_update_py --> f_packages_cli_src_repowise_cli_commands_init_cmd_workspace_py
f_packages_core_src_repowise_core_workspace_update_py --> f_packages_cli_src_repowise_cli_commands_update_cmd_command_py
f_packages_cli_src_repowise_cli_commands_update_cmd_incremental_py[".../update_cmd/incremental.py"]
f_packages_core_src_repowise_core_workspace_update_py --> f_packages_cli_src_repowise_cli_commands_update_cmd_incremental_py
more(["+7 more dependents"])
PR --> more
w_packages_cli_src_repowise_cli_commands_update_cmd_command_py(["⚠️ .../update_cmd/command.py changed together 13×, not in PR"]):::warn
f_packages_core_src_repowise_core_workspace_update_py -.- w_packages_cli_src_repowise_cli_commands_update_cmd_command_py
t_tests_unit_cli_test_uninstall_cmd_py(["✅ .../cli/test_uninstall_cmd.py"]):::guard
t_tests_unit_cli_test_uninstall_cmd_py -.-> f_packages_core_src_repowise_core_update_lock_py
t_tests_unit_cli_test_update_pending_cleanup_py(["✅ .../cli/test_update_pending_cleanup.py"]):::guard
t_tests_unit_cli_test_update_pending_cleanup_py -.-> f_packages_core_src_repowise_core_workspace_update_py
classDef changed fill:#dbeafe,stroke:#1d4ed8,color:#1e3a5f
classDef warn fill:#fef3c7,stroke:#b45309,color:#78350f
classDef guard fill:#dcfce7,stroke:#15803d,color:#14532d
Solid arrows: code that imports the changed files (14 direct dependents, from the last indexed snapshot). Dashed: history/tests. 🔥 Hotspots touched (4)
1 more
🔗 Hidden coupling (1 file)
👀 Suggested reviewers @RaghavChamadiya 📊 See the full report for this PR |
Ayush7614
left a comment
There was a problem hiding this comment.
Verified: packages/core/src/repowise/core/update_lock.py:42 generalizes to _try_acquire_lock_at and adds workspace lock at .repowise-workspace/.update.lock; packages/core/src/repowise/core/workspace/update.py:1 acquires before parallel pass and defers with pending marker if in-flight (#1831). Coalesces N rebase hooks into one pass. Tests test_update_lock.py + test_update.py 50 passed. LGTM.
What
Fixes #1831 — workspace updates are now single-flighted at the workspace level, so a rebase that produces N commits no longer spawns N full workspace update passes.
Root cause
The single-flight update lock (
update_lock.py) lives at<repo>/.repowise/.update.lock— per member repo. It correctly stops two updates from racing on the same repo's index, but it does nothing to stop the post-commit hook from firing a freshrepowise updateper rebase commit, each of which runs a fullupdate_workspace()pass over every stale member. A rebase that rewrites N commits therefore triggers N redundant whole-workspace re-indexes.The fix
packages/core/src/repowise/core/update_lock.py— generalize the lock helpers behind an explicit lock-path primitive (_try_acquire_lock_at,_release_lock_at,_read_lock_at) and add a workspace lock at<workspace>/.repowise-workspace/.update.lock(update_workspace_lock/release_workspace_lock). The workspace guard inherits the exact same crash/liveness/PID-reuse semantics as the per-repo lock.packages/core/src/repowise/core/workspace/update.py—update_workspace()now acquires the workspace lock before the parallel pass. If another workspace update already holds it, the whole run defers (skipped_reason="in_flight") and writes a.update.pendingmarker per stale member so the in-flight pass rolls forward to the latest HEAD (mirroring the existing per-repo deferral). The lock is released in afinallyafter the pass.A rebase's post-commit hooks now coalesce: the first invocation wins the workspace lock and updates everything; the rest record pending heads and exit instead of re-indexing the whole workspace.
Tests
tests/unit/cli/test_update_lock.py— workspace lock path isolation, single-flight, and stale-lock recovery.tests/unit/workspace/test_update.py— a concurrentupdate_workspacedefers to an in-flight workspace update (writes pending markers, updates nothing); the lock is released after a successful pass.Verification
uv run ruff check .— all checks passeduv run pytest tests/unit/cli/test_update_lock.py tests/unit/workspace/test_update.py— 50 passed