Skip to content

fix(control): disable broken sandbox idle timeout - #191

Merged
osolmaz merged 2 commits into
mainfrom
fix/sandbox-idle-stopgap
Sep 9, 2026
Merged

fix(control): disable broken sandbox idle timeout#191
osolmaz merged 2 commits into
mainfrom
fix/sandbox-idle-stopgap

Conversation

@osolmaz

@osolmaz osolmaz commented Sep 9, 2026

Copy link
Copy Markdown
Member

Summary

A running Hugging Face Sandbox command can be stopped when the configured idle timeout expires because the deployed Sandbox server does not count a foreground command as active.
This change sets Harbor's native environment.kwargs.job_timeout to none for every reviewed preset and new direct job until the upstream correction is merged and deployed.
Harbor phase timeouts, Sandbox cleanup, and the Hugging Face 24-hour Job limit still bound the work.

What Changed

The change uses Harbor's existing configuration and adds no timer, keepalive loop, or replacement lifecycle state.
The restriction applies to all Hugging Face Sandbox runs without checking the model, provider, agent, or benchmark.

  • Changed all reviewed benchmark presets and the preset schema to require job_timeout: "none".
  • Changed new direct jobs and browser drafts to default to none.
  • Temporarily reject finite direct-job idle timeouts so an old value cannot reproduce the failure.
  • Added an interface notice that links to huggingface/sandbox-server#21.
  • Recorded the removal gate in the current design and launch documents: merge the correction, deploy the fixed server, and pass a foreground-command canary longer than 30 minutes.

The pinned Harbor revision is dcd0a7ac74b7bd417780d9cb27cd819c7ec82e4e.
I checked src/harbor/environments/hf_sandbox.py, including HFSandboxEnvironment.start() and _parse_idle_timeout().
Harbor already forwards job_timeout to Sandbox.create(idle_timeout=...), and the native value none disables that idle timer.
I also checked src/exec.rs and src/main.rs in huggingface/sandbox-server at a8ac9cf6e3f5622b5083f53ffaa2b4f21e9fd5b1.
The foreground /exec path is absent from the process registry that the idle watchdog checks.

Testing

The unit, type, format, build, generated-file, privacy, audit, and repository checks pass.

  • npx vitest run packages/contracts/test/contracts.test.ts packages/control-core/test/direct-config.test.ts packages/control-core/test/control.test.ts apps/control-web/test/launch.test.tsx apps/control-web/test/App.test.tsx — 95 passed
  • npm run format:check
  • npm run lint — passed with two existing warnings outside this change
  • npm run typecheck
  • npm test — 456 passed
  • npm run build
  • npm run check:generated
  • npm audit --audit-level=low
  • uv run python scripts/check_public_privacy.py .
  • uv run ruff check .
  • uv run ruff format --check .
  • uv run ty check
  • uv run pytest --cov=src/harbor_hf --cov-fail-under=85 — 46 passed, 87.98% coverage
  • uv run pip-audit
  • uv run slophammer-py dry .
  • uv run slophammer-py check .
  • uv run --directory packages/harbor-hf-agents ruff check .
  • uv run --directory packages/harbor-hf-agents ruff format --check .
  • uv run --directory packages/harbor-hf-agents ty check
  • uv run --directory packages/harbor-hf-agents pytest — 111 passed
  • git diff --check

npm run test:e2e fails locally because every page fails to render its first expected element. The unchanged main branch has the same first-test failure in this environment.
Both local linux/amd64 Docker builds reach exec format error because the Docker host is ARM64. Pull request CI must run the browser suite and both Docker builds on AMD64.
SimpleDoc reports the same four existing filename migrations on main.

Risks

A Sandbox orphan can now remain until the Hugging Face 24-hour Job limit instead of the shorter idle limit.
Normal Harbor cleanup and phase timeouts still apply, and Harbor-HF still requires Sandbox deletion after execution.

This is a temporary stopgap.
Remove it only after huggingface/sandbox-server#21 is merged, the fixed server binary is deployed, and a foreground command runs for more than 30 minutes in a canary.

osolmaz and others added 2 commits September 9, 2026 12:59
Co-Authored-By: GPT-5.6 Sol <noreply@pi.dev>
Generated-By: pi 0.85.1 (https://pi.dev)
Co-Authored-By: GPT-5.6 Sol <noreply@pi.dev>
Generated-By: pi 0.85.1 (https://pi.dev)
@osolmaz
osolmaz merged commit 18b013d into main Sep 9, 2026
@osolmaz
osolmaz deleted the fix/sandbox-idle-stopgap branch September 9, 2026 05:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant