Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 6 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,6 @@ the home, and it assigns ownership through the resulting descriptor rather than
| `SBX_PORT` | `8000` | listen port (the client uses 49983 to keep common dev ports free) |
| `SBX_TOKEN` | **required** | all endpoints except `/health` require this value in the `X-Sandbox-Token` header (constant-time compare); removed from the env before any child process spawns. The server refuses to start without it, unless launched with `--allow-no-auth` (local development only — it is an argv flag, not an env var, so a Job's user-supplied env can never set it) |
| `SBX_IDLE_TIMEOUT` | unset | seconds of inactivity (no authed request, no running process) before clean exit |
| `SBX_COMPAT_HOST_TOKEN` | `1` | host mode: whether the host token is still accepted on per-sandbox routes, for clients that predate per-sandbox tokens. Set to `0` to require scoped tokens |
| `SBX_CAPACITY` | `64` | host mode: max concurrent sandboxes. Refuses to start if unparseable (it used to fall back to unlimited) |
| `SBX_MAX_CONNECTIONS` | `512` | max concurrent connections; past it the server answers 503 without spawning a worker |
| `SBX_MIN_LANDLOCK_ABI` | `6` | host mode: minimum Landlock ABI to start with. 4 adds TCP-bind denial, 6 adds abstract-socket scoping — both are part of the documented model, so the default requires them. Lower it to accept a reduced set (`/health` reports what is in force) |
Expand Down Expand Up @@ -118,12 +117,11 @@ In host mode there are two kinds of credential:
hand to whoever operates a single sandbox, including into a browser or WebSocket client via
the port proxy.

The host token is *also* accepted on per-sandbox routes while `SBX_COMPAT_HOST_TOKEN=1` (the
default), so clients that predate per-sandbox tokens keep working when this binary is
published under them — every job fetches the binary fresh, so a hard break would break every
old client at once. That is a management credential having authority over the sandboxes it
created, not a sandbox credential reaching a sibling. Set `SBX_COMPAT_HOST_TOKEN=0` to close
it once clients have upgraded.
Per-sandbox routes, including proxies, require that sandbox's capability token. The host
management token is never accepted there. The former SBX_COMPAT_HOST_TOKEN setting has been
removed; setting it cannot restore the fallback. This is a breaking authentication change
(protocol 3): upgrade clients to use scoped tokens before deploying this server. Dedicated
sandboxes and host management/token recovery retain their existing credentials.

### Known limitations

Expand All @@ -139,8 +137,7 @@ sandbox, a real VM) for mutually distrusting code.
- **Shared channels remain:** outbound TCP, loopback access to the control server, UDP,
kernel IPC, and readable process-list metadata. GPU isolation is untested in host mode.
- **The host token remains a management capability.** It can recover every sandbox token;
compatibility mode also accepts it on scoped routes. Use `SBX_COMPAT_HOST_TOKEN=0` to
require per-sandbox tokens there.
keep it private even though it cannot be used directly on scoped routes.
- **A hijacked proxy connection is authenticated and routed only once**, then bytes are
spliced until EOF. A second HTTP request written on that connection reaches the first
backend without new routing, depending on the upstream proxy's behaviour.
Expand Down
6 changes: 4 additions & 2 deletions scripts/auth-routes-regression.sh
Original file line number Diff line number Diff line change
Expand Up @@ -72,10 +72,12 @@ expect 404 "POST /v1/files/mkdir" -H "$A" -X POST "$U/v1/files/mkdir?p
expect 404 "ANY /v1/proxy/<port>" -H "$A" "$U/v1/proxy/22/"

say "host mode: its own surface still works"
T=$(curl -s -H "$A" "$U/v1/sandboxes/$S/token" | sed 's/.*"token":"\([^" ]*\)".*/\1/')
SCOPED="X-Sandbox-Token: $T"
expect 200 "GET /v1/sandboxes" -H "$A" "$U/v1/sandboxes"
expect 200 "POST /v1/sandboxes/<id>/exec" -H "$A" -X POST "$U/v1/sandboxes/$S/exec" -d '{"cmd":"id"}'
expect 200 "POST /v1/sandboxes/<id>/exec" -H "$SCOPED" -X POST "$U/v1/sandboxes/$S/exec" -d '{"cmd":"id"}'
grep -q 'uid=20' /tmp/body && pass "scoped exec runs as the sandbox uid" || fail "scoped exec uid: $(head -c 120 /tmp/body)"
expect 200 "PUT /v1/sandboxes/<id>/files/write" -H "$A" -X PUT "$U/v1/sandboxes/$S/files/write?path=f" -d 'x'
expect 200 "PUT /v1/sandboxes/<id>/files/write" -H "$SCOPED" -X PUT "$U/v1/sandboxes/$S/files/write?path=f" -d 'x'
expect 200 "GET /v1/health without a token" "$U/health"

say "host mode: a bad or missing token is refused"
Expand Down
12 changes: 6 additions & 6 deletions scripts/lifecycle-race-regression.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ import time

port = int(os.environ["PORT"])

def request(method, path, body=None):
def request(method, path, body=None, *, token="test"):
conn = http.client.HTTPConnection("127.0.0.1", port, timeout=10)
conn.request(method, path, body=json.dumps(body) if body else None,
headers={"X-Sandbox-Token": "test"})
headers={"X-Sandbox-Token": token})
response = conn.getresponse()
data = response.read()
conn.close()
Expand All @@ -43,19 +43,19 @@ with socket.create_connection(("127.0.0.1", port), timeout=10) as conn:
f"Content-Length: {len(body)}\r\nConnection: close\r\n\r\n").encode()
conn.sendall(head + body[:1])
time.sleep(.3) # handler has acquired the entry and is waiting for its body
request("DELETE", f"/v1/sandboxes/{old['id']}")
request("DELETE", f"/v1/sandboxes/{old['id']}", token=old["token"])
new = request("POST", "/v1/sandboxes", {"count": 1})["sandboxes"][0]
assert old["uid"] == new["uid"], "regression must exercise uid reuse"
conn.sendall(body[1:])
response = http.client.HTTPResponse(conn)
response.begin()
payload = response.read()
assert response.status == 400 and b"sandbox has been deleted" in payload, payload
proc = request("POST", f"/v1/sandboxes/{new['id']}/processes", {"cmd": "sleep 5 & exit 0"})
proc = request("POST", f"/v1/sandboxes/{new['id']}/processes", {"cmd": "sleep 5 & exit 0"}, token=new["token"])
time.sleep(.3) # leader is reaped, but its descendant still holds stdout open
result = request("DELETE", f"/v1/sandboxes/{new['id']}/processes/{proc['id']}")
result = request("DELETE", f"/v1/sandboxes/{new['id']}/processes/{proc['id']}", token=new["token"])
assert result["killed"] is False, "signalled a reaped PID while its exit event was pending"
print("PASS: delayed output does not leave a reaped PID signalable")
request("DELETE", f"/v1/sandboxes/{new['id']}")
request("DELETE", f"/v1/sandboxes/{new['id']}", token=new["token"])
print("PASS: delayed exec refused after deletion and uid reuse")
PY
4 changes: 2 additions & 2 deletions scripts/process-supervision-regression.sh
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ curl -s -H "X-Sandbox-Token: $T" "$U/v1/sandboxes/$S/processes" >/tmp/plist
grep -q '"tag":"keeper"' /tmp/plist && pass "the running process is still listed" || fail "a running process was evicted"

say "teardown reports what it actually achieved"
out=$(curl -s -o /tmp/body -w '%{http_code}' -H "$A" -X DELETE "$U/v1/sandboxes/$S")
out=$(curl -s -o /tmp/body -w '%{http_code}' -H "X-Sandbox-Token: $T" -X DELETE "$U/v1/sandboxes/$S")
[ "$out" = 200 ] && pass "a clean delete answers 200" || fail "delete answered $out: $(cat /tmp/body)"
grep -q '"deleted":true' /tmp/body && pass "and says so" || fail "delete body: $(cat /tmp/body)"
# The detached survivor must be gone: the uid sweep catches what a group kill misses.
Expand All @@ -134,7 +134,7 @@ curl -s -m 20 -H "X-Sandbox-Token: $T2" -X POST "$U/v1/sandboxes/$S2/exec" \
-d '{"cmd":"setsid sleep 300 >/dev/null 2>&1 & exit 0"}' >/dev/null
sleep 1
uid=$(sed 's/.*"uid":\([0-9]*\).*/\1/' /tmp/created2)
curl -s -o /dev/null -H "$A" -X DELETE "$U/v1/sandboxes/$S2"
curl -s -o /dev/null -H "X-Sandbox-Token: $T2" -X DELETE "$U/v1/sandboxes/$S2"
sleep 1
if ps -eo uid= 2>/dev/null | tr -d ' ' | grep -qx "$uid"; then
fail "a setsid descendant survived the sandbox delete"
Expand Down
62 changes: 33 additions & 29 deletions scripts/symlink-regression.sh
Original file line number Diff line number Diff line change
Expand Up @@ -64,11 +64,15 @@ B=$(curl -s -H "$AUTH" -X POST "$BASE/v1/sandboxes" -d '{"count":1}' |
sed 's/.*"id":"\([^"]*\)".*/\1/')
[ -n "$A" ] && [ -n "$B" ] || { echo "could not create two sandboxes"; exit 1; }
echo " sandbox A=$A B=$B"
A_TOKEN=$(curl -s -H "$AUTH" "$BASE/v1/sandboxes/$A/token" | sed 's/.*"token":"\([^" ]*\)".*/\1/')
B_TOKEN=$(curl -s -H "$AUTH" "$BASE/v1/sandboxes/$B/token" | sed 's/.*"token":"\([^" ]*\)".*/\1/')
AUTH_A="X-Sandbox-Token: $A_TOKEN"
AUTH_B="X-Sandbox-Token: $B_TOKEN"
HOME_A=/sbx/homes/$A
HOME_B=/sbx/homes/$B

# B has something worth stealing, created by B's own code so it is B's to lose.
curl -s -H "$AUTH" -X POST "$BASE/v1/sandboxes/$B/exec" \
curl -s -H "$AUTH_B" -X POST "$BASE/v1/sandboxes/$B/exec" \
-d '{"cmd":"echo b-secret > $HOME/secret"}' >/dev/null

# B also exposes a service on its own proxy socket. Started up front so the
Expand All @@ -86,22 +90,22 @@ while True:
conn.sendall(b"HTTP/1.1 200 OK\r\nContent-Length: 8\r\n\r\nB-SECRET")
conn.close()
SERVICE
curl -s -H "$AUTH" -X PUT "$BASE/v1/sandboxes/$B/files/write?path=service.py" \
curl -s -H "$AUTH_B" -X PUT "$BASE/v1/sandboxes/$B/files/write?path=service.py" \
--data-binary @/tmp/service.py >/dev/null
curl -s -H "$AUTH" -X POST "$BASE/v1/sandboxes/$B/processes" \
curl -s -H "$AUTH_B" -X POST "$BASE/v1/sandboxes/$B/processes" \
-d '{"cmd":["python3","service.py"]}' >/dev/null

say "ordinary file operations still work"
allows "write" -H "$AUTH" -X PUT "$BASE/v1/sandboxes/$A/files/write?path=data/hello.txt" -d 'hello'
allows "read" -H "$AUTH" "$BASE/v1/sandboxes/$A/files/read?path=data/hello.txt"
allows "write" -H "$AUTH_A" -X PUT "$BASE/v1/sandboxes/$A/files/write?path=data/hello.txt" -d 'hello'
allows "read" -H "$AUTH_A" "$BASE/v1/sandboxes/$A/files/read?path=data/hello.txt"
[ "$(cat /tmp/body)" = "hello" ] && pass "read returned what was written" || fail "read content mismatch"
allows "list" -H "$AUTH" "$BASE/v1/sandboxes/$A/files/list?path=data"
allows "stat" -H "$AUTH" "$BASE/v1/sandboxes/$A/files/stat?path=data/hello.txt"
allows "mkdir" -H "$AUTH" -X POST "$BASE/v1/sandboxes/$A/files/mkdir?path=nested/deep"
allows "delete" -H "$AUTH" -X DELETE "$BASE/v1/sandboxes/$A/files/delete?path=data/hello.txt"
allows "list" -H "$AUTH_A" "$BASE/v1/sandboxes/$A/files/list?path=data"
allows "stat" -H "$AUTH_A" "$BASE/v1/sandboxes/$A/files/stat?path=data/hello.txt"
allows "mkdir" -H "$AUTH_A" -X POST "$BASE/v1/sandboxes/$A/files/mkdir?path=nested/deep"
allows "delete" -H "$AUTH_A" -X DELETE "$BASE/v1/sandboxes/$A/files/delete?path=data/hello.txt"
# Files written through the API must be usable by the sandbox's own (unprivileged) code.
curl -s -H "$AUTH" -X PUT "$BASE/v1/sandboxes/$A/files/write?path=owned.txt" -d 'x' >/dev/null
if curl -s -H "$AUTH" -X POST "$BASE/v1/sandboxes/$A/exec" \
curl -s -H "$AUTH_A" -X PUT "$BASE/v1/sandboxes/$A/files/write?path=owned.txt" -d 'x' >/dev/null
if curl -s -H "$AUTH_A" -X POST "$BASE/v1/sandboxes/$A/exec" \
-d '{"cmd":"cat $HOME/owned.txt"}' | grep -q '"data":"x"'; then
pass "API-written files are owned by the sandbox uid"
else
Expand All @@ -113,30 +117,30 @@ plant="ln -s /etc/passwd \$HOME/host-file"
plant="$plant; ln -s $HOME_B \$HOME/sibling-home"
plant="$plant; ln -s $HOME_B/secret \$HOME/sibling-file"
plant="$plant; mkdir -p \$HOME/via; ln -s $HOME_B \$HOME/via/link"
curl -s -H "$AUTH" -X POST "$BASE/v1/sandboxes/$A/exec" -d "{\"cmd\":\"$plant\"}" >/dev/null
curl -s -H "$AUTH_A" -X POST "$BASE/v1/sandboxes/$A/exec" -d "{\"cmd\":\"$plant\"}" >/dev/null
# If planting fails, every "refused" below would pass vacuously — so assert it worked.
for link in host-file sibling-home sibling-file via/link; do
[ -L "$HOME_A/$link" ] || fail "could not plant symlink $link — the escapes below are vacuous"
done
[ -L "$HOME_A/host-file" ] && pass "symlinks planted by the sandbox's own code"

say "H-02a: the root file API must not follow them"
refuses "read a host file through a final symlink" -H "$AUTH" "$BASE/v1/sandboxes/$A/files/read?path=host-file"
refuses "read a sibling's file" -H "$AUTH" "$BASE/v1/sandboxes/$A/files/read?path=sibling-file"
refuses "list a sibling's home" -H "$AUTH" "$BASE/v1/sandboxes/$A/files/list?path=sibling-home"
refuses "write through a final symlink" -H "$AUTH" -X PUT "$BASE/v1/sandboxes/$A/files/write?path=sibling-file" -d 'pwned'
refuses "write through an intermediate symlink" -H "$AUTH" -X PUT "$BASE/v1/sandboxes/$A/files/write?path=via/link/planted" -d 'pwned'
refuses "read via an intermediate symlink" -H "$AUTH" "$BASE/v1/sandboxes/$A/files/read?path=via/link/secret"
refuses "mkdir through an intermediate symlink" -H "$AUTH" -X POST "$BASE/v1/sandboxes/$A/files/mkdir?path=via/link/planted"
refuses "read a host file through a final symlink" -H "$AUTH_A" "$BASE/v1/sandboxes/$A/files/read?path=host-file"
refuses "read a sibling's file" -H "$AUTH_A" "$BASE/v1/sandboxes/$A/files/read?path=sibling-file"
refuses "list a sibling's home" -H "$AUTH_A" "$BASE/v1/sandboxes/$A/files/list?path=sibling-home"
refuses "write through a final symlink" -H "$AUTH_A" -X PUT "$BASE/v1/sandboxes/$A/files/write?path=sibling-file" -d 'pwned'
refuses "write through an intermediate symlink" -H "$AUTH_A" -X PUT "$BASE/v1/sandboxes/$A/files/write?path=via/link/planted" -d 'pwned'
refuses "read via an intermediate symlink" -H "$AUTH_A" "$BASE/v1/sandboxes/$A/files/read?path=via/link/secret"
refuses "mkdir through an intermediate symlink" -H "$AUTH_A" -X POST "$BASE/v1/sandboxes/$A/files/mkdir?path=via/link/planted"

# Deleting the link must remove the link, never what it points at.
allows "delete the symlink itself" -H "$AUTH" -X DELETE "$BASE/v1/sandboxes/$A/files/delete?path=sibling-home"
allows "delete the symlink itself" -H "$AUTH_A" -X DELETE "$BASE/v1/sandboxes/$A/files/delete?path=sibling-home"
[ -f "$HOME_B/secret" ] && pass "B's file survived the delete" || fail "B's file was deleted"
[ "$(cat "$HOME_B/secret")" = "b-secret" ] && pass "B's file was not modified" || fail "B's file was modified"
[ -f /etc/passwd ] && pass "/etc/passwd untouched" || fail "/etc/passwd damaged"

# stat must describe the link, not its target.
curl -s -H "$AUTH" "$BASE/v1/sandboxes/$A/files/stat?path=sibling-file" >/tmp/body
curl -s -H "$AUTH_A" "$BASE/v1/sandboxes/$A/files/stat?path=sibling-file" >/tmp/body
grep -q '"type":"symlink"' /tmp/body && pass "stat reports the link" || fail "stat followed the link: $(cat /tmp/body)"

say "H-02b: the root port proxy must not follow a socket symlink"
Expand All @@ -145,7 +149,7 @@ say "H-02b: the root port proxy must not follow a socket symlink"
reached=no
i=0
while [ "$i" -lt 10 ]; do
if curl -s -m 5 -H "$AUTH" "$BASE/v1/sandboxes/$B/proxy/9000/" | grep -q B-SECRET; then
if curl -s -m 5 -H "$AUTH_B" "$BASE/v1/sandboxes/$B/proxy/9000/" | grep -q B-SECRET; then
reached=yes
break
fi
Expand All @@ -156,25 +160,25 @@ if [ "$reached" = yes ]; then
pass "B reaches its own service through the proxy"
else
fail "B cannot reach its own service — the refusal below would prove nothing"
echo " processes: $(curl -s -H "$AUTH" "$BASE/v1/sandboxes/$B/processes")"
echo " processes: $(curl -s -H "$AUTH_B" "$BASE/v1/sandboxes/$B/processes")"
fi

# A points its own proxy socket name at B's socket.
curl -s -H "$AUTH" -X POST "$BASE/v1/sandboxes/$A/exec" \
curl -s -H "$AUTH_A" -X POST "$BASE/v1/sandboxes/$A/exec" \
-d "{\"cmd\":\"ln -s $HOME_B/.sbx/proxy/9000.sock \$SBX_PROXY_DIR/9000.sock\"}" >/dev/null
[ -L "$HOME_A/.sbx/proxy/9000.sock" ] || fail "could not plant the socket symlink — the check below is vacuous"
out=$(curl -s -m 5 -H "$AUTH" "$BASE/v1/sandboxes/$A/proxy/9000/" || true)
out=$(curl -s -m 5 -H "$AUTH_A" "$BASE/v1/sandboxes/$A/proxy/9000/" || true)
case "$out" in
*B-SECRET*) fail "A reached B's service through a socket symlink" ;;
*) pass "A's socket symlink was refused" ;;
esac

# A regular file, and a bogus port, must not be accepted either.
curl -s -H "$AUTH" -X POST "$BASE/v1/sandboxes/$A/exec" \
curl -s -H "$AUTH_A" -X POST "$BASE/v1/sandboxes/$A/exec" \
-d '{"cmd":"rm -f $SBX_PROXY_DIR/9001.sock; echo x > $SBX_PROXY_DIR/9001.sock"}' >/dev/null
refuses "a regular file as a socket" -m 5 -H "$AUTH" "$BASE/v1/sandboxes/$A/proxy/9001/"
refuses "a non-numeric port" -m 5 -H "$AUTH" "$BASE/v1/sandboxes/$A/proxy/..%2F..%2Fetc/"
refuses "port 0" -m 5 -H "$AUTH" "$BASE/v1/sandboxes/$A/proxy/0/"
refuses "a regular file as a socket" -m 5 -H "$AUTH_A" "$BASE/v1/sandboxes/$A/proxy/9001/"
refuses "a non-numeric port" -m 5 -H "$AUTH_A" "$BASE/v1/sandboxes/$A/proxy/..%2F..%2Fetc/"
refuses "port 0" -m 5 -H "$AUTH_A" "$BASE/v1/sandboxes/$A/proxy/0/"

say "result"
if [ "$failures" -eq 0 ]; then
Expand Down
23 changes: 8 additions & 15 deletions scripts/token-scope-regression.sh
Original file line number Diff line number Diff line change
Expand Up @@ -96,21 +96,14 @@ say "an unknown token is refused"
expect 403 "not-a-real-token" "garbage token" -X POST "$U/v1/sandboxes/$A_ID/exec" -d '{"cmd":"id"}'
expect 403 "" "empty token" -X POST "$U/v1/sandboxes/$A_ID/exec" -d '{"cmd":"id"}'

say "the compat window (host token on scoped routes) is on by default"
expect 200 "$HOST_TOKEN" "host token on a scoped route" -X POST "$U/v1/sandboxes/$A_ID/exec" -d '{"cmd":"echo hi"}'
kill $server 2>/dev/null || true
wait $server 2>/dev/null || true

say "SBX_COMPAT_HOST_TOKEN=0 closes it"
SBX_PORT=$PORT SBX_TOKEN=$HOST_TOKEN SBX_HOST_MODE=1 SBX_COMPAT_HOST_TOKEN=0 "$BIN" &
server=$!
sleep 1
curl -s -H "X-Sandbox-Token: $HOST_TOKEN" -X POST "$U/v1/sandboxes" -d '{"count":1}' >/tmp/created
C_ID=$(sed 's/.*"id":"\([^"]*\)".*/\1/' /tmp/created)
C_TOK=$(sed 's/.*"token":"\([^"]*\)".*/\1/' /tmp/created)
expect 403 "$HOST_TOKEN" "host token refused on a scoped route" -X POST "$U/v1/sandboxes/$C_ID/exec" -d '{"cmd":"id"}'
expect 200 "$C_TOK" "the scoped token still works" -X POST "$U/v1/sandboxes/$C_ID/exec" -d '{"cmd":"echo hi"}'
expect 200 "$HOST_TOKEN" "management still works" "$U/v1/sandboxes"
say "host credentials never authorize scoped routes"
expect 403 "$HOST_TOKEN" "exec" -X POST "$U/v1/sandboxes/$A_ID/exec" -d '{"cmd":"id"}'
expect 403 "$HOST_TOKEN" "files" "$U/v1/sandboxes/$A_ID/files/read?path=f"
expect 403 "$HOST_TOKEN" "processes" "$U/v1/sandboxes/$A_ID/processes"
expect 403 "$HOST_TOKEN" "proxy" "$U/v1/sandboxes/$A_ID/proxy/9000/"
expect 403 "$HOST_TOKEN" "delete one sandbox" -X DELETE "$U/v1/sandboxes/$A_ID"
expect 200 "$A_TOK" "scoped token can delete its sandbox" -X DELETE "$U/v1/sandboxes/$A_ID"
expect 200 "$HOST_TOKEN" "management can still delete all" -X DELETE "$U/v1/sandboxes"

say "result"
if [ "$failures" -eq 0 ]; then
Expand Down
Loading
Loading