Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ io-uring = "0.7.9"
tokio-tungstenite = "0.28"
sha2 = "0.10"
hmac = "0.12"
subtle = "2.6"
hex = "0.4"
semver = "1"
iroh = { version = "=1.0.0-rc.0" }
Expand Down
23 changes: 18 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,9 @@ If your server does not support standard KVM, see the [PVM deployment guide](htt
## ⚡ Quick Start (Single Node)

> [!WARNING]
> **AgentENV currently does not support authorization.** Do not expose the AgentENV
> API to the public network. Run it only on a trusted network or behind an
> authorization proxy with appropriate network controls.
> AgentENV authenticates API requests but does not encrypt traffic. Do not send
> the API key over an untrusted plaintext network. Run AgentENV on a trusted
> network or terminate HTTPS at a reverse proxy or load balancer.

**1. Install and start the server**

Expand All @@ -66,7 +66,7 @@ Set up the server:
```bash
curl -fsSL https://raw.githubusercontent.com/kvcache-ai/AgentENV/main/scripts/docker-setup.sh | sudo bash
docker pull ghcr.io/kvcache-ai/aenv-server:latest
docker run -d --privileged -v /dev:/dev -p 8000:8000 ghcr.io/kvcache-ai/aenv-server:latest
docker run -d --name aenv-server --privileged -v /dev:/dev -p 8000:8000 ghcr.io/kvcache-ai/aenv-server:latest
```

The server is accessible at `http://127.0.0.1:8000` by default.
Expand All @@ -83,10 +83,23 @@ curl -fsSL https://raw.githubusercontent.com/kvcache-ai/AgentENV/main/scripts/in

**3. Authenticate**

The server generates an API key on its first startup. Retrieve it for the
installation method used in step 1:

```bash
# Native install
sudo cat /var/lib/aenv/secrets/api-key

# Docker
docker exec aenv-server cat /workspace/env/secrets/api-key
```

Then run `aenv auth` and paste that key:

```bash
aenv auth
# AENV server URL [http://localhost:8000]: http://127.0.0.1:8000
# API key: dummy
# API key: <paste the generated key>
```

**4. Pull a template and run a sandbox**
Expand Down
2 changes: 1 addition & 1 deletion config/default.toml
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@ init_timeout_secs = 60
poll_ms = 3

[sandbox]
# Optional secret used to derive per-sandbox envd access tokens. When unset,
# Optional secret used to derive per-sandbox envd and traffic access tokens. When unset,
# AgentENV creates a node-local seed under $AENV_HOME/secrets. Configure the
# same explicit value on every node when cross-node sandbox recovery is required.
# access_token_hash_seed = "replace-with-a-secret"
Expand Down
8 changes: 6 additions & 2 deletions deploy/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@ x-agentenv-base: &agentenv-base
- /dev:/dev
- ${CONFIG_PATH:-../config/default.toml}:/workspace/config/default.toml:ro
# Runtime assets are baked into the image by `server --setup-only`; compose
# persists only committed snapshots across container restarts.
# persists committed snapshots and deployment secrets across restarts.
- agentenv-snapshot-store:/workspace/env/snapshot-store
- agentenv-auth:/workspace/env/secrets
devices:
- /dev/kvm:/dev/kvm
privileged: true
Expand Down Expand Up @@ -69,7 +70,9 @@ services:
depends_on:
scheduler:
condition: service_healthy
volumes: *control-plane-config-volume
volumes:
- ./docker/config/default.json:/config/default.json:ro
- agentenv-auth:/run/secrets:ro
environment:
GATEWAY_HTTP_LISTEN_ADDR: :8080
GATEWAY_SCHEDULER_ADDR: scheduler:9090
Expand Down Expand Up @@ -102,4 +105,5 @@ services:
- "8002:8000"

volumes:
agentenv-auth:
agentenv-snapshot-store:
7 changes: 6 additions & 1 deletion deploy/k8s/base/agentenv-daemonset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,11 @@ spec:
image: agentenv-runtime:latest
imagePullPolicy: IfNotPresent
env:
- name: AENV_API_KEY
valueFrom:
secretKeyRef:
name: agentenv-auth
key: AENV_API_KEY
- name: AENV_CONFIG_PATH
value: /workspace/config/agentenv.toml
- name: AENV_SANDBOX_ACCESS_TOKEN_HASH_SEED
Expand Down Expand Up @@ -76,7 +81,7 @@ spec:
- |
echo "preStop: waiting for sandboxes to drain..."
while true; do
count=$(curl -sf -H 'X-API-Key: preStop' http://localhost:8000/sandboxes | jq 'length') || count=""
count=$(curl -sf -H "X-API-Key: ${AENV_API_KEY}" http://localhost:8000/sandboxes | jq 'length') || count=""
if [ -z "$count" ]; then
echo "preStop: failed to query sandbox count, retrying..."
sleep 3
Expand Down
5 changes: 5 additions & 0 deletions deploy/k8s/base/gateway-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ spec:
- name: http
containerPort: 8080
env:
- name: AENV_API_KEY
valueFrom:
secretKeyRef:
name: agentenv-auth
key: AENV_API_KEY
- name: GATEWAY_SANDBOX_PROXY_DOMAINS
valueFrom:
configMapKeyRef:
Expand Down
5 changes: 5 additions & 0 deletions deploy/k8s/base/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,11 @@ configMapGenerator:
literals:
- SANDBOX_PROXY_DOMAINS=

secretGenerator:
- name: agentenv-auth
literals:
- AENV_API_KEY=

images:
- name: agentenv-gateway
newName: agentenv-gateway
Expand Down
56 changes: 56 additions & 0 deletions deploy/k8s/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@ MODE="$1"
shift
KUBECTL_BIN="${KUBECTL:-kubectl}"
OVERLAY_NAME="${K8S_OVERLAY:-default}"
NAMESPACE="${K8S_NAMESPACE:-agentenv-system}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug · high
K8S_NAMESPACE is used for the preflight lookup, Secret read, and informational output, but the rendered Kustomization still applies resources to its configured agentenv-system namespace. With K8S_NAMESPACE set to any other value, this command can generate a new key after checking the wrong namespace and then store it in a different namespace than the one reported, so subsequent runs will not preserve the key. Ensure the temporary Kustomization namespace and namespace resource are rewritten to NAMESPACE, or reject custom values explicitly.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in amended commit b6bedef. The temporary Kustomization, Namespace resource, and scheduler discovery config now use the validated K8S_NAMESPACE value, so preflight, generated resources, and scheduler registration agree.

if [[ ${#NAMESPACE} -gt 63 || ! "${NAMESPACE}" =~ ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ ]]; then
echo "K8S_NAMESPACE must be a valid Kubernetes namespace name" >&2
exit 1
fi

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
Expand All @@ -28,6 +33,54 @@ sed_in_place() {

cp -R "${SCRIPT_DIR}" "${TEMP_DIR}/k8s"
cp "${REPO_ROOT}/config/default.toml" "${TEMP_DIR}/k8s/base/config/agentenv.toml"
sed_in_place "s#^namespace: agentenv-system#namespace: ${NAMESPACE}#" "${TEMP_DIR}/k8s/base/kustomization.yaml"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug · medium
When K8S_OVERLAY=local-dev, the copied overlay still declares namespace: agentenv-system; only the base kustomization is rewritten here. The overlay's namespace transformer therefore continues to place namespaced resources (including the agentenv-auth Secret and workloads) in agentenv-system, while this script reads the Secret and reports success under the requested custom namespace. Update all selected overlay namespace declarations (or use a shared namespace substitution) before applying.

sed_in_place "s# name: agentenv-system# name: ${NAMESPACE}#" "${TEMP_DIR}/k8s/base/namespace.yaml"
sed_in_place "s#\"namespace\": \"agentenv-system\"#\"namespace\": \"${NAMESPACE}\"#" "${TEMP_DIR}/k8s/base/config/scheduler.json"

namespace_name=""
if [[ "${MODE}" == "apply" ]]; then
if ! namespace_name="$("${KUBECTL_BIN}" get namespace "${NAMESPACE}" --ignore-not-found -o name)"; then
echo "failed to check namespace ${NAMESPACE}" >&2
exit 1
fi
fi

read_existing_api_key() {
local encoded_value=""

if [[ -z "${namespace_name}" ]]; then
return 0
fi
if ! encoded_value="$("${KUBECTL_BIN}" -n "${NAMESPACE}" get secret agentenv-auth \
--ignore-not-found -o 'go-template={{index .data "AENV_API_KEY"}}')"; then
echo "failed to read AENV_API_KEY from Secret ${NAMESPACE}/agentenv-auth" >&2
return 1
fi
if [[ -n "${encoded_value}" ]]; then
printf '%s' "${encoded_value}" | base64 -d
fi
}

if [[ "${MODE}" != "delete" ]]; then
API_KEY_VALUE=""
if [[ "${AENV_API_KEY+x}" == "x" ]]; then
API_KEY_VALUE="${AENV_API_KEY}"
elif ! API_KEY_VALUE="$(read_existing_api_key)"; then
exit 1
fi

if [[ -z "${API_KEY_VALUE}" ]]; then
API_KEY_VALUE="e2b_$(od -An -N32 -tx1 /dev/urandom | tr -d '[:space:]')"
fi
if [[ ! "${API_KEY_VALUE}" =~ ^[A-Za-z0-9._~-]{32,4096}$ ]]; then
echo "AENV_API_KEY must contain between 32 and 4096 URL-safe characters" >&2
exit 1
fi

sed_in_place \
"s#- AENV_API_KEY=.*#- AENV_API_KEY=${API_KEY_VALUE}#" \
"${TEMP_DIR}/k8s/base/kustomization.yaml"
Comment on lines +80 to +82

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

security · medium
The API key is embedded directly in the sed expression passed as a command-line argument. While this command is running, local users with process-inspection access (for example, ps/procfs) can read the credential, and shell tracing would also expose it. Avoid putting the secret in argv; for example, generate/render the Kustomize input through a temporary file or use a mechanism that supplies the value via stdin/environment without logging it, and ensure temporary files have restrictive permissions.

fi

if [[ "${SANDBOX_PROXY_DOMAINS+x}" == "x" ]]; then
ESCAPED_SANDBOX_PROXY_DOMAINS="${SANDBOX_PROXY_DOMAINS//\\/\\\\}"
Expand Down Expand Up @@ -65,6 +118,9 @@ case "${MODE}" in
;;
apply)
"${KUBECTL_BIN}" apply -k "${OVERLAY_PATH}" "$@"
echo "AgentENV API key stored in Secret ${NAMESPACE}/agentenv-auth." >&2
echo "Read it with:" >&2
echo " ${KUBECTL_BIN} -n ${NAMESPACE} get secret agentenv-auth -o go-template='{{index .data \"AENV_API_KEY\" | base64decode}}{{\"\\n\"}}'" >&2
;;
delete)
"${KUBECTL_BIN}" delete --ignore-not-found -k "${OVERLAY_PATH}" "$@"
Expand Down
1 change: 1 addition & 0 deletions docs/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

# Configuration

- [Authentication](./configuration/authentication.md)
- [Configuration Reference](./configuration/reference.md)
- [Environment Variables](./configuration/env-vars.md)

Expand Down
2 changes: 1 addition & 1 deletion docs/src/concepts/sandboxes.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ aenv start --cold ubuntu:24.04

The cold-start API accepts an optional `diskSizeMB` field to set the root filesystem's virtual size in MiB. Explicit values must be at least 1024 MiB and divisible by 1024 because the current resize tool operates at 1 GiB granularity. Growth is allowed by default; shrinking below the source image size requires `ublk.overlaybd.allow_shrink = true`. If omitted, the image's built-in virtual size is used. Resizing applies only when creating a fresh writable root filesystem, not to read-only images, images with an existing upper, or snapshot resume. Sandbox responses also report disk size as `diskSizeMB`.

Use `aenv start --secure` with either warm or cold starts to require an envd access token for command and file operations. The CLI obtains and sends the token automatically. Secure mode protects the envd control port only; it does not add authentication to application ports. Each fork derives a distinct envd token from the child sandbox ID.
Use `aenv start --secure` with either warm or cold starts to require an envd access token for command and file operations. The CLI obtains and sends the token automatically. Secure mode protects the envd control port, while application proxy requests use the independent sandbox-scoped traffic token. Each fork derives distinct envd and traffic credentials from the child sandbox ID.

---

Expand Down
124 changes: 124 additions & 0 deletions docs/src/configuration/authentication.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# Authentication

AgentENV uses one shared API key for a single-tenant deployment. The gateway
and every runtime node in a cluster must resolve the same key.

Clients authenticate API requests with:

```text
X-API-Key: <AENV_API_KEY>
```

`Authorization`, `X-Admin-Token`, and `X-Team-ID` do not authenticate
AgentENV. The `Authorization` header is left unchanged when a request is
proxied into a sandbox, so applications inside a sandbox can use it normally.
`GET /health` is public for load balancer and container health checks.

E2B SDK users set `E2B_API_KEY` to the same value. Sandbox create responses
include an independent `trafficAccessToken`; send it as
`e2b-traffic-access-token` on application proxy requests. The token is scoped to
the sandbox and is not accepted for control-plane API calls.

For secure sandboxes, `envdAccessToken` is a separate credential for envd
control traffic and must be sent as `X-Access-Token` only when targeting the
envd control-plane port. It is absent for insecure sandboxes.

Both sandbox credentials are derived from the sandbox ID and one independent
`AENV_SANDBOX_ACCESS_TOKEN_HASH_SEED`. They are not derived from the API key.

## Key Resolution

On normal startup, a runtime node uses the first available source:

1. `AENV_API_KEY`
2. `/run/secrets/api-key`
3. `$AENV_HOME/secrets/api-key`

If neither an environment value nor an external secret exists, the server
generates a 256-bit key and atomically stores it in the managed path with
`0600` permissions. It reuses that key on later starts. Dependency and host
setup modes do not create a key.

The gateway uses `AENV_API_KEY` or `/run/secrets/api-key`; it never generates a
key. Runtime nodes validate sandbox-scoped tokens, so the gateway does not need
the sandbox seed.

## Installation Methods

For a native installation, start the service once and read the managed key:

```bash
sudo cat /var/lib/aenv/secrets/api-key
```

When upgrading an installation that already has `AENV_API_KEY` in
`/etc/default/aenv`, the installer preserves that entry and the server keeps
using it. Fresh installations leave key creation to the server.

For a single Docker container, no auth volume is required. The server creates
the key in its writable container layer:

```bash
docker exec aenv-server cat /workspace/env/secrets/api-key
```

Removing the container removes this generated key. Supply an explicit key or
mount a secret at `/run/secrets/api-key` when it must remain stable across
container replacements.

The checked-in Compose deployment mounts one named volume read-write on both
runtime nodes and read-only at `/run/secrets` on the gateway. Concurrent node
startup is safe: atomic creation makes both nodes converge on the same key and
sandbox seed. The gateway reads only the API key from that volume. Read it with:

```bash
docker compose -f deploy/docker-compose.yml exec -T agentenv-a \
cat /workspace/env/secrets/api-key
```

`docker compose down` preserves the key. `docker compose down -v` removes the
auth volume, so the next startup generates a new key.

`make k8s-apply` creates `Secret/agentenv-auth` with an API key on the first
apply, then reuses it. Read the key with:

```bash
kubectl -n agentenv-system get secret agentenv-auth \
-o go-template='{{index .data "AENV_API_KEY" | base64decode}}{{"\n"}}'
```

For a single-node manual build, start the server and read
`$AENV_HOME/secrets/api-key`. To provide your own key instead, export it before
startup:

```bash
export AENV_API_KEY="e2b_$(openssl rand -hex 32)"
make start-server
```

Custom keys must contain between 32 and 4096 URL-safe characters. In a multi-node
deployment, use exactly the same value for the gateway and every runtime node.
The generated keys use `e2b_` followed by hexadecimal characters so they pass
the E2B SDK default API-key validation. Use that format for custom keys when
you need E2B SDK compatibility.

Docker Compose secrets can supply a pre-existing key without another AgentENV
configuration variable. In an override file, define a file-backed secret and
mount it with `target: api-key` on the gateway and every runtime node. Compose
then exposes the standard `/run/secrets/api-key` path. Compose secret sources
must already exist, so the named-volume setup remains the zero-configuration
default that allows Rust to generate the key during startup.

## Transport Security

API key authentication does not encrypt HTTP traffic. Do not send the key over
an untrusted plaintext network. Keep AgentENV on loopback or a trusted private
network, use a VPN, or terminate HTTPS at a reverse proxy or load balancer.

## Rotation

Changing `AENV_API_KEY` invalidates existing client API credentials without
changing sandbox credentials; apply it to the gateway and every runtime node
together. Changing
`AENV_SANDBOX_ACCESS_TOKEN_HASH_SEED` rotates both `trafficAccessToken` and
`envdAccessToken` values and must be changed on every runtime node together.
Loading
Loading