-
Notifications
You must be signed in to change notification settings - Fork 267
feat: add single-tenant API key authentication #123
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
92bcbad
636dc52
ab10d19
f7b6706
b6bedef
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,6 +10,11 @@ MODE="$1" | |
| shift | ||
| KUBECTL_BIN="${KUBECTL:-kubectl}" | ||
| OVERLAY_NAME="${K8S_OVERLAY:-default}" | ||
| NAMESPACE="${K8S_NAMESPACE:-agentenv-system}" | ||
| if [[ ${#NAMESPACE} -gt 63 || ! "${NAMESPACE}" =~ ^[a-z0-9]([-a-z0-9]*[a-z0-9])?$ ]]; then | ||
| echo "K8S_NAMESPACE must be a valid Kubernetes namespace name" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
| REPO_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)" | ||
|
|
@@ -28,6 +33,54 @@ sed_in_place() { | |
|
|
||
| cp -R "${SCRIPT_DIR}" "${TEMP_DIR}/k8s" | ||
| cp "${REPO_ROOT}/config/default.toml" "${TEMP_DIR}/k8s/base/config/agentenv.toml" | ||
| sed_in_place "s#^namespace: agentenv-system#namespace: ${NAMESPACE}#" "${TEMP_DIR}/k8s/base/kustomization.yaml" | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| sed_in_place "s# name: agentenv-system# name: ${NAMESPACE}#" "${TEMP_DIR}/k8s/base/namespace.yaml" | ||
| sed_in_place "s#\"namespace\": \"agentenv-system\"#\"namespace\": \"${NAMESPACE}\"#" "${TEMP_DIR}/k8s/base/config/scheduler.json" | ||
|
|
||
| namespace_name="" | ||
| if [[ "${MODE}" == "apply" ]]; then | ||
| if ! namespace_name="$("${KUBECTL_BIN}" get namespace "${NAMESPACE}" --ignore-not-found -o name)"; then | ||
| echo "failed to check namespace ${NAMESPACE}" >&2 | ||
| exit 1 | ||
| fi | ||
| fi | ||
|
|
||
| read_existing_api_key() { | ||
| local encoded_value="" | ||
|
|
||
| if [[ -z "${namespace_name}" ]]; then | ||
| return 0 | ||
| fi | ||
| if ! encoded_value="$("${KUBECTL_BIN}" -n "${NAMESPACE}" get secret agentenv-auth \ | ||
| --ignore-not-found -o 'go-template={{index .data "AENV_API_KEY"}}')"; then | ||
| echo "failed to read AENV_API_KEY from Secret ${NAMESPACE}/agentenv-auth" >&2 | ||
| return 1 | ||
| fi | ||
| if [[ -n "${encoded_value}" ]]; then | ||
| printf '%s' "${encoded_value}" | base64 -d | ||
| fi | ||
| } | ||
|
|
||
| if [[ "${MODE}" != "delete" ]]; then | ||
| API_KEY_VALUE="" | ||
| if [[ "${AENV_API_KEY+x}" == "x" ]]; then | ||
| API_KEY_VALUE="${AENV_API_KEY}" | ||
| elif ! API_KEY_VALUE="$(read_existing_api_key)"; then | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [[ -z "${API_KEY_VALUE}" ]]; then | ||
| API_KEY_VALUE="e2b_$(od -An -N32 -tx1 /dev/urandom | tr -d '[:space:]')" | ||
| fi | ||
| if [[ ! "${API_KEY_VALUE}" =~ ^[A-Za-z0-9._~-]{32,4096}$ ]]; then | ||
| echo "AENV_API_KEY must contain between 32 and 4096 URL-safe characters" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| sed_in_place \ | ||
| "s#- AENV_API_KEY=.*#- AENV_API_KEY=${API_KEY_VALUE}#" \ | ||
| "${TEMP_DIR}/k8s/base/kustomization.yaml" | ||
|
Comment on lines
+80
to
+82
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| fi | ||
|
|
||
| if [[ "${SANDBOX_PROXY_DOMAINS+x}" == "x" ]]; then | ||
| ESCAPED_SANDBOX_PROXY_DOMAINS="${SANDBOX_PROXY_DOMAINS//\\/\\\\}" | ||
|
|
@@ -65,6 +118,9 @@ case "${MODE}" in | |
| ;; | ||
| apply) | ||
| "${KUBECTL_BIN}" apply -k "${OVERLAY_PATH}" "$@" | ||
| echo "AgentENV API key stored in Secret ${NAMESPACE}/agentenv-auth." >&2 | ||
| echo "Read it with:" >&2 | ||
| echo " ${KUBECTL_BIN} -n ${NAMESPACE} get secret agentenv-auth -o go-template='{{index .data \"AENV_API_KEY\" | base64decode}}{{\"\\n\"}}'" >&2 | ||
| ;; | ||
| delete) | ||
| "${KUBECTL_BIN}" delete --ignore-not-found -k "${OVERLAY_PATH}" "$@" | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,124 @@ | ||
| # Authentication | ||
|
|
||
| AgentENV uses one shared API key for a single-tenant deployment. The gateway | ||
| and every runtime node in a cluster must resolve the same key. | ||
|
|
||
| Clients authenticate API requests with: | ||
|
|
||
| ```text | ||
| X-API-Key: <AENV_API_KEY> | ||
| ``` | ||
|
|
||
| `Authorization`, `X-Admin-Token`, and `X-Team-ID` do not authenticate | ||
| AgentENV. The `Authorization` header is left unchanged when a request is | ||
| proxied into a sandbox, so applications inside a sandbox can use it normally. | ||
| `GET /health` is public for load balancer and container health checks. | ||
|
|
||
| E2B SDK users set `E2B_API_KEY` to the same value. Sandbox create responses | ||
| include an independent `trafficAccessToken`; send it as | ||
| `e2b-traffic-access-token` on application proxy requests. The token is scoped to | ||
| the sandbox and is not accepted for control-plane API calls. | ||
|
|
||
| For secure sandboxes, `envdAccessToken` is a separate credential for envd | ||
| control traffic and must be sent as `X-Access-Token` only when targeting the | ||
| envd control-plane port. It is absent for insecure sandboxes. | ||
|
|
||
| Both sandbox credentials are derived from the sandbox ID and one independent | ||
| `AENV_SANDBOX_ACCESS_TOKEN_HASH_SEED`. They are not derived from the API key. | ||
|
|
||
| ## Key Resolution | ||
|
|
||
| On normal startup, a runtime node uses the first available source: | ||
|
|
||
| 1. `AENV_API_KEY` | ||
| 2. `/run/secrets/api-key` | ||
| 3. `$AENV_HOME/secrets/api-key` | ||
|
|
||
| If neither an environment value nor an external secret exists, the server | ||
| generates a 256-bit key and atomically stores it in the managed path with | ||
| `0600` permissions. It reuses that key on later starts. Dependency and host | ||
| setup modes do not create a key. | ||
|
|
||
| The gateway uses `AENV_API_KEY` or `/run/secrets/api-key`; it never generates a | ||
| key. Runtime nodes validate sandbox-scoped tokens, so the gateway does not need | ||
| the sandbox seed. | ||
|
|
||
| ## Installation Methods | ||
|
|
||
| For a native installation, start the service once and read the managed key: | ||
|
|
||
| ```bash | ||
| sudo cat /var/lib/aenv/secrets/api-key | ||
| ``` | ||
|
|
||
| When upgrading an installation that already has `AENV_API_KEY` in | ||
| `/etc/default/aenv`, the installer preserves that entry and the server keeps | ||
| using it. Fresh installations leave key creation to the server. | ||
|
|
||
| For a single Docker container, no auth volume is required. The server creates | ||
| the key in its writable container layer: | ||
|
|
||
| ```bash | ||
| docker exec aenv-server cat /workspace/env/secrets/api-key | ||
| ``` | ||
|
|
||
| Removing the container removes this generated key. Supply an explicit key or | ||
| mount a secret at `/run/secrets/api-key` when it must remain stable across | ||
| container replacements. | ||
|
|
||
| The checked-in Compose deployment mounts one named volume read-write on both | ||
| runtime nodes and read-only at `/run/secrets` on the gateway. Concurrent node | ||
| startup is safe: atomic creation makes both nodes converge on the same key and | ||
| sandbox seed. The gateway reads only the API key from that volume. Read it with: | ||
|
|
||
| ```bash | ||
| docker compose -f deploy/docker-compose.yml exec -T agentenv-a \ | ||
| cat /workspace/env/secrets/api-key | ||
| ``` | ||
|
|
||
| `docker compose down` preserves the key. `docker compose down -v` removes the | ||
| auth volume, so the next startup generates a new key. | ||
|
|
||
| `make k8s-apply` creates `Secret/agentenv-auth` with an API key on the first | ||
| apply, then reuses it. Read the key with: | ||
|
|
||
| ```bash | ||
| kubectl -n agentenv-system get secret agentenv-auth \ | ||
| -o go-template='{{index .data "AENV_API_KEY" | base64decode}}{{"\n"}}' | ||
| ``` | ||
|
|
||
| For a single-node manual build, start the server and read | ||
| `$AENV_HOME/secrets/api-key`. To provide your own key instead, export it before | ||
| startup: | ||
|
|
||
| ```bash | ||
| export AENV_API_KEY="e2b_$(openssl rand -hex 32)" | ||
| make start-server | ||
| ``` | ||
|
|
||
| Custom keys must contain between 32 and 4096 URL-safe characters. In a multi-node | ||
| deployment, use exactly the same value for the gateway and every runtime node. | ||
| The generated keys use `e2b_` followed by hexadecimal characters so they pass | ||
| the E2B SDK default API-key validation. Use that format for custom keys when | ||
| you need E2B SDK compatibility. | ||
|
|
||
| Docker Compose secrets can supply a pre-existing key without another AgentENV | ||
| configuration variable. In an override file, define a file-backed secret and | ||
| mount it with `target: api-key` on the gateway and every runtime node. Compose | ||
| then exposes the standard `/run/secrets/api-key` path. Compose secret sources | ||
| must already exist, so the named-volume setup remains the zero-configuration | ||
| default that allows Rust to generate the key during startup. | ||
|
|
||
| ## Transport Security | ||
|
|
||
| API key authentication does not encrypt HTTP traffic. Do not send the key over | ||
| an untrusted plaintext network. Keep AgentENV on loopback or a trusted private | ||
| network, use a VPN, or terminate HTTPS at a reverse proxy or load balancer. | ||
|
|
||
| ## Rotation | ||
|
|
||
| Changing `AENV_API_KEY` invalidates existing client API credentials without | ||
| changing sandbox credentials; apply it to the gateway and every runtime node | ||
| together. Changing | ||
| `AENV_SANDBOX_ACCESS_TOKEN_HASH_SEED` rotates both `trafficAccessToken` and | ||
| `envdAccessToken` values and must be changed on every runtime node together. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
K8S_NAMESPACEis used for the preflight lookup, Secret read, and informational output, but the rendered Kustomization still applies resources to its configuredagentenv-systemnamespace. WithK8S_NAMESPACEset to any other value, this command can generate a new key after checking the wrong namespace and then store it in a different namespace than the one reported, so subsequent runs will not preserve the key. Ensure the temporary Kustomization namespace and namespace resource are rewritten toNAMESPACE, or reject custom values explicitly.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in amended commit b6bedef. The temporary Kustomization, Namespace resource, and scheduler discovery config now use the validated K8S_NAMESPACE value, so preflight, generated resources, and scheduler registration agree.