Closed
Conversation
…Kubernetes pod isolation (#2) Addresses anomalyco#20067 (as MVP / alternative approach) ## Intent `opencode serve` runs a single-user server — everyone hitting the same instance shares sessions, filesystem, and API keys. This is the core problem raised in anomalyco#20067. This PR takes a **proxy-based approach** rather than adding auth into `opencode serve` itself: a lightweight router sits in front of one `opencode serve` pod *per session*, providing: - **Multi-user isolation** — each authenticated user gets their own pod with their own filesystem and conversation history - **Multi-session isolation** — each (user, repo, branch) triple gets its own dedicated pod, matching the Claude Code web UX where separate coding sessions are independent - **Per-session credentials** — API keys are injected per-pod via Kubernetes Secrets, enabling per-user or per-team key management at the infrastructure level - **Auth delegation** — identity comes from an upstream auth proxy (oauth2-proxy with GitHub OAuth), so `opencode serve` itself needs no auth changes This is an MVP. It is opinionated about the deployment target (Kubernetes) and auth mechanism (oauth2-proxy), but the router itself is a standard Node.js HTTP/WebSocket proxy that could front other auth mechanisms. ## Architecture ``` Browser │ HTTPS (<hash>-oc.<domain> or opencode-router.<domain>) ▼ Cloudflare Tunnel → Traefik ├─ opencode-router.<domain> ──[oauth2-chain]──▶ opencode-router (:3000) │ │ │ setup SPA: pick repo + branch │ │ POST /api/sessions │ ▼ │ K8s: create Pod + PVC │ (one per email+repo+branch) │ └─ <hash>-oc.<domain> ──[oauth2-chain]──▶ opencode-router (:3000) │ proxy by hash from Host header ▼ opencode-session-<hash> Pod (:4096) (opencode serve, isolated filesystem) ``` Each session gets a **dedicated subdomain** (`<hash>-oc.<domain>`) — the router identifies which pod to proxy to by extracting the hash from the `Host` header. This is the same pattern used by Claude Code web, where each coding session has its own URL. ## Major changes **New: `packages/opencode-router/`** — HTTP/WebSocket reverse proxy - `src/pod-manager.ts` — creates/deletes Kubernetes Pods and PVCs on demand; session key is `sha256(email + repoUrl + branch)[0:12]`; full `restricted` PSS-compliant securityContext; git-init init container clones the repo on first pod creation - `src/api.ts` — `GET /api/sessions`, `POST /api/sessions {repoUrl, branch}`, `GET /api/sessions/:hash`; session URLs include `ROUTE_SUFFIX` for first-level subdomain TLS coverage - `src/index.ts` — HTTP server + WebSocket upgrade handler; `getSessionHash()` extracts session from `Host` header; background idle pod cleanup - `src/config.ts` — `ROUTER_DOMAIN` (required base domain), `ROUTE_SUFFIX` (default `""`, set to `-oc` in production), plus K8s/storage config - `Dockerfile` — multi-stage build from monorepo root - `docs/deployment.md` — full Kubernetes deployment guide (RBAC, Secrets, Ingress, WebSocket config) - `docs/adr-001-per-user-pod-isolation.md` — rationale for proxy pattern over in-process auth - `docs/adr-002-restricted-pss-securitycontext.md` — Kubernetes security hardening **New: `packages/opencode-router-app/`** — session management SPA - Session picker: choose repo URL + branch, displays existing sessions for the authenticated user - Loading screen while pod is initialising (polls `GET /api/sessions/:hash`) - Redirects to the session URL once the pod is running **`ROUTE_SUFFIX` design** — session hostnames are `<hash><ROUTE_SUFFIX>.<ROUTER_DOMAIN>`. The suffix (e.g. `-oc`) keeps sessions at the first subdomain level, covered by a standard `*.<domain>` wildcard TLS certificate. Without it, sessions at `<hash>.opencode-router.<domain>` (second level) would require a paid certificate product. ## Side effects - `ROUTER_DOMAIN` is a **required** env var - `ROUTE_SUFFIX` defaults to `""` (backward compatible); set to `-oc` in production - PVCs are never deleted — only pods are removed on idle timeout; storage must be monitored externally - `DEV_EMAIL` enables local dev auth bypass; must never be set in production - The router is stateless across replicas; the in-process idle-cleanup timer and activity throttle run independently on each replica (acceptable for homelab scale)
# Intent Deploy opencode-router and its cloudflare-operator sidecar to the homelab Kubernetes cluster from this repo, with full CI/CD and a local make-based workflow. The homelab repo previously owned these deployment recipes; they now live here so the opencode team can ship infrastructure changes independently. # Key Components Affected - deployment/homelab/: standalone Pulumi stack (outside bun workspace to avoid catalog: protocol conflicts) that deploys the router + operator sidecar using @mrsimpson/homelab-core-components (npmjs.com) and a StackReference to the homelab base stack for shared infra facts - deployment/opencode-cloudflare-operator/: operator source + Dockerfile moved here from homelab repo; now has its own package-lock.json so the Docker build can use npm ci - deployment/homelab/Makefile: local CD — build-router, build-operator, build-push, deploy, release targets covering the full build → push → deploy cycle; image tags encode package version + git SHA matching CI convention - .github/workflows/build-cloudflare-operator.yml: builds and pushes the operator image to GHCR on changes to deployment/opencode-cloudflare-operator/ - .github/workflows/build-opencode-router.yml: builds and pushes the router image on changes to packages/opencode-router/ - .github/workflows/deploy-homelab.yml: runs pulumi up in deployment/homelab/ after operator image builds succeed; no auth needed for npm install since @mrsimpson/homelab-core-components is now on public npmjs.com - packages/opencode-router/scripts/build-image.sh: local build script for the router image (kept in packages/ since the router stays there) # Side effects - deployment/ is intentionally not a bun workspace package; it is a plain npm directory to avoid bun catalog: protocol incompatibilities with Node.js tooling - PULUMI_ACCESS_TOKEN, KUBECONFIG (homelab cluster), and GITHUB_TOKEN must be set as GitHub Actions secrets in this repo for CI to work
github-actions
bot
added
the
needs:compliance
Contributor
|
This PR doesn't fully meet our contributing guidelines and PR template. What needs to be fixed:
Please edit this PR description to address the above within 2 hours, or it will be automatically closed. If you believe this was flagged incorrectly, please let a maintainer know. |
Contributor
|
Hey! Your PR title Please update it to start with one of:
Where See CONTRIBUTING.md for details. |
Author
|
sorry, wrong target repo |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.