diff --git a/.gitignore b/.gitignore index bec3c314..40684ce0 100644 --- a/.gitignore +++ b/.gitignore @@ -2,4 +2,7 @@ __pycache__/ .pytest_cache/ .sandcat/settings.local.json .devcontainer -.orca \ No newline at end of file +.orca +.worktrees/ +docs/examples/proxy-peer/.env +docs/examples/netbird-server/config.local.yaml diff --git a/cli/README.md b/cli/README.md index 8d2b8940..8b3ccbfc 100644 --- a/cli/README.md +++ b/cli/README.md @@ -22,6 +22,17 @@ Options: - `--stacks` - Comma-separated development stacks to install: `node`, `python`, `java`, `rust`, `go`, `scala`, `ruby`, `dotnet`, `zig` (skips prompt) - `--proxy` - Proxy UI mode: `web` (default, mitmweb browser UI) or `tui` (mitmproxy console, use with `sandcat proxy` to attach) - `--secret-provider` / `--sp` - Secret backend: `none` (default), `1password`, `protonpass` (skips prompt when set) +- `--netbird` - Enable dynamic WireGuard control via NetBird. The NetBird client + daemon runs inside the **mitmproxy** container and manages `wt0` — a second + WireGuard interface for the NetBird overlay mesh. Agent egress always flows + `wg0 (wg-client) → mitmproxy L7 inspect → internet or wt0 mesh`, so all + traffic — including mesh traffic — is subject to mitmproxy network rules and + secret substitution. Seeds `netbird_enrollment_key` and `netbird_api_token` + in `~/.config/sandcat/settings.json`. +- `--netbird-management-url` - Existing NetBird management server URL (requires + `--netbird`). Omit to use NetBird Cloud (`https://api.netbird.io`). Sandcat + does not create or start a management server; see + [`docs/examples/netbird-server/`](../docs/examples/netbird-server/). - `--1password` - Deprecated alias for `--secret-provider 1password` - `--features` - Comma-separated optional non-provider features: `tui` (proxy console mode; prefer `--proxy tui`), `no-gitignore` (skip appending the `# Sandcat` block to the project's `.gitignore`; equivalent to `SANDCAT_GITIGNORE=false`), `no-rtk` (skip RTK installation; equivalent to `SANDCAT_RTK=false`), `strict-network` (project settings get network presets for the selected stacks instead of the allow-all-GET wildcard; equivalent to `SANDCAT_STRICT_NETWORK=true`) - `--name` - Project name for Docker Compose (default: derived from directory name) @@ -43,6 +54,13 @@ sandcat init --agent claude --ide vscode --secret-provider 1password --name mypr # With Proton Pass integration sandcat init --agent claude --ide vscode --secret-provider protonpass --name myproject + +# With NetBird dynamic WireGuard +sandcat init --agent claude --ide vscode --netbird --name myproject + +# Point at an existing self-hosted management server +sandcat init --agent cursor --ide vscode --netbird \ + --netbird-management-url https://netbird.example.com --name myproject ``` #### Proton Pass setup (scoped Personal Access Token) @@ -213,6 +231,116 @@ shell, `sandcat run npm install` runs npm inside the container. Options: - `--build` — Rebuild images before running (e.g. after editing `Dockerfile.app`) +## Dynamic networking (NetBird) + +When initialized with `--netbird`, sandcat enrolls **mitmproxy** as a NetBird peer. +The NetBird client daemon runs inside the mitmproxy container and manages `wt0` — a +second WireGuard interface for the overlay mesh. Agent traffic always flows: + +``` +agent → wg0 (wg-client kill switch) → mitmproxy (L7 inspect + secrets) → internet + ↘ wt0 (NetBird mesh) +``` + +This design eliminates the routing collision that occurred when NetBird ran on `wg-client` +alongside `wg0` (WireGuard-in-WireGuard). wg-client is now a pure tunnel shim with no +NetBird involvement. + +The NetBird client binary is pinned by version and per-arch sha256 in +[`templates/devcontainer/sandcat/netbird.env`](templates/devcontainer/sandcat/netbird.env). +`sandcat init --netbird` injects these as compose build args for `Dockerfile.mitmproxy` +automatically. NetBird is downloaded and checksum-verified in a throwaway builder stage, +then copied into a final image built `FROM $BASE_IMAGE`. + +`BASE_IMAGE` defaults to `mitmproxy/mitmproxy:latest`. When a secret provider is also +selected, `sandcat init` sets it to that provider's variant +(`ghcr.io/virtuslab/sandcat-mitmproxy-pass` or `-op`), so the proxy ends up with **both** +NetBird and the provider CLI. Combining `--netbird` with `--secret-provider` therefore +keeps `pass://` and `op://` references resolvable. + +To build the image manually: + +```bash +cd cli/templates/devcontainer/sandcat +set -a; . netbird.env; set +a +docker build -f Dockerfile.mitmproxy \ + --build-arg NETBIRD_VERSION \ + --build-arg NETBIRD_SHA256_AMD64 \ + --build-arg NETBIRD_SHA256_ARM64 \ + --build-arg BASE_IMAGE=ghcr.io/virtuslab/sandcat-mitmproxy-pass:latest \ + -t mitmproxy-netbird-test . +``` + +### Setup + +NetBird uses **two separate credentials**. Both go in `~/.config/sandcat/settings.json` +(created by `sandcat init`; edit with `sandcat edit user-settings`): + +| Setting key | Used for | Where to get it | +|-------------|----------|-----------------| +| `netbird_enrollment_key` | Enrolling mitmproxy as a mesh peer (`NB_SETUP_KEY`); may be a literal or `op://` / `pass://` (resolved in the container) | NetBird dashboard → **Setup Keys** | +| `netbird_api_token` | Used by mitmproxy for same-name replace and dns_label; may be a literal or `op://` / `pass://` (resolved in the container) | NetBird dashboard → **API Keys** (Personal Access Token) | +| `netbird_management_url` | Management API and dashboard | Empty = cloud; otherwise your server URL | +| `netbird_enrollment_management_url` | mitmproxy enrollment URL (container cannot use `localhost`) | Docker host LAN IP for a local server; see [docs/examples/netbird-server](../docs/examples/netbird-server/) | + +Complete steps 1–4 below before enrollment. Container enrollment +(`netbird_enrollment_key`) is separate from same-name replace and dns_label +(`netbird_api_token`) — you need the API token even if the setup key is already +in settings. + +1. Create a NetBird account at or self-host the server. +2. In the dashboard, create a **Setup Key** (for peer enrollment). +3. In the dashboard, create an **API Key** / personal access token (for mitmproxy same-name replace and dns_label). +4. Add both values to user settings: + +```json +{ + "netbird_enrollment_key": "your-setup-key-here", + "netbird_api_token": "your-api-token-here" +} +``` + +Or edit interactively: + +```bash +sandcat edit user-settings +``` + +`sandcat compose` and `sandcat run` read `netbird_enrollment_key` and +`netbird_api_token` from **user** settings (`~/.config/sandcat/settings.json`) +only — not from project `.sandcat/` files, which are bind-mounted into the +agent. Environment variables `NB_SETUP_KEY` and `NB_API_TOKEN` override +user settings when set. The agent mount is a filtered copy with those two +keys stripped. + +### Management server + +Sandcat configures connection details; it does not create or lifecycle a +NetBird management server. + +- **Cloud** — omit `--netbird-management-url` (defaults to `https://api.netbird.io`). +- **Existing self-hosted** — pass `--netbird-management-url `. +- **Run a local server yourself** — follow + [`docs/examples/netbird-server/`](../docs/examples/netbird-server/), then + point sandcat at it. + +```bash +# Cloud +sandcat init --agent claude --ide vscode --netbird --name myproject + +# Existing self-hosted management server +sandcat init --agent claude --ide vscode --netbird \ + --netbird-management-url https://netbird.example.com --name myproject +``` + +Interactive `sandcat init --netbird` (when other options are also prompted) +offers cloud vs “I have a server running”. + +## Optional mesh gateway (proxy-peer) + +Sandcat does not create a proxy-peer container. See +[`docs/examples/proxy-peer/`](../docs/examples/proxy-peer/). + ## Directory Structure Each module is contained in its own directory under `cli/libexec/`. diff --git a/cli/lib/composefile.bash b/cli/lib/composefile.bash index c510ea6f..c989edc1 100644 --- a/cli/lib/composefile.bash +++ b/cli/lib/composefile.bash @@ -55,47 +55,92 @@ customize_compose_file() { : "${SANDCAT_MOUNT_IDEA_READONLY:=true}" fi - set_workspace "$compose_file" "$project_name" + # Agent is declared by the included sandcat/compose-agent.yml. Declaring + # it again in compose-all.yml makes Compose reject the project + # ("conflicts with imported resource") — include copies resources into + # the model, it never merges them (same rule as mitmproxy below). Write + # user-facing agent mounts into the included file and re-base relative + # paths one level deeper (sandcat/ → project root needs ../..). + local agent_compose="$compose_dir/sandcat/compose-agent.yml" + local agent_target="$compose_file" + local project_rel=".." + if [[ -f "$agent_compose" ]] + then + agent_target="$agent_compose" + project_rel="../.." + fi - add_settings_volume "$compose_file" "$settings_file" + set_workspace "$agent_target" "$project_name" "$project_rel" + + # mitmproxy is declared by the included sandcat/compose-proxy.yml, so the + # mount has to go there. Declaring the service here as well makes Compose + # reject the project ("conflicts with imported resource") — include copies + # resources into the model, it never merges them. The extra ".." re-bases + # the path on the included file's own directory, which is how Compose + # resolves relative paths inside it. + local proxy_compose="$compose_dir/sandcat/compose-proxy.yml" + if [[ -f "$proxy_compose" ]] + then + add_settings_volume "$proxy_compose" "../$settings_file" + fi case "$agent" in claude) - add_claude_config_volumes "$compose_file" "${SANDCAT_MOUNT_CLAUDE_CONFIG:=true}" + add_claude_config_volumes "$agent_target" "${SANDCAT_MOUNT_CLAUDE_CONFIG:=true}" ;; codex) - add_codex_config_volumes "$compose_file" "${SANDCAT_MOUNT_CODEX_CONFIG:=true}" + add_codex_config_volumes "$agent_target" "${SANDCAT_MOUNT_CODEX_CONFIG:=true}" ;; copilot) - add_copilot_config_volumes "$compose_file" "${SANDCAT_MOUNT_COPILOT_CONFIG:=true}" + add_copilot_config_volumes "$agent_target" "${SANDCAT_MOUNT_COPILOT_CONFIG:=true}" ;; cursor) - add_cursor_config_volumes "$compose_file" "${SANDCAT_MOUNT_CURSOR_CONFIG:=true}" "$project_name" + add_cursor_config_volumes "$agent_target" "${SANDCAT_MOUNT_CURSOR_CONFIG:=true}" "$project_name" ;; esac - add_git_readonly_volume "$compose_file" "${SANDCAT_MOUNT_GIT_READONLY:=false}" - add_idea_readonly_volume "$compose_file" "${SANDCAT_MOUNT_IDEA_READONLY:-false}" + add_git_readonly_volume "$agent_target" "${SANDCAT_MOUNT_GIT_READONLY:=false}" "$project_rel" + add_idea_readonly_volume "$agent_target" "${SANDCAT_MOUNT_IDEA_READONLY:-false}" "$project_rel" local -a stacks_arr=() if [[ -n "$stacks" ]]; then read -ra stacks_arr <<< "$stacks" fi - add_shared_cache_volumes "$compose_file" "${SANDCAT_MOUNT_SHARED_CACHE:=true}" "${stacks_arr[@]+"${stacks_arr[@]}"}" + add_shared_cache_volumes "$agent_target" "${SANDCAT_MOUNT_SHARED_CACHE:=true}" "${stacks_arr[@]+"${stacks_arr[@]}"}" if [[ $ide == "jetbrains" ]] then - add_jetbrains_capabilities "$compose_file" + add_jetbrains_capabilities "$agent_target" fi - # Remove blank lines between volume entries/comments. - # yq inserts blank lines between foot comments and the next sibling. - # When a blank line is followed by an indented line, strip the blank line - # via substitution to keep the indented line intact. + strip_entry_blank_lines "$agent_target" + if [[ -f "$proxy_compose" ]] + then + strip_entry_blank_lines "$proxy_compose" + fi +} + +# Removes blank lines between volume entries/comments. +# yq inserts blank lines between foot comments and the next sibling. +# When a blank line is followed by an indented line, strip the blank line +# via substitution to keep the indented line intact. +# Args: +# $1 - Path to the Docker Compose file +strip_entry_blank_lines() { + local compose_file=$1 + sed '/^$/{ N; /^\n[[:space:]]/{ s/^\n//; }; }' "$compose_file" > "$compose_file.tmp" && mv "$compose_file.tmp" "$compose_file" } # Configures the mitmproxy image and secret-backend environment for compose-proxy.yml. +# +# Environment entries are appended (idempotent), never assigned as a fresh array — +# otherwise a later call would wipe NetBird passthrough vars such as NB_SETUP_KEY +# that enable_netbird() injects for the Dockerfile.mitmproxy variant. +# +# When mitmproxy already builds from Dockerfile.mitmproxy, the stock/op/pass +# image pin is skipped so NetBird enrollment keeps its custom entrypoint. +# # Args: # $1 - Path to the compose-proxy.yml file # $2 - Secret provider: none | 1password | protonpass @@ -103,30 +148,50 @@ apply_secret_provider() { require yq local compose_file=$1 local provider=${2:-none} + local token_env="" + local provider_image="" case "$provider" in none) return 0 ;; 1password) - mitm_ver="$SCT_MITMPROXY_VERSION" yq -i ' - .services.mitmproxy.image = "ghcr.io/virtuslab/sandcat-mitmproxy-op:" + env(mitm_ver) | - .services.mitmproxy.environment = ["OP_SERVICE_ACCOUNT_TOKEN"] - ' "$compose_file" + token_env="OP_SERVICE_ACCOUNT_TOKEN" + provider_image="ghcr.io/virtuslab/sandcat-mitmproxy-op:${SCT_MITMPROXY_VERSION}" ;; protonpass) - mitm_ver="$SCT_MITMPROXY_VERSION" yq -i ' - .services.mitmproxy.image = "ghcr.io/virtuslab/sandcat-mitmproxy-pass:" + env(mitm_ver) | - .services.mitmproxy.environment = ["PROTON_PASS_PERSONAL_ACCESS_TOKEN"] - ' "$compose_file" + token_env="PROTON_PASS_PERSONAL_ACCESS_TOKEN" + provider_image="ghcr.io/virtuslab/sandcat-mitmproxy-pass:${SCT_MITMPROXY_VERSION}" ;; *) echo "Unknown secret provider: $provider" >&2 return 1 ;; esac -} + local dockerfile + dockerfile=$(yq -r '.services.mitmproxy.build.dockerfile // ""' "$compose_file") + if [[ "$dockerfile" == "Dockerfile.mitmproxy" ]]; then + # NetBird already replaced image: with a build. Pinning image: here would + # be ignored by compose and the provider CLI would be missing from the + # built image, so pass the provider variant in as the build base instead. + provider_image="$provider_image" yq -i ' + .services.mitmproxy.build.args.BASE_IMAGE = env(provider_image) + ' "$compose_file" + else + provider_image="$provider_image" yq -i ' + .services.mitmproxy.image = env(provider_image) + ' "$compose_file" + fi + + local has_token + has_token=$(yq "[(.services.mitmproxy.environment // [])[] | select(. == \"$token_env\")] | length" "$compose_file") + if [[ "$has_token" -eq 0 ]]; then + token_env="$token_env" yq -i ' + .services.mitmproxy.environment = ((.services.mitmproxy.environment // []) + [env(token_env)]) + ' "$compose_file" + fi +} # Switches the mitmproxy service from web UI to console (mitmdump) mode. # Replaces the mitmweb command with mitmdump, strips mitmweb-only flags # (--web-host and --set web_password), and removes the web UI port. @@ -217,6 +282,66 @@ add_volume_foot_comment() { add_foot_comment "$compose_file" ".services.agent.volumes" "$comment" } +# Adds one or more agent volume entries in a single yq -i. +# Args: +# $1 - Path to the Docker Compose file +# $2 - true to add as active entries, false to add as foot comments +# $3.. - Repeating (volume_entry, comment) pairs. Comment may be empty. +add_volume_entries() { + require yq + local compose_file=$1 + local active=$2 + shift 2 + + [[ $# -gt 0 ]] || return 0 + + local -a entries=() comments=() + while [[ $# -gt 0 ]]; do + entries+=("$1") + shift + if [[ $# -gt 0 ]]; then + comments+=("$1") + shift + else + comments+=("") + fi + done + + if [[ $active != "true" ]]; then + local foot="" i + for i in "${!entries[@]}"; do + if [[ -n "${comments[$i]}" ]]; then + foot+="${comments[$i]}"$'\n'"- ${entries[$i]}"$'\n' + else + foot+="- ${entries[$i]}"$'\n' + fi + done + add_volume_foot_comment "$compose_file" "${foot%$'\n'}" + return + fi + + local n=${#entries[@]} + local expr='.services.agent.volumes += [' + local i varname from_end + for i in "${!entries[@]}"; do + varname="SCT_VOL_${i}" + export "$varname=${entries[$i]}" + expr+="env(${varname})," + done + expr="${expr%,}]" + for i in "${!entries[@]}"; do + [[ -n "${comments[$i]}" ]] || continue + varname="SCT_VOLC_${i}" + export "$varname=${comments[$i]}" + from_end=$((n - i)) + expr+=" | (.services.agent.volumes | .[-${from_end}]) head_comment = strenv(${varname})" + done + yq -i "$expr" "$compose_file" + for i in "${!entries[@]}"; do + unset -v "SCT_VOL_${i}" "SCT_VOLC_${i}" + done +} + # Adds a volume entry to the agent service, either as active or commented. # Args: # $1 - Path to the Docker Compose file @@ -224,29 +349,12 @@ add_volume_foot_comment() { # $3 - true to add as active entry, false to add as comment # $4 - Optional description comment add_volume_entry() { - require yq local compose_file=$1 local volume_entry=$2 local active=$3 local comment=${4:-} - if [[ $active == "true" ]] - then - volume_entry="$volume_entry" yq -i \ - '.services.agent.volumes += [env(volume_entry)]' "$compose_file" - if [[ -n $comment ]] - then - comment="$comment" yq -i \ - '(.services.agent.volumes | .[-1]) head_comment = strenv(comment)' "$compose_file" - fi - else - if [[ -n $comment ]] - then - add_volume_foot_comment "$compose_file" "$comment"$'\n'"- $volume_entry" - else - add_volume_foot_comment "$compose_file" "- $volume_entry" - fi - fi + add_volume_entries "$compose_file" "$active" "$volume_entry" "$comment" } # Adds Claude config volume mounts to the agent service. @@ -258,11 +366,10 @@ add_claude_config_volumes() { local active=${2:-true} # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.claude/CLAUDE.md:/home/vscode/.claude/CLAUDE.md:ro' "$active" 'Host Claude config (optional)' - # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.claude/agents:/home/vscode/.claude/agents:ro' "$active" - # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.claude/commands:/home/vscode/.claude/commands:ro' "$active" + add_volume_entries "$compose_file" "$active" \ + '${HOME}/.claude/CLAUDE.md:/home/vscode/.claude/CLAUDE.md:ro' 'Host Claude config (optional)' \ + '${HOME}/.claude/agents:/home/vscode/.claude/agents:ro' '' \ + '${HOME}/.claude/commands:/home/vscode/.claude/commands:ro' '' } # Adds Codex config volume mounts to the agent service. @@ -274,11 +381,10 @@ add_codex_config_volumes() { local active=${2:-true} # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.codex/AGENTS.md:/home/vscode/.codex-host/AGENTS.md:ro' "$active" 'Host Codex config (optional) — copied into writable ~/.codex/AGENTS.md by app-user-init.sh so rtk can patch it' - # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.codex/skills:/home/vscode/.codex/skills:ro' "$active" - # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.codex/commands:/home/vscode/.codex/commands:ro' "$active" + add_volume_entries "$compose_file" "$active" \ + '${HOME}/.codex/AGENTS.md:/home/vscode/.codex-host/AGENTS.md:ro' 'Host Codex config (optional) — copied into writable ~/.codex/AGENTS.md by app-user-init.sh so rtk can patch it' \ + '${HOME}/.codex/skills:/home/vscode/.codex/skills:ro' '' \ + '${HOME}/.codex/commands:/home/vscode/.codex/commands:ro' '' } # Adds Copilot config volume mounts to the agent service. @@ -289,14 +395,14 @@ add_copilot_config_volumes() { local compose_file=$1 local active=${2:-true} - # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.copilot/mcp-config.json:/home/vscode/.copilot/mcp-config.json:ro' "$active" 'Host Copilot MCP config (optional)' # session-state is read-write: Copilot CLI persists chat session events # there (events.jsonl per session UUID). Read-only mount fails with # EROFS on every prompt. Same trust posture as other host-mounted agent # data — user chose to bind-mount, we honor read+write. # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.copilot/session-state:/home/vscode/.copilot/session-state:rw' "$active" + add_volume_entries "$compose_file" "$active" \ + '${HOME}/.copilot/mcp-config.json:/home/vscode/.copilot/mcp-config.json:ro' 'Host Copilot MCP config (optional)' \ + '${HOME}/.copilot/session-state:/home/vscode/.copilot/session-state:rw' '' } # Adds Cursor config volume mounts to the agent service. @@ -311,29 +417,21 @@ add_cursor_config_volumes() { local project_id project_id=$(sct_cursor_workspace_project_id "$project_name") - # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.cursor/AGENTS.md:/home/vscode/.cursor/AGENTS.md:ro' "$active" 'Host Cursor config (optional)' - # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.cursor/rules:/home/vscode/.cursor/rules:ro' "$active" - # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.cursor/skills:/home/vscode/.cursor/skills:ro' "$active" - # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.cursor/commands:/home/vscode/.cursor/commands:ro' "$active" - # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.cursor/hooks.json:/home/vscode/.cursor/hooks.json:ro' "$active" - # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.cursor/hooks:/home/vscode/.cursor/hooks:ro' "$active" - # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.cursor/agents:/home/vscode/.cursor/agents:ro' "$active" - # shellcheck disable=SC2016 - add_volume_entry "$compose_file" '${HOME}/.cursor/mcp.json:/home/vscode/.cursor/mcp.json:ro' "$active" # Workspace-scoped runtime state — only this sandcat project's Cursor # projects// tree is mounted (agent transcripts, terminals, etc.). # chats/, plugins/, and subagents/ remain in agent-home to avoid leaking # other workspaces' data from the host profile. - add_volume_entry "$compose_file" \ - "\${HOME}/.cursor/projects/${project_id}:/home/vscode/.cursor/projects/${project_id}" \ - "$active" + # shellcheck disable=SC2016 + add_volume_entries "$compose_file" "$active" \ + '${HOME}/.cursor/AGENTS.md:/home/vscode/.cursor/AGENTS.md:ro' 'Host Cursor config (optional)' \ + '${HOME}/.cursor/rules:/home/vscode/.cursor/rules:ro' '' \ + '${HOME}/.cursor/skills:/home/vscode/.cursor/skills:ro' '' \ + '${HOME}/.cursor/commands:/home/vscode/.cursor/commands:ro' '' \ + '${HOME}/.cursor/hooks.json:/home/vscode/.cursor/hooks.json:ro' '' \ + '${HOME}/.cursor/hooks:/home/vscode/.cursor/hooks:ro' '' \ + '${HOME}/.cursor/agents:/home/vscode/.cursor/agents:ro' '' \ + '${HOME}/.cursor/mcp.json:/home/vscode/.cursor/mcp.json:ro' '' \ + "\${HOME}/.cursor/projects/${project_id}:/home/vscode/.cursor/projects/${project_id}" '' } @@ -341,22 +439,26 @@ add_cursor_config_volumes() { # Args: # $1 - Path to the Docker Compose file # $2 - true to add as active, false to add as comment +# $3 - Relative path from that compose file to the project root (default: ..) add_git_readonly_volume() { local compose_file=$1 local active=${2:-true} + local project_rel=${3:-..} - add_volume_entry "$compose_file" '../.git:/workspace/.git:ro' "$active" 'Read-only Git directory' + add_volume_entry "$compose_file" "${project_rel}/.git:/workspace/.git:ro" "$active" 'Read-only Git directory' } # Adds .idea directory mount as read-only to the agent service. # Args: # $1 - Path to the Docker Compose file # $2 - true to add as active, false to add as comment +# $3 - Relative path from that compose file to the project root (default: ..) add_idea_readonly_volume() { local compose_file=$1 local active=${2:-true} + local project_rel=${3:-..} - add_volume_entry "$compose_file" '../.idea:/workspace/.idea:ro' "$active" 'Read-only IntelliJ IDEA project directory' + add_volume_entry "$compose_file" "${project_rel}/.idea:/workspace/.idea:ro" "$active" 'Read-only IntelliJ IDEA project directory' } # Adds shared-cache mount entries + top-level external volume declarations @@ -405,47 +507,73 @@ add_shared_cache_volumes() { [[ ${#entries[@]} -eq 0 ]] && return 0 + local -a vol_args=() local entry name path first=true for entry in "${entries[@]}"; do name=${entry%%:*} path=${entry#*:} if [[ $first == true ]]; then first=false - add_volume_entry "$compose_file" "$name:$path" "$active" 'Shared dependency caches for the selected stacks (SANDCAT_MOUNT_SHARED_CACHE=false to disable)' + vol_args+=("$name:$path" 'Shared dependency caches for the selected stacks (SANDCAT_MOUNT_SHARED_CACHE=false to disable)') else - add_volume_entry "$compose_file" "$name:$path" "$active" + vol_args+=("$name:$path" "") fi done + add_volume_entries "$compose_file" "$active" "${vol_args[@]}" # Declare each cache as an external volume with a stable host-scoped # name so multiple compose projects reference the same physical volume. if [[ $active == "true" ]]; then + local expr='.' i=0 varname for entry in "${entries[@]}"; do name=${entry%%:*} - name="$name" yq -i \ - '.volumes[env(name)] = {"external": true, "name": env(name)}' \ - "$compose_file" + varname="SCT_CACHENAME_${i}" + export "$varname=$name" + expr+=" | .volumes[env(${varname})] = {\"external\": true, \"name\": env(${varname})}" + i=$((i + 1)) + done + yq -i "$expr" "$compose_file" + i=0 + for entry in "${entries[@]}"; do + unset -v "SCT_CACHENAME_${i}" + i=$((i + 1)) done fi } # Sets the working directory and adds workspace volume mounts for the agent service. # Args: -# $1 - Path to the Docker Compose file +# $1 - Path to the Docker Compose file that declares services.agent # $2 - Project name (used to construct /workspaces/) +# $3 - Relative path from that compose file to the project root (default: ..) set_workspace() { require yq local compose_file=$1 local project_name=$2 + local project_rel=${3:-..} local workspace="/workspaces/$project_name" - - project_name="$project_name" yq -i \ - '.services.agent.working_dir = "/workspaces/" + env(project_name)' "$compose_file" - - add_volume_entry "$compose_file" "..:${workspace}" "true" "Mount the project's code" - add_volume_entry "$compose_file" "../.devcontainer:${workspace}/.devcontainer:ro" "true" "Read-only devcontainer directory" - add_volume_entry "$compose_file" "../.sandcat:${workspace}/.sandcat:ro" "true" "Read-only settings directory" + local vol_code="${project_rel}:${workspace}" + local vol_dc="${project_rel}/.devcontainer:${workspace}/.devcontainer:ro" + local vol_sandcat="\${SANDCAT_AGENT_SANDCAT:?}:${workspace}/.sandcat:ro" + local c_code="Mount the project's code" + local c_dc="Read-only devcontainer directory" + local c_sandcat='Filtered settings copy (SANDCAT_AGENT_SANDCAT; never the live .sandcat)' + + project_name="$project_name" \ + vol_code="$vol_code" \ + vol_dc="$vol_dc" \ + vol_sandcat="$vol_sandcat" \ + c_code="$c_code" \ + c_dc="$c_dc" \ + c_sandcat="$c_sandcat" \ + yq -i ' + .services.agent.working_dir = "/workspaces/" + env(project_name) | + .services.agent.volumes += [env(vol_code), env(vol_dc), env(vol_sandcat)] | + (.services.agent.volumes | .[-3]) head_comment = strenv(c_code) | + (.services.agent.volumes | .[-2]) head_comment = strenv(c_dc) | + (.services.agent.volumes | .[-1]) head_comment = strenv(c_sandcat) + ' "$compose_file" } # Adds JetBrains-specific capabilities to the agent service. @@ -455,10 +583,12 @@ add_jetbrains_capabilities() { require yq local compose_file=$1 - yq -i '.services.agent.cap_add += ["DAC_OVERRIDE", "CHOWN", "FOWNER"]' "$compose_file" - yq -i '(.services.agent.cap_add[] | select(. == "DAC_OVERRIDE")) head_comment = "JetBrains IDE: bypass file permission checks on mounted volumes"' "$compose_file" - yq -i '(.services.agent.cap_add[] | select(. == "CHOWN")) head_comment = "JetBrains IDE: change ownership of IDE cache and state files"' "$compose_file" - yq -i '(.services.agent.cap_add[] | select(. == "FOWNER")) head_comment = "JetBrains IDE: bypass ownership checks on IDE-managed files"' "$compose_file" + yq -i ' + .services.agent.cap_add += ["DAC_OVERRIDE", "CHOWN", "FOWNER"] | + (.services.agent.cap_add[] | select(. == "DAC_OVERRIDE")) head_comment = "JetBrains IDE: bypass file permission checks on mounted volumes" | + (.services.agent.cap_add[] | select(. == "CHOWN")) head_comment = "JetBrains IDE: change ownership of IDE cache and state files" | + (.services.agent.cap_add[] | select(. == "FOWNER")) head_comment = "JetBrains IDE: bypass ownership checks on IDE-managed files" + ' "$compose_file" } # Reads the merged `upstream_ca_bundles` list from user settings @@ -590,3 +720,194 @@ apply_upstream_ca_bundles() { '.services.mitmproxy.entrypoint = ["/bin/sh", "-c", strenv(new_entrypoint), "sh"]' \ "$compose_file" } + +# Injects NetBird version and per-arch checksum build args into a service's +# compose build section, sourced from netbird.env (sibling to the compose file). +# Args: +# $1 - Path to compose file +# $2 - Service name (default: wg-client) +apply_netbird_build_args() { + require yq + local compose_file=$1 + local service_name=${2:-wg-client} + local netbird_env + netbird_env="$(dirname "$compose_file")/netbird.env" + + if [[ ! -f "$netbird_env" ]]; then + echo "netbird.env not found beside compose file: $netbird_env" >&2 + return 1 + fi + + # shellcheck disable=SC1090 + source "$netbird_env" + + : "${NETBIRD_VERSION:?NETBIRD_VERSION missing from $netbird_env}" + : "${NETBIRD_SHA256_AMD64:?NETBIRD_SHA256_AMD64 missing from $netbird_env}" + : "${NETBIRD_SHA256_ARM64:?NETBIRD_SHA256_ARM64 missing from $netbird_env}" + + yq -i " + .services.\"${service_name}\".build.args.NETBIRD_VERSION = \"${NETBIRD_VERSION}\" | + .services.\"${service_name}\".build.args.NETBIRD_SHA256_AMD64 = \"${NETBIRD_SHA256_AMD64}\" | + .services.\"${service_name}\".build.args.NETBIRD_SHA256_ARM64 = \"${NETBIRD_SHA256_ARM64}\" + " "$compose_file" +} + +# Wires NetBird enrollment into the mitmproxy service in compose-proxy.yml. +# mitmproxy is the sole NetBird mesh participant in the agent stack; wg-client +# remains a pure tunnel shim (wg0 only). Traffic always flows: +# agent → wg0 (wg-client) → mitmproxy L7 inspect → internet or wt0 mesh. +# +# When NetBird is enabled, this function: +# 1. Switches mitmproxy from the stock image to a build using Dockerfile.mitmproxy +# (which installs the pinned NetBird binary and mitmproxy-init.sh entrypoint). +# 2. Removes the compose-level entrypoint override. mitmproxy-init.sh +# restores master's CA publish, dns.conf clear, and docker-entrypoint.sh. +# 3. Adds cap_add: [NET_ADMIN] and the WireGuard src_valid_mark sysctl. +# 4. Adds NB_SETUP_KEY (and optionally NB_MANAGEMENT_URL) to the environment. +# 5. Injects NetBird build args (version + per-arch checksums) from netbird.env. +# 6. Adds extra_hosts host.docker.internal:172.17.0.1 so STUN hits docker0 +# (Colima host-gateway is the LAN IP; UDP hairpin fails). Do not set +# server.stuns — that disables the embedded listener. +# +# Args: +# $1 - Path to compose-proxy.yml +# $2 - Optional NetBird management server URL +# $3 - NetBird peer hostname for mitmproxy (required for project-scoped naming) +enable_netbird() { + require yq + # shellcheck source=netbird.bash + source "$SCT_LIBDIR/netbird.bash" + + local compose_file=$1 + local netbird_management_url=${2:-} + local peer_name=${3:-} + local enrollment_url + + [[ -n "$peer_name" ]] || { + echo "enable_netbird: peer name (\$3) is required (e.g. myapp-sandbox-proxy)" >&2 + return 1 + } + + mkdir -p "$(dirname "$compose_file")/scripts" + cp "$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/netbird-peer-lifecycle.sh" \ + "$(dirname "$compose_file")/scripts/" + + # Switch mitmproxy from image: to build: using Dockerfile.mitmproxy. + # Idempotent: skip if a build section is already present. + local has_build + has_build=$(yq '(.services.mitmproxy | has("build"))' "$compose_file") + if [[ "$has_build" != "true" ]]; then + # Carry the pinned image over as the build base. apply_secret_provider + # runs first, so this is where a provider variant (pass/op) would + # otherwise be dropped, taking pass-cli / op with it. + local base_image + base_image=$(yq -r '.services.mitmproxy.image // ""' "$compose_file") + yq -i ' + del(.services.mitmproxy.image) | + del(.services.mitmproxy.entrypoint) | + .services.mitmproxy.build = {"context": ".", "dockerfile": "Dockerfile.mitmproxy"} + ' "$compose_file" + if [[ -n "$base_image" ]]; then + base_image="$base_image" yq -i ' + .services.mitmproxy.build.args.BASE_IMAGE = env(base_image) + ' "$compose_file" + fi + fi + + # Caps, sysctls, extra_hosts, env passthrough, and the state volume in one + # idempotent write. Each field is rewritten as "existing minus this key, + # plus this key" so a second enable_netbird does not duplicate entries. + # src_valid_mark is matched by name only; `== '...=1'` segfaults some yq + # builds. extra_hosts always use docker0: Colima's host-gateway is the VM + # LAN IP and UDP hairpin to STUN times out. Do not set server.stuns — + # that disables the embedded listener. + # No jq `if`/`then`/`end`: mikefarah yq's lexer rejects it here. + peer_name="$peer_name" yq -i ' + .services.mitmproxy.cap_add = ( + ((.services.mitmproxy.cap_add // []) | map(select(. != "NET_ADMIN"))) + + ["NET_ADMIN"] + ) | + .services.mitmproxy.sysctls = ( + ((.services.mitmproxy.sysctls // []) | map(select(test("src_valid_mark") | not))) + + ["net.ipv4.conf.all.src_valid_mark=1"] + ) | + .services.mitmproxy.extra_hosts = ( + ((.services.mitmproxy.extra_hosts // []) + | map(select(test("^host.docker.internal:") | not))) + + ["host.docker.internal:172.17.0.1"] + ) | + .services.mitmproxy.environment = ( + ((.services.mitmproxy.environment // []) + | map(select( + . != "NB_SETUP_KEY" + and . != "NB_API_TOKEN" + and (test("^NB_PEER_NAME=") | not) + )) + + ["NB_SETUP_KEY", "NB_API_TOKEN", "NB_PEER_NAME=" + env(peer_name)] + ) + ) | + .services."wg-client".environment = ( + (.services."wg-client".environment // []) + | map(select( + . != "NB_SETUP_KEY" + and (test("^NB_MANAGEMENT_URL=") | not) + and (test("^NB_USE_LEGACY_ROUTING=") | not) + )) + ) | + .services.mitmproxy.volumes = ( + ((.services.mitmproxy.volumes // []) + | map(select(. != "netbird-mitmproxy-state:/var/lib/netbird"))) + + ["netbird-mitmproxy-state:/var/lib/netbird"] + ) | + .volumes."netbird-mitmproxy-state" = (.volumes."netbird-mitmproxy-state" // {}) + ' "$compose_file" + + # Default mesh DNS domain unless the compose file already sets one. + local has_dns_domain + has_dns_domain=$(yq '[(.services.mitmproxy.environment // [])[] | select(test("^NETBIRD_DNS_DOMAIN="))] | length' "$compose_file") + if [[ "$has_dns_domain" -eq 0 ]]; then + yq -i '.services.mitmproxy.environment += ["NETBIRD_DNS_DOMAIN=netbird.selfhosted"]' "$compose_file" + fi + + # Remove an empty environment block left after stripping the last entries. + local wg_env_len + wg_env_len=$(yq '[.services."wg-client".environment[]?] | length' "$compose_file") + if [[ "$wg_env_len" -eq 0 ]]; then + yq -i 'del(.services."wg-client".environment)' "$compose_file" + fi + + + if [[ -n "$netbird_management_url" ]]; then + enrollment_url=$(netbird_enrollment_management_url_from "$netbird_management_url") + if [[ -n "$enrollment_url" ]]; then + enrollment_url="$enrollment_url" \ + yq -i ' + .services.mitmproxy.environment = ( + (.services.mitmproxy.environment // []) + | map(select(test("^NB_MANAGEMENT_URL=") | not)) + ) + ["NB_MANAGEMENT_URL=" + env(enrollment_url)] + ' "$compose_file" + if netbird_enrollment_url_uses_host_bypass "$enrollment_url"; then + yq -i ' + .services.mitmproxy.environment = ( + (.services.mitmproxy.environment // []) + | map(select(test("^NB_USE_LEGACY_ROUTING=") | not)) + ) + ["NB_USE_LEGACY_ROUTING=true"] + ' "$compose_file" + fi + else + # localhost/127.0.0.1 resolves to the container itself, so no + # NB_MANAGEMENT_URL is emitted. netbird then falls back to its + # api.netbird.io default and rejects a self-hosted setup key with + # "invalid setup-key" — warn rather than fail silently. + echo "mitmproxy has no NB_MANAGEMENT_URL: $netbird_management_url is not reachable from inside the container." | warning + echo " NetBird would enroll against the cloud default (https://api.netbird.io) and reject a self-hosted setup key." | warning + echo " Set netbird_enrollment_management_url to a container-reachable address in $(sct_home)/settings.json:" | warning + echo " \"netbird_enrollment_management_url\": \"http://:33073\"" | warning + echo " Then re-run: sandcat init --netbird ..." | warning + fi + fi + + # Inject pinned NetBird build args (version + per-arch checksums) from netbird.env. + apply_netbird_build_args "$compose_file" "mitmproxy" +} diff --git a/cli/lib/devcontainer.bash b/cli/lib/devcontainer.bash index 1b775c72..47acdad9 100644 --- a/cli/lib/devcontainer.bash +++ b/cli/lib/devcontainer.bash @@ -2,6 +2,8 @@ # shellcheck source=constants.bash source "$SCT_LIBDIR/constants.bash" +# shellcheck source=require.bash +source "$SCT_LIBDIR/require.bash" # shellcheck source=stacks.bash source "$SCT_LIBDIR/stacks.bash" # shellcheck source=agents.bash @@ -230,31 +232,35 @@ apply_inline_placeholders() { } # Adds stack-contributed environment variables (e.g. uv's TLS config for the -# python stack) to services.agent.environment in compose-all.yml. +# python stack) to services.agent.environment in sandcat/compose-agent.yml. # Args: -# $1 - Path to compose-all.yml +# $1 - Path to compose-all.yml (agent file is resolved beside it) # $@ - Stack names (remaining args) customize_compose_stack_environment() { local compose_file=$1 shift + local agent_compose + agent_compose="$(dirname "$compose_file")/sandcat/compose-agent.yml" + [[ -f "$agent_compose" ]] || agent_compose="$compose_file" + local entries="" stack env for stack in "$@"; do env=$(stack_env_entries "$stack") [[ -n "$env" ]] && entries="${entries}${env}"$'\n' done - merge_compose_agent_environment "$compose_file" "$entries" + merge_compose_agent_environment "$agent_compose" "$entries" } # Merges KEY=value environment entries into services.agent.environment in -# compose-all.yml. Appends to any entries already present (rather than -# overwriting) so agent- and stack-contributed variables coexist regardless -# of call order. Building the array structurally avoids fragile -# line-counting in compose-all.yml. No-op when passed no entries — compose +# the compose file that declares the agent service. Appends to any entries +# already present (rather than overwriting) so agent- and stack-contributed +# variables coexist regardless of call order. Building the array structurally +# avoids fragile line-counting. No-op when passed no entries — compose # rejects `environment: {}`. # Args: -# $1 - Path to compose-all.yml +# $1 - Path to the compose file that declares services.agent # $2 - Newline-separated "KEY=value" entries (empty lines ignored) merge_compose_agent_environment() { local compose_file=$1 @@ -323,7 +329,9 @@ customize_agent_templates() { "__AGENT_EXTENSION__" "$extension_replacement" \ "__AGENT_SETTINGS__" "$settings_block" - merge_compose_agent_environment "$devcontainer_dir/compose-all.yml" "$environment_entries" + local agent_compose="$devcontainer_dir/sandcat/compose-agent.yml" + [[ -f "$agent_compose" ]] || agent_compose="$devcontainer_dir/compose-all.yml" + merge_compose_agent_environment "$agent_compose" "$environment_entries" apply_template_placeholders \ "$devcontainer_dir/Dockerfile.app" \ diff --git a/cli/lib/netbird.bash b/cli/lib/netbird.bash new file mode 100644 index 00000000..5ab6095f --- /dev/null +++ b/cli/lib/netbird.bash @@ -0,0 +1,294 @@ +#!/usr/bin/env bash + +# shellcheck source=constants.bash +source "${BASH_SOURCE%/*}/constants.bash" +# shellcheck source=path.bash +source "${BASH_SOURCE%/*}/path.bash" +# shellcheck source=logging.bash +source "${BASH_SOURCE%/*}/logging.bash" +# shellcheck source=require.bash +source "${BASH_SOURCE%/*}/require.bash" + +# Reads a NetBird setting from sandcat settings layers. Later layers win when +# non-empty (user < project < project local), matching mitmproxy addon precedence. +# netbird_api_token and netbird_enrollment_key skip project layers. +# Args: +# $1 - settings key (e.g. netbird_api_token) +netbird_read_setting() { + local key=$1 + + local value="" + local file layer_value repo_root + + local -a layers=() + layers+=("$(sct_home)/settings.json") + if ! netbird_setting_is_secret_key "$key"; then + if repo_root=$(netbird_project_root 2>/dev/null); then + layers+=("$repo_root/$SCT_PROJECT_DIR/settings.json") + layers+=("$repo_root/$SCT_PROJECT_DIR/settings.local.json") + fi + fi + + for file in "${layers[@]}"; do + [[ -f "$file" ]] || continue + grep -q "\"$key\"" "$file" 2>/dev/null || continue + require yq || return 1 + layer_value=$(yq -r ".$key // \"\"" "$file") + if [[ -n "$layer_value" ]]; then + value="$layer_value" + fi + done + + printf '%s' "$value" +} + +# Enrollment key and API token are operator credentials. Project files are +# cloned with the repo, so those two keys are read from user settings only. +netbird_setting_is_secret_key() { + local key=$1 + [[ "$key" == "netbird_api_token" || "$key" == "netbird_enrollment_key" ]] +} + +# Settings lookup start dir. `sandcat init --path other` exports +# SANDCAT_PROJECT_ROOT so we do not read $PWD's .sandcat. +netbird_project_root() { + if [[ -n "${SANDCAT_PROJECT_ROOT:-}" ]]; then + find_repo_root "$SANDCAT_PROJECT_ROOT" + else + find_repo_root + fi +} + +# Same layers as netbird_read_setting, but each value is JSON (yq -o json). +# Later non-null layers win. Prints `null` when unset. Keeps JSON string quotes +# so digit-only or boolean-looking tokens are not re-parsed as YAML scalars. +netbird_read_setting_json() { + local key=$1 + local value="null" + local file layer_value repo_root + local -a layers=() + layers+=("$(sct_home)/settings.json") + if ! netbird_setting_is_secret_key "$key"; then + if repo_root=$(netbird_project_root 2>/dev/null); then + layers+=("$repo_root/$SCT_PROJECT_DIR/settings.json") + layers+=("$repo_root/$SCT_PROJECT_DIR/settings.local.json") + fi + fi + for file in "${layers[@]}"; do + [[ -f "$file" ]] || continue + grep -q "\"$key\"" "$file" 2>/dev/null || continue + require yq || return 1 + layer_value=$(yq -o json ".$key" "$file") + layer_value=${layer_value%$'\n'} + if [[ "$layer_value" != "null" ]]; then + value="$layer_value" + fi + done + printf '%s' "$value" +} + +# Args: $1 JSON (string, object, empty, or null) +# Object must have exactly one of value, op, pass. +netbird_flatten_secret_setting() { + local json=${1-} + if [[ -z "$json" || "$json" == "null" ]]; then + printf '' + return 0 + fi + require yq || return 1 + # Parse as JSON so quoted digit-only / boolean-looking strings stay strings. + # YAML input would type 0123456789 as !!float and abort compose/run. + local typ + typ=$(printf '%s' "$json" | yq -p json -r 'type') + case "$typ" in + string | !!str | number | !!float | !!int | bool | !!bool) + printf '%s' "$(printf '%s' "$json" | yq -p json -r '.')" + return 0 + ;; + object | !!map) + local n + n=$(printf '%s' "$json" | yq -p json '[.value, .op, .pass] | map(select(. != null)) | length') + if [[ "$n" != "1" ]]; then + echo "netbird secret must specify exactly one of 'value', 'op', or 'pass'" >&2 + return 1 + fi + printf '%s' "$(printf '%s' "$json" | yq -p json -r '.value // .op // .pass')" + return 0 + ;; + *) + echo "netbird secret must be a string or object" >&2 + return 1 + ;; + esac +} + +# Always writes netbird_peer_name_proxy as {project}-proxy. +# Committed settings must not choose the name: replace DELETEs that peer. +# Args: +# $1 - path to a JSON settings file (usually .sandcat/settings.json) +# $2 - compose project name (e.g. myapp-sandbox) +netbird_ensure_peer_name_settings() { + local settings_file=$1 + local project_name=$2 + require yq + + mkdir -p "$(dirname "$settings_file")" + [[ -f "$settings_file" ]] || printf '%s\n' '{}' >"$settings_file" + + local proxy + proxy=$(printf '%s-proxy' "$project_name") + + proxy="$proxy" yq -i -o json '.netbird_peer_name_proxy = env(proxy)' "$settings_file" +} + +# Copies project .sandcat to dest and drops enrollment/API secrets so the +# agent bind-mount cannot read them even if they were committed. +# Args: +# $1 - source .sandcat directory +# $2 - destination directory (replaced) +prepare_agent_sandcat_mount() { + local src=$1 + local dest=$2 + + [[ -n "$dest" && "$dest" != "/" ]] || return 1 + rm -rf "$dest" + mkdir -p "$dest" + [[ -d "$src" ]] || return 0 + if ! cp -a "$src/." "$dest/"; then + rm -rf "$dest" + return 1 + fi + + local f + for f in "$dest/settings.json" "$dest/settings.local.json"; do + [[ -s "$f" ]] || continue + if ! require yq || ! yq -i -o json 'del(.netbird_api_token) | del(.netbird_enrollment_key)' "$f"; then + rm -rf "$dest" + return 1 + fi + done +} + +# Prepares a filtered .sandcat copy and exports SANDCAT_AGENT_SANDCAT for compose. +export_agent_sandcat_mount() { + local repo_root src dest + repo_root=$(netbird_project_root 2>/dev/null) || return 0 + src="$repo_root/$SCT_PROJECT_DIR" + dest="$(sct_home)/agent-sandcat/${repo_root//\//_}" + prepare_agent_sandcat_mount "$src" "$dest" || return 1 + write_agent_sandcat_compose_env "$repo_root" "$dest" || return 1 + export SANDCAT_AGENT_SANDCAT="$dest" +} + +# Writes SANDCAT_AGENT_SANDCAT into .devcontainer/.env so Dev Containers +# compose interpolation works (initializeCommand cannot export into compose). +write_agent_sandcat_compose_env() { + local repo_root=$1 + local dest=$2 + local envf tmp + envf="$repo_root/.devcontainer/.env" + [[ -d "$(dirname "$envf")" ]] || return 0 + tmp=$(mktemp) + if [[ -f "$envf" ]]; then + grep -v '^SANDCAT_AGENT_SANDCAT=' "$envf" >"$tmp" || true + fi + printf 'SANDCAT_AGENT_SANDCAT=%s\n' "$dest" >>"$tmp" + chmod 600 "$tmp" + mv "$tmp" "$envf" +} + +# Export NB_SETUP_KEY from settings when not already set in the environment. +# Used before docker compose so wg-client receives the enrollment key on create. +export_netbird_compose_env() { + if [[ -z "${NB_SETUP_KEY:-}" ]]; then + local enrollment_key + enrollment_key=$(netbird_flatten_secret_setting "$(netbird_read_setting_json netbird_enrollment_key)") || return 1 + if [[ -n "$enrollment_key" ]]; then + export NB_SETUP_KEY="$enrollment_key" + fi + fi + if [[ -z "${NB_API_TOKEN:-}" ]]; then + local api_token + api_token=$(netbird_flatten_secret_setting "$(netbird_read_setting_json netbird_api_token)") || return 1 + if [[ -n "$api_token" ]]; then + export NB_API_TOKEN="$api_token" + fi + fi +} + +# Export NB_MANAGEMENT_URL from settings when not already set in environment. +export_netbird_management_url() { + [[ -n "${NB_MANAGEMENT_URL:-}" ]] && return 0 + + local management_url + management_url=$(netbird_read_setting netbird_management_url) + if [[ -n "$management_url" ]]; then + export NB_MANAGEMENT_URL="$management_url" + fi +} + +# Returns the management URL wg-client should use for NetBird enrollment. +# netbird_enrollment_management_url in settings wins when set. Remote URLs pass +# through unchanged. localhost / 127.0.0.1 require an explicit enrollment URL +# (wg-client cannot reach the host via localhost). +# Args: +# $1 - Host-side management URL (e.g. http://localhost:33073) +netbird_enrollment_management_url_from() { + local management_url=$1 + local explicit + + explicit=$(netbird_read_setting netbird_enrollment_management_url) + if [[ -n "$explicit" ]]; then + printf '%s' "$explicit" + return 0 + fi + + [[ -n "$management_url" ]] || return 0 + + if [[ "$management_url" =~ ^https?://(localhost|127\.0\.0\.1)([:/]|$) ]]; then + return 0 + fi + + printf '%s' "$management_url" +} + +# Prints a literal IPv4 address of the Docker host that containers can dial, +# or nothing when detection fails. Literal IPv4 rather than host.docker.internal +# so netbird_enrollment_url_uses_host_bypass matches and management traffic is +# routed off wg0. +netbird_detect_docker_host_ip() { + local ip="" + local iface + + case "$(uname -s)" in + Darwin) + for iface in en0 en1; do + ip=$(ipconfig getifaddr "$iface" 2>/dev/null || true) + if [[ -n "$ip" ]]; then + break + fi + done + ;; + *) + if command -v ip >/dev/null 2>&1; then + # Source address the kernel picks for off-link traffic is the + # host's LAN address, which containers can route to. + ip=$(ip -4 route get 1.1.1.1 2>/dev/null | sed -n 's/.*[[:space:]]src[[:space:]]\([0-9.]*\).*/\1/p' | head -n1) + fi + ;; + esac + + if [[ ! "$ip" =~ ^([0-9]{1,3}\.){3}[0-9]{1,3}$ ]]; then + return 0 + fi + printf '%s' "$ip" +} + +# Returns 0 when the enrollment URL targets the Docker host by literal IPv4 and +# wg-client must bypass wg0 for management traffic. +# Args: +# $1 - Enrollment management URL +netbird_enrollment_url_uses_host_bypass() { + local url=$1 + [[ "$url" =~ ^https?://([0-9]{1,3}\.){3}[0-9]{1,3}([:/]|$) ]] +} diff --git a/cli/lib/require.bash b/cli/lib/require.bash index bfd7ace8..62b43332 100644 --- a/cli/lib/require.bash +++ b/cli/lib/require.bash @@ -3,6 +3,62 @@ set -euo pipefail : "${exitcode_expectation_failed:=168}" +# Set after a successful mikefarah probe so `sandcat init`'s many +# `require yq` calls do not spawn `yq --version` each time. `:` so a later +# `source require.bash` does not drop a cache already set in this shell. +: "${_SCT_REQUIRE_YQ_OK:=}" + +# Retries yq when it dies on SIGSEGV. +# +# Some yq builds crash intermittently (measured ~0.5% of invocations on one +# aarch64 build that reports v4.53.3 but is not the upstream release binary). +# A single `sandcat init` runs ~100 sequential yq mutations under `set -e`, so +# one transient crash aborts init and leaves a half-written .devcontainer that +# looks like a generation bug. Retrying is safe: yq edits in place via +# temp-file-and-rename, so a crashed call leaves the target file untouched. +# +# stdout is buffered so a crash that already emitted partial output cannot be +# concatenated with the retry's output. Only SIGSEGV is retried; every other +# non-zero status is a real yq error and is returned unchanged. +# +# In-place edits skip the temp file: yq -i writes the result itself, and +# capturing empty stdout was doubling process overhead on the hot path. +yq() { + local attempt status stdout_file inplace=false arg + for arg in "$@"; do + if [[ "$arg" == "-i" || "$arg" == "--inplace" ]]; then + inplace=true + break + fi + done + + if [[ "$inplace" == true ]]; then + for attempt in 1 2 3; do + status=0 + command yq "$@" || status=$? + if [[ "$status" -ne 139 ]]; then + return "$status" + fi + done + return "$status" + fi + + stdout_file=$(mktemp) + + for attempt in 1 2 3; do + status=0 + command yq "$@" >"$stdout_file" || status=$? + if [[ "$status" -ne 139 ]]; then + break + fi + : >"$stdout_file" + done + + cat "$stdout_file" + rm -f "$stdout_file" + return "$status" +} + # Ensures a command is available # Args: # $1 - The command name to require @@ -11,22 +67,37 @@ set -euo pipefail require() { local -r cmd="$1" - if ! command -v "$cmd" &>/dev/null - then - >&2 echo "$0: $cmd required" - return "$exitcode_expectation_failed" + if [[ "$cmd" == "yq" ]]; then + if [[ -n "${_SCT_REQUIRE_YQ_OK:-}" ]]; then + return 0 + fi + + # yq is wrapped by the shell function above, so command -v would + # report it as present even with no binary installed. + if ! type -P yq &>/dev/null + then + >&2 echo "$0: yq required" + return "$exitcode_expectation_failed" + fi + + # Two unrelated tools share the name `yq`: Mike Farah's Go yq + # (what sandcat uses) and Python yq (kislyuk/yq), which is what + # `apt install yq` ships on Debian/Ubuntu. Probe the PATH binary + # directly so the retry wrapper cannot shadow --version. + if ! command yq --version 2>&1 | grep -q mikefarah + then + >&2 echo "$0: sandcat needs Mike Farah's yq (https://github.com/mikefarah/yq); a different 'yq' is on PATH." + >&2 echo "On Debian/Ubuntu, 'apt install yq' installs the incompatible Python yq (kislyuk/yq)." + >&2 echo "Install Mike Farah's yq from https://github.com/mikefarah/yq/#install (e.g. 'snap install yq')." + return "$exitcode_expectation_failed" + fi + _SCT_REQUIRE_YQ_OK=1 + return 0 fi - # Two unrelated tools share the name `yq`: Mike Farah's Go yq (what sandcat - # uses, with `-o json`, `head_comment`, `env(...)` etc.) and Python yq - # (kislyuk/yq), which is what `apt install yq` ships on Debian/Ubuntu. - # Detect the wrong variant up front so users see a clear pointer instead - # of an opaque parse error mid-init. - if [[ "$cmd" == "yq" ]] && ! yq --version 2>&1 | grep -q mikefarah + if ! command -v "$cmd" &>/dev/null then - >&2 echo "$0: sandcat needs Mike Farah's yq (https://github.com/mikefarah/yq); a different 'yq' is on PATH." - >&2 echo "On Debian/Ubuntu, 'apt install yq' installs the incompatible Python yq (kislyuk/yq)." - >&2 echo "Install Mike Farah's yq from https://github.com/mikefarah/yq/#install (e.g. 'snap install yq')." + >&2 echo "$0: $cmd required" return "$exitcode_expectation_failed" fi } diff --git a/cli/lib/volume.bash b/cli/lib/volume.bash index 762cb0fa..6bb0efc3 100644 --- a/cli/lib/volume.bash +++ b/cli/lib/volume.bash @@ -2,6 +2,42 @@ # shellcheck source=logging.bash source "$SCT_LIBDIR/logging.bash" +# shellcheck source=require.bash +source "$SCT_LIBDIR/require.bash" + +# Parse a Docker ISO-8601 timestamp to epoch seconds (GNU date or BSD date). +_volume_timestamp_epoch() { + local timestamp=$1 + local no_frac epoch bsd + + no_frac=$(printf '%s' "$timestamp" | sed -E 's/\.[0-9]+//') + + if epoch=$(date -d "$timestamp" +%s 2>/dev/null); then + printf '%s' "$epoch" + return 0 + fi + if epoch=$(date -d "$no_frac" +%s 2>/dev/null); then + printf '%s' "$epoch" + return 0 + fi + if [[ "$no_frac" == *Z ]] && epoch=$(date -d "${no_frac%Z} UTC" +%s 2>/dev/null); then + printf '%s' "$epoch" + return 0 + fi + + bsd=$no_frac + if [[ "$bsd" == *Z ]]; then + bsd="${bsd%Z}+0000" + elif [[ "$bsd" =~ ([+-][0-9]{2}):([0-9]{2})$ ]]; then + bsd=$(printf '%s' "$bsd" | sed -E 's/([+-][0-9]{2}):([0-9]{2})$/\1\2/') + fi + if epoch=$(date -ju -f '%Y-%m-%dT%H:%M:%S%z' "$bsd" +%s 2>/dev/null); then + printf '%s' "$epoch" + return 0 + fi + + return 1 +} # Warns if the agent-home volume is meaningfully older than the agent # image — i.e. the image has been rebuilt since the volume was populated, @@ -39,8 +75,8 @@ warn_stale_home_volume() { # silently if unavailable (e.g. BSD date on macOS without coreutils) # rather than risk false positives from lexicographic comparison. local vol_epoch img_epoch - vol_epoch=$(date -d "$volume_time" +%s 2>/dev/null) || return 0 - img_epoch=$(date -d "$image_time" +%s 2>/dev/null) || return 0 + vol_epoch=$(_volume_timestamp_epoch "$volume_time") || return 0 + img_epoch=$(_volume_timestamp_epoch "$image_time") || return 0 (( img_epoch - vol_epoch > tolerance_seconds )) || return 0 diff --git a/cli/libexec/attach/attach b/cli/libexec/attach/attach index d66a0764..b494b8f9 100755 --- a/cli/libexec/attach/attach +++ b/cli/libexec/attach/attach @@ -5,6 +5,8 @@ set -euo pipefail source "$SCT_LIBDIR/require.bash" # shellcheck source=../../lib/path.bash source "$SCT_LIBDIR/path.bash" +# shellcheck source=../../lib/netbird.bash +source "$SCT_LIBDIR/netbird.bash" attach() { require docker @@ -12,6 +14,10 @@ attach() { local compose_file compose_file="$(find_compose_file)" + export_netbird_compose_env + export_netbird_management_url + export_agent_sandcat_mount + if (($# == 0)) then exec docker compose -f "$compose_file" exec -u vscode agent bash --login diff --git a/cli/libexec/cache/list b/cli/libexec/cache/list index 7359b08d..2df541f3 100755 --- a/cli/libexec/cache/list +++ b/cli/libexec/cache/list @@ -1,6 +1,8 @@ #!/usr/bin/env bash set -euo pipefail +# shellcheck source=../../lib/compat.bash +source "$SCT_LIBDIR/compat.bash" # shellcheck source=../../lib/require.bash source "$SCT_LIBDIR/require.bash" # shellcheck source=../../lib/cache.bash diff --git a/cli/libexec/cache/rm b/cli/libexec/cache/rm index 678c4367..cb86cb39 100755 --- a/cli/libexec/cache/rm +++ b/cli/libexec/cache/rm @@ -1,6 +1,8 @@ #!/usr/bin/env bash set -euo pipefail +# shellcheck source=../../lib/compat.bash +source "$SCT_LIBDIR/compat.bash" # shellcheck source=../../lib/require.bash source "$SCT_LIBDIR/require.bash" # shellcheck source=../../lib/cache.bash diff --git a/cli/libexec/cache/size b/cli/libexec/cache/size index 24c2fe1a..44f96b9a 100755 --- a/cli/libexec/cache/size +++ b/cli/libexec/cache/size @@ -1,6 +1,8 @@ #!/usr/bin/env bash set -euo pipefail +# shellcheck source=../../lib/compat.bash +source "$SCT_LIBDIR/compat.bash" # shellcheck source=../../lib/require.bash source "$SCT_LIBDIR/require.bash" # shellcheck source=../../lib/cache.bash diff --git a/cli/libexec/compose/compose b/cli/libexec/compose/compose index 81fe1e61..c0318e0a 100755 --- a/cli/libexec/compose/compose +++ b/cli/libexec/compose/compose @@ -5,6 +5,8 @@ set -euo pipefail source "$SCT_LIBDIR/require.bash" # shellcheck source=../../lib/path.bash source "$SCT_LIBDIR/path.bash" +# shellcheck source=../../lib/netbird.bash +source "$SCT_LIBDIR/netbird.bash" # Helper that automatically locates the docker compose file # All arguments are passed to docker compose @@ -14,6 +16,10 @@ compose() { local compose_file compose_file="$(find_compose_file)" + export_netbird_compose_env + export_netbird_management_url + export_agent_sandcat_mount + exec docker compose -f "$compose_file" "$@" } diff --git a/cli/libexec/init/devcontainer b/cli/libexec/init/devcontainer index 4dd307cf..7679c5f5 100755 --- a/cli/libexec/init/devcontainer +++ b/cli/libexec/init/devcontainer @@ -15,6 +15,7 @@ source "$SCT_LIBDIR/devcontainer.bash" # --project-path - Path to the project directory # --agent - The agent name (e.g., "claude") # --ide - The IDE name (e.g., "vscode", "jetbrains", "none") (optional) +# --netbird-management-url - NetBird management server URL for mitmproxy (optional) devcontainer() { local settings_file="" local project_path="" @@ -24,11 +25,13 @@ devcontainer() { local stacks="" local proxy_mode="web" local secret_provider="none" + local netbird="false" + local netbird_management_url="" while [[ $# -gt 0 ]] do case $1 in - --settings-file|--project-path|--agent|--ide|--name|--stacks|--proxy|--secret-provider) + --settings-file|--project-path|--agent|--ide|--name|--stacks|--proxy|--secret-provider|--netbird-management-url) if [[ $# -lt 2 ]]; then echo "Option $1 requires a value" | error return 1 @@ -72,6 +75,14 @@ devcontainer() { secret_provider="1password" shift 1 ;; + --netbird) + netbird="true" + shift 1 + ;; + --netbird-management-url) + netbird_management_url="$2" + shift 2 + ;; *) echo "Unknown option: $1" | error return 1 @@ -119,6 +130,26 @@ devcontainer() { apply_upstream_ca_bundles "$devcontainer_dir/sandcat/compose-proxy.yml" "$project_path" + local project_settings="$project_path/$SCT_PROJECT_DIR/settings.json" + local peer_name_proxy="" + if [[ "$netbird" == "true" ]]; then + # shellcheck source=../../lib/netbird.bash + source "$SCT_LIBDIR/netbird.bash" + mkdir -p "$project_path/$SCT_PROJECT_DIR" + export SANDCAT_PROJECT_ROOT="$project_path" + peer_name_proxy="${project_name}-proxy" + netbird_ensure_peer_name_settings "$project_settings" "$project_name" + fi + + if [[ "$netbird" == "true" ]]; then + if ! declare -f enable_netbird &>/dev/null; then + # shellcheck source=../../lib/composefile.bash + source "$SCT_LIBDIR/composefile.bash" + fi + enable_netbird "$devcontainer_dir/sandcat/compose-proxy.yml" \ + "$netbird_management_url" "$peer_name_proxy" + fi + customize_compose_file "$rel_settings_file" "$compose_file" "$agent" "$ide" "$project_name" "$stacks" set_project_name "$compose_file" "$project_name" @@ -132,6 +163,14 @@ devcontainer() { customize_devcontainer_plugins "$devcontainer_dir/devcontainer.json" "${stacks_arr_jb[@]}" fi + # Compose interpolates ${SANDCAT_AGENT_SANDCAT:?} with no fallback to the + # live project .sandcat. Write .devcontainer/.env now so `docker compose + # config` and Dev Containers work before initializeCommand runs. + # shellcheck source=../../lib/netbird.bash + source "$SCT_LIBDIR/netbird.bash" + export SANDCAT_PROJECT_ROOT="$project_path" + export_agent_sandcat_mount + echo "Devcontainer dir created at ${devcontainer_dir#"$project_path"/}" | info } diff --git a/cli/libexec/init/init b/cli/libexec/init/init index ca9a97f3..51a114a0 100755 --- a/cli/libexec/init/init +++ b/cli/libexec/init/init @@ -17,6 +17,8 @@ source "$SCT_LIBDIR/stacks.bash" source "$SCT_LIBDIR/agents.bash" # shellcheck source=../../lib/gitignore.bash source "$SCT_LIBDIR/gitignore.bash" +# shellcheck source=../../lib/netbird.bash +source "$SCT_LIBDIR/netbird.bash" # Returns the user settings template path for a selected agent. # Args: @@ -91,6 +93,65 @@ ensure_cursor_user_settings_defaults() { fi } +persist_netbird_management_url() { + local user_settings=$1 + local selected_management_url=${2:-} + if [[ -f "$user_settings" ]]; then + netbird_management_url="$selected_management_url" yq -i -o json ' + .netbird_management_url = strenv(netbird_management_url) + ' "$user_settings" + fi +} + +# Asks for an address the mitmproxy container can dial when the chosen +# management URL is localhost-only. Without this the compose file gets no +# NB_MANAGEMENT_URL and netbird silently enrolls against the cloud +# default, which rejects a self-hosted setup key. +prompt_netbird_enrollment_url() { + local user_settings=$1 + local netbird_management_url=$2 + local host_ip mgmt_port default_url entered_url + host_ip=$(netbird_detect_docker_host_ip) + mgmt_port="${netbird_management_url##*:}" + mgmt_port="${mgmt_port%%/*}" + if [[ ! "$mgmt_port" =~ ^[0-9]+$ ]]; then + mgmt_port="33073" + fi + default_url="" + if [[ -n "$host_ip" ]]; then + default_url="http://${host_ip}:${mgmt_port}" + fi + + echo " $netbird_management_url is not reachable from inside the container." | info + echo " Enter an address the container can dial (your Docker host's LAN IP)." | info + while true; do + if [[ -n "$default_url" ]]; then + entered_url=$(read_line "Enrollment URL [$default_url]:") + entered_url="${entered_url:-$default_url}" + else + entered_url=$(read_line "Enrollment URL (e.g. http://192.168.1.10:${mgmt_port}):") + fi + if [[ -n "$entered_url" ]]; then + break + fi + echo "Enrollment URL is required for a localhost management server" | error + done + + if [[ -f "$user_settings" ]]; then + enrollment_url="$entered_url" yq -i -o json ' + .netbird_enrollment_management_url = strenv(enrollment_url) + ' "$user_settings" + fi +} + +maybe_prompt_enrollment_url() { + local user_settings=$1 + local netbird_management_url=$2 + if [[ -z "$(netbird_enrollment_management_url_from "$netbird_management_url")" ]]; then + prompt_netbird_enrollment_url "$user_settings" "$netbird_management_url" + fi +} + # Ensures codex-specific defaults exist in an already-created user settings file # without overriding user-provided values. ensure_codex_user_settings_defaults() { @@ -145,6 +206,7 @@ add_secret_provider_tokens_to_user_settings() { # --secret-provider / --sp - Secret backend: none, 1password, protonpass # --1password - Deprecated; same as --secret-provider 1password # --features - Comma-separated optional non-provider features (tui) +# --netbird-management-url - Existing NetBird management server URL (omit = cloud) init() { require yq @@ -160,11 +222,15 @@ init() { local onepassword_alias=false local features_csv="" local features_provided=false + local netbird="false" + local netbird_management_url="" + local netbird_management_url_provided=false + local init_prompted=false while [[ $# -gt 0 ]] do case $1 in - --name|--path|--agent|--ide|--stacks|--proxy|--features|--secret-provider|--sp) + --name|--path|--agent|--ide|--stacks|--proxy|--features|--secret-provider|--sp|--netbird-management-url) if [[ $# -lt 2 ]]; then echo "Option $1 requires a value" | error return 1 @@ -211,6 +277,15 @@ init() { features_provided=true shift 2 ;; + --netbird) + netbird="true" + shift 1 + ;; + --netbird-management-url) + netbird_management_url="$2" + netbird_management_url_provided=true + shift 2 + ;; *) echo "Unknown option: $1" | error return 1 @@ -226,6 +301,14 @@ init() { secret_provider="1password" secret_provider_provided=true fi + if [[ "$netbird_management_url_provided" == "true" && "$netbird" != "true" ]]; then + echo "--netbird-management-url requires --netbird" | error + return 1 + fi + if [[ "$netbird_management_url_provided" == "true" && -z "$netbird_management_url" ]]; then + echo "--netbird-management-url requires a value" | error + return 1 + fi if [[ -z "$project_path" ]] then @@ -245,6 +328,7 @@ init() { default_name="$(derive_project_name "$project_path")" if [[ -z "$name" ]] then + init_prompted=true name=$(read_line "Project name [$default_name]:") fi @@ -252,6 +336,7 @@ init() { read -ra available_agents <<< "$(sct_available_agents)" if [[ -z "$agent" ]] then + init_prompted=true agent=$(select_option "Select agent:" "${available_agents[@]}") elif ! sct_is_valid_agent "$agent" then @@ -269,6 +354,7 @@ init() { local available_ides=(vscode jetbrains none) if [[ -z "$ide" ]] then + init_prompted=true ide=$(select_option "Select IDE:" "${available_ides[@]}") elif [[ ! " ${available_ides[*]} " =~ [[:space:]]${ide}[[:space:]] ]] then @@ -278,6 +364,7 @@ init() { # Secret provider (none | 1password | protonpass) if [[ "$secret_provider_provided" != "true" ]]; then + init_prompted=true local existing_settings existing_settings="$(sct_home)/settings.json" if [[ -f "$existing_settings" ]] && \ @@ -308,6 +395,7 @@ init() { local rtk_enabled=${SANDCAT_RTK:-true} local strict_network=${SANDCAT_STRICT_NETWORK:-false} if [[ "$features_provided" != "true" ]]; then + init_prompted=true local available_features=( "tui (mitmproxy console instead of web UI)" "no-shared-cache (per-project dep cache instead of shared)" @@ -369,6 +457,7 @@ init() { fi stacks_resolved=$(resolve_stacks "${stacks_arr[@]}") elif [[ "$stacks_provided" != "true" ]]; then + init_prompted=true local selected selected=$(select_multiple "Select development stacks (comma-separated numbers, empty for none):" "${STACK_NAMES[@]}") if [[ -n "$selected" ]]; then @@ -394,6 +483,69 @@ init() { add_secret_provider_tokens_to_user_settings "$secret_provider" + if [[ "$netbird" == "true" ]]; then + local user_settings + user_settings="$(sct_home)/settings.json" + if [[ -f "$user_settings" ]]; then + yq -i -o json ' + .netbird_enrollment_key = (.netbird_enrollment_key // "") | + .netbird_api_token = (.netbird_api_token // "") | + .netbird_management_url = (.netbird_management_url // "") + ' "$user_settings" + fi + + if [[ "$netbird_management_url_provided" == "true" ]]; then + persist_netbird_management_url "$user_settings" "$netbird_management_url" + if [[ "$init_prompted" == "true" ]]; then + maybe_prompt_enrollment_url "$user_settings" "$netbird_management_url" + fi + elif [[ "$init_prompted" == "true" ]]; then + local netbird_server_selection="" + while true; do + echo "NetBird management server [cloud]:" | info + echo " 1) cloud (api.netbird.io)" | info + echo " 2) self-hosted — I have a server running" | info + echo " To create a server, see docs/examples/netbird-server/" | info + netbird_server_selection=$(read_line ">") + case "$netbird_server_selection" in + ""|1|cloud) + netbird_server_selection="cloud" + break + ;; + 2|existing|self-hosted-existing|self-hosted\ existing\ URL|self-hosted\ —\ I\ have\ a\ server\ running) + netbird_server_selection="existing" + break + ;; + *) + echo "Invalid NetBird management server selection: $netbird_server_selection (expected: cloud or existing)" | error + ;; + esac + done + case "$netbird_server_selection" in + cloud) + netbird_management_url="" + ;; + existing) + while true; do + netbird_management_url=$(read_line "Management URL:") + if [[ -z "$netbird_management_url" ]]; then + echo "URL is required" | error + continue + fi + persist_netbird_management_url "$user_settings" "$netbird_management_url" + maybe_prompt_enrollment_url "$user_settings" "$netbird_management_url" + break + done + ;; + esac + else + # Fully flagged `--netbird` without `--netbird-management-url` is cloud. + netbird_management_url="" + fi + + persist_netbird_management_url "$user_settings" "$netbird_management_url" + fi + local settings_args=() if [[ "$strict_network" == "true" ]]; then settings_args+=(--strict-network --stacks "$stacks_resolved") @@ -410,6 +562,12 @@ init() { --secret-provider "$secret_provider" ) export SANDCAT_RTK="$rtk_enabled" + if [[ -n "$netbird_management_url" ]]; then + devcontainer_args+=(--netbird-management-url "$netbird_management_url") + fi + if [[ "$netbird" == "true" ]]; then + devcontainer_args+=(--netbird) + fi devcontainer "${devcontainer_args[@]}" local gitignore_status="skipped" @@ -511,6 +669,19 @@ init() { echo " GITHUB_TOKEN a GitHub personal access token (for git push, gh cli)" | info ;; esac + if [[ "$netbird" == "true" ]]; then + echo "" >&2 + echo " NetBird setup:" | info + if [[ -n "$netbird_management_url" ]]; then + echo " Management server: $netbird_management_url" | info + else + echo " Management server: cloud (https://api.netbird.io)" | info + fi + echo " Add keys to ~/.config/sandcat/settings.json:" | info + echo " \"netbird_enrollment_key\": \"\" (mitmproxy enrollment)" | info + echo " \"netbird_api_token\": \"\" (mitmproxy peer replace)" | info + echo " To run a self-hosted management server, see docs/examples/netbird-server/" | info + fi echo " Then run: sandcat run, or reopen the project using the dev container" | info } diff --git a/cli/libexec/restart/restart b/cli/libexec/restart/restart index f4ccd2af..f47d3951 100755 --- a/cli/libexec/restart/restart +++ b/cli/libexec/restart/restart @@ -7,6 +7,8 @@ source "$SCT_LIBDIR/logging.bash" source "$SCT_LIBDIR/require.bash" # shellcheck source=../../lib/path.bash source "$SCT_LIBDIR/path.bash" +# shellcheck source=../../lib/netbird.bash +source "$SCT_LIBDIR/netbird.bash" # Restarts the mitmproxy, wg-client, and agent services to pick up settings # changes. The agent is re-linked to the fresh wg-client netns (see #69). @@ -16,6 +18,10 @@ restart() { local compose_file compose_file="$(find_compose_file)" + export_netbird_compose_env + export_netbird_management_url + export_agent_sandcat_mount + local proxy_status proxy_status=$(docker compose -f "$compose_file" ps mitmproxy --status running --quiet 2>/dev/null) || true diff --git a/cli/libexec/run/run b/cli/libexec/run/run index 3c1cafa4..aa2cc1ec 100755 --- a/cli/libexec/run/run +++ b/cli/libexec/run/run @@ -7,6 +7,8 @@ source "$SCT_LIBDIR/require.bash" source "$SCT_LIBDIR/path.bash" # shellcheck source=../../lib/volume.bash source "$SCT_LIBDIR/volume.bash" +# shellcheck source=../../lib/netbird.bash +source "$SCT_LIBDIR/netbird.bash" # Run a command in the agent container. # Starts dependencies, runs the command, then tears everything down. @@ -38,6 +40,10 @@ run() { warn_stale_home_volume "$compose_file" ensure_shared_cache_volumes "$compose_file" + export_netbird_compose_env + export_netbird_management_url + export_agent_sandcat_mount + local rc=0 docker compose -f "$compose_file" run --rm "${run_opts[@]+"${run_opts[@]}"}" agent "${1-bash}" "${@:2}" || rc=$? docker compose -f "$compose_file" down diff --git a/cli/templates/devcontainer/compose-all.yml b/cli/templates/devcontainer/compose-all.yml index 014cd5ef..88f545c0 100644 --- a/cli/templates/devcontainer/compose-all.yml +++ b/cli/templates/devcontainer/compose-all.yml @@ -2,12 +2,7 @@ include: - path: sandcat/compose-proxy.yml # Constant (non-user-editable) parts of the agent service — image build, # security hardening (no-new-privileges), network wiring through - # wg-client, dependency ordering. The `services.agent` entries below are - # merged OVER that base by Docker Compose. + # wg-client, dependency ordering. User-facing agent volumes and + # environment are written into this included file by sandcat init + # (Compose include never merges a service redefined in compose-all.yml). - path: sandcat/compose-agent.yml - -services: - # User-customizable parts of the agent service (volumes, environment). - # sandcat init populates this section; edit or uncomment entries freely — - # the security-critical base stays in sandcat/compose-agent.yml. - agent: {} diff --git a/cli/templates/devcontainer/devcontainer.json b/cli/templates/devcontainer/devcontainer.json index 4f9bac46..13d10b89 100644 --- a/cli/templates/devcontainer/devcontainer.json +++ b/cli/templates/devcontainer/devcontainer.json @@ -11,6 +11,10 @@ // sandcat's security boundary. Matches the Dev Containers spec // default for dockerComposeFile scenarios. "overrideCommand": false, + // Host-side: copy .sandcat, strip NetBird secrets, write + // .devcontainer/.env (SANDCAT_AGENT_SANDCAT). Compose interpolation + // has no fallback to the live project dir. + "initializeCommand": "bash .devcontainer/sandcat/scripts/prepare-agent-sandcat-mount.sh", // Remove credential sockets that VS Code forwards into the container // (SSH agent, git credential helper). Clearing env vars alone only // hides the paths — the socket files in /tmp can still be discovered diff --git a/cli/templates/devcontainer/sandcat/Dockerfile.mitmproxy b/cli/templates/devcontainer/sandcat/Dockerfile.mitmproxy new file mode 100644 index 00000000..bd5f46f6 --- /dev/null +++ b/cli/templates/devcontainer/sandcat/Dockerfile.mitmproxy @@ -0,0 +1,72 @@ +# mitmproxy + NetBird, layered on top of whichever secret-provider image the +# project selected. +# +# The NetBird client is fetched and checksum-verified in a throwaway builder +# stage, then copied into the final image. That keeps the download tooling out +# of the runtime image and, more importantly, decouples the NetBird pin from the +# base: BASE_IMAGE can be the stock mitmproxy image or a provider variant +# (sandcat-mitmproxy-pass / -op), and the result has both NetBird and the +# provider CLI. Building FROM the provider image instead of replacing it is what +# keeps pass:// and op:// references resolvable once NetBird is enabled. +# +# BASE_IMAGE is set by enable_netbird()/apply_secret_provider() in +# cli/lib/composefile.bash; the default matches the no-secret-provider case. +ARG BASE_IMAGE=mitmproxy/mitmproxy:latest + +# Version + per-arch checksums are the single source of truth in netbird.env +# (sibling to this Dockerfile) and MUST be supplied as build args. No defaults +# here on purpose, so the pin lives in exactly one place. +FROM debian:stable-slim AS netbird-build + +ARG NETBIRD_VERSION +ARG NETBIRD_SHA256_AMD64 +ARG NETBIRD_SHA256_ARM64 + +RUN apt-get update \ + && apt-get install -y --no-install-recommends ca-certificates curl \ + && rm -rf /var/lib/apt/lists/* + +RUN test -n "$NETBIRD_VERSION" || { echo "NETBIRD_VERSION build arg is required (source netbird.env)" >&2; exit 1; } \ + && ARCH=$(dpkg --print-architecture) \ + && case "$ARCH" in \ + amd64) NETBIRD_SHA256="$NETBIRD_SHA256_AMD64" ;; \ + arm64) NETBIRD_SHA256="$NETBIRD_SHA256_ARM64" ;; \ + *) echo "Unsupported architecture: $ARCH" >&2; exit 1 ;; \ + esac \ + && test -n "$NETBIRD_SHA256" \ + && curl -sSLf -o /tmp/netbird.tar.gz \ + "https://github.com/netbirdio/netbird/releases/download/v${NETBIRD_VERSION}/netbird_${NETBIRD_VERSION}_linux_${ARCH}.tar.gz" \ + && echo "${NETBIRD_SHA256} /tmp/netbird.tar.gz" | sha256sum -c - \ + && mkdir -p /out \ + && tar xzf /tmp/netbird.tar.gz -C /out netbird \ + && chmod +x /out/netbird \ + && rm /tmp/netbird.tar.gz + +FROM ${BASE_IMAGE} + +# ca-certificates - update-ca-certificates trusts the mitmproxy CA so the +# NetBird daemon's TLS connections to management/signal +# servers succeed (those connections go out via eth0 directly, +# not wg0, so no MITM interception occurs here) +# curl - management-server reachability probe in mitmproxy-init.sh +# iproute2 - ip route/rule for host management bypass routing +# iptables - host management bypass via mangle table (self-hosted NB_MANAGEMENT_URL) +# jq - parse `netbird status --json` for DNS peer record publishing +RUN apt-get update \ + && apt-get install -y --no-install-recommends \ + ca-certificates curl iproute2 iptables jq \ + && rm -rf /var/lib/apt/lists/* + +# The NetBird client daemon manages wt0 (the NetBird overlay mesh) inside this +# container, which already holds NET_ADMIN. mitmproxy's WireGuard server mode +# runs in userspace; wt0 is the separate kernel interface for mesh connectivity. +COPY --from=netbird-build /out/netbird /usr/local/bin/netbird +RUN netbird version + +COPY scripts/netbird-peer-lifecycle.sh /usr/local/lib/netbird-peer-lifecycle.sh +COPY scripts/mitmproxy-init.sh /usr/local/bin/mitmproxy-init.sh +RUN chmod +x /usr/local/bin/mitmproxy-init.sh + +# mitmproxy-init.sh runs as root, sets up NetBird in the background, then +# runs docker-entrypoint.sh as a child so SIGTERM can call netbird down. +ENTRYPOINT ["/usr/local/bin/mitmproxy-init.sh"] diff --git a/cli/templates/devcontainer/sandcat/Dockerfile.wg-client b/cli/templates/devcontainer/sandcat/Dockerfile.wg-client index 53968be6..18807f6d 100644 --- a/cli/templates/devcontainer/sandcat/Dockerfile.wg-client +++ b/cli/templates/devcontainer/sandcat/Dockerfile.wg-client @@ -6,11 +6,13 @@ FROM debian:trixie-slim # iptables - firewall rules used as a kill switch (blocks traffic if tunnel drops) # jq - parse mitmproxy's wireguard.conf JSON to extract key pairs # openresolv - `resolvconf` command to configure DNS through the tunnel -# dnsmasq - local DNS forwarder providing split-DNS: Docker compose -# network names go to 127.0.0.11; everything else to upstream +# dnsmasq - local DNS forwarder providing split-DNS +# ca-certificates - base system CA bundle (used by dnsmasq and any TLS clients +# sharing this network namespace) RUN apt-get update \ && apt-get install -y --no-install-recommends \ wireguard-tools iproute2 iptables jq openresolv dnsmasq \ + ca-certificates \ && rm -rf /var/lib/apt/lists/* COPY scripts/wg-client-init.sh /usr/local/bin/wg-client-init.sh diff --git a/cli/templates/devcontainer/sandcat/compose-agent.yml b/cli/templates/devcontainer/sandcat/compose-agent.yml index 1883870c..e582bc93 100644 --- a/cli/templates/devcontainer/sandcat/compose-agent.yml +++ b/cli/templates/devcontainer/sandcat/compose-agent.yml @@ -1,8 +1,8 @@ # Constant (non-user-editable) parts of the agent service. User-customizable -# parts — volumes, environment — live in ../compose-all.yml and are merged -# over this base by Docker Compose's include-override mechanism (the same -# mechanism compose-all.yml already uses to add the project-settings mount -# to the mitmproxy service defined in compose-proxy.yml). +# parts — volumes, environment — are appended here by sandcat init (same +# pattern as project-settings mounts written into compose-proxy.yml). Do not +# also declare `services.agent` in ../compose-all.yml: Compose include copies +# resources into the model and rejects a redefined service. services: agent: build: diff --git a/cli/templates/devcontainer/sandcat/netbird.env b/cli/templates/devcontainer/sandcat/netbird.env new file mode 100644 index 00000000..66f8d7c3 --- /dev/null +++ b/cli/templates/devcontainer/sandcat/netbird.env @@ -0,0 +1,18 @@ +# Single source of truth for the pinned NetBird client binary in wg-client. +# +# Consumed by: +# - cli/templates/devcontainer/sandcat/Dockerfile.wg-client (via compose build args) +# - cli/lib/composefile.bash apply_netbird_build_args +# - cli/test/composefile/netbird_contract.bats +# +# When bumping the version: +# 1. Update NETBIRD_VERSION and BOTH checksums below from the release assets: +# https://github.com/netbirdio/netbird/releases/download/v/netbird__checksums.txt +# 2. Look for netbird__linux_amd64.tar.gz and netbird__linux_arm64.tar.gz +# +# Format note: simple KEY=value lines only (no quotes, no spaces around `=`) so +# this file is consumable by `source`, compose build-arg injection, and contract +# tests alike. +NETBIRD_VERSION=0.72.4 +NETBIRD_SHA256_AMD64=8ee7807d716ed088ab05976bc161838120730f9cf9fec794aacb5b51d904f1fc +NETBIRD_SHA256_ARM64=7d2be0ef0cbe82bc18071505f69ff7d9967b492664aafdc39ce531233d6e6405 diff --git a/cli/templates/devcontainer/sandcat/scripts/mitmproxy-init.sh b/cli/templates/devcontainer/sandcat/scripts/mitmproxy-init.sh new file mode 100644 index 00000000..ac65e1d1 --- /dev/null +++ b/cli/templates/devcontainer/sandcat/scripts/mitmproxy-init.sh @@ -0,0 +1,308 @@ +#!/usr/bin/env bash +# cli/templates/devcontainer/sandcat/scripts/mitmproxy-init.sh +# +# Entrypoint wrapper for the mitmproxy container when NetBird is enabled. +# Clears healthcheck sentinels, publishes the CA cert, optionally enrolls +# mitmproxy as a NetBird peer on wt0 in the background, then runs +# docker-entrypoint.sh as a child so SIGTERM can call netbird down. +# +# NetBird enrollment is optional: if NB_SETUP_KEY is not set (directly or via +# settings.json), mitmproxy starts normally without mesh connectivity. +# +# Expects: +# NB_SETUP_KEY - NetBird enrollment key (optional; read from settings +# if absent from environment) +# NB_MANAGEMENT_URL - NetBird management server URL (optional; default +# api.netbird.io; read from settings if absent) +# NB_PEER_NAME - Stable peer hostname in the NetBird management UI +# (required; set by compose) +# NETBIRD_IFACE - WireGuard interface name for NetBird mesh +# (default: wt0) +# NETBIRD_WG_PORT - UDP listen port for NetBird's wt0 (default: 51821). +# Must NOT be 51820: that port belongs to mitmproxy's +# userspace WireGuard server that wg-client dials. +# NETBIRD_DNS_DOMAIN - NetBird mesh DNS domain (default: netbird.selfhosted) +# NETBIRD_DNS_CONF_PATH - Volume path where peer DNS records are published for +# wg-client dnsmasq (default: /home/mitmproxy/.mitmproxy/netbird-peers.conf) + +set -euo pipefail + +NB_MANAGEMENT_URL="${NB_MANAGEMENT_URL:-}" +NB_PEER_NAME="${NB_PEER_NAME:?NB_PEER_NAME must be set by compose}" +NETBIRD_IFACE="${NETBIRD_IFACE:-wt0}" +# mitmproxy --mode wireguard binds UDP 51820; NetBird must use another port or +# mitmweb/mitmdump fails with "Failed to bind UDP socket to 0.0.0.0:51820". +NETBIRD_WG_PORT="${NETBIRD_WG_PORT:-51821}" +NETBIRD_DNS_DOMAIN="${NETBIRD_DNS_DOMAIN:-netbird.selfhosted}" +# Path in the mitmproxy-config volume where peer DNS records are published. +# wg-client reads this file and appends the records to its dnsmasq config so +# that NetBird FQDNs (e.g. myproject-proxy-peer.netbird.selfhosted) resolve for agents. +NETBIRD_DNS_CONF_PATH="${NETBIRD_DNS_CONF_PATH:-/home/mitmproxy/.mitmproxy/netbird-peers.conf}" +NETBIRD_PEER_LOG_PREFIX="${NETBIRD_PEER_LOG_PREFIX:-mitmproxy}" +NETBIRD_PEER_LIFECYCLE_PATH="${NETBIRD_PEER_LIFECYCLE_PATH:-/usr/local/lib/netbird-peer-lifecycle.sh}" +MITMPROXY_HOME="${MITMPROXY_HOME:-/home/mitmproxy/.mitmproxy}" +MITMPROXY_PUBLIC="${MITMPROXY_PUBLIC:-/mitmproxy-public}" +MITMPROXY_WEB_PASSWORD_FILE="${MITMPROXY_WEB_PASSWORD_FILE:-$MITMPROXY_HOME/web_password}" + +# shellcheck source=/usr/local/lib/netbird-peer-lifecycle.sh +source "$NETBIRD_PEER_LIFECYCLE_PATH" + +# True when $1 is a DNS-safe FQDN under $NETBIRD_DNS_DOMAIN (suffix match). +# Rejects substring matches (evil..attacker) and names with '/'. +netbird_dns_fqdn_allowed() { + local fqdn=$1 + [[ "$fqdn" =~ ^[A-Za-z0-9._-]+$ ]] || return 1 + [[ "$fqdn" == "$NETBIRD_DNS_DOMAIN" || "$fqdn" == *".$NETBIRD_DNS_DOMAIN" ]] +} + +# Write dnsmasq-compatible address= records for all connected NetBird peers +# to $NETBIRD_DNS_CONF_PATH in the mitmproxy-config shared volume. wg-client +# reads this file via patch_dnsmasq_from_netbird_volume() so that NetBird +# FQDNs resolve inside agent containers without wg-client running NetBird. +# +# Also writes a `server=//` forward line when the management +# server has published a nameserver group (NetBird >= 0.28 with DNS enabled in +# the dashboard), enabling full wildcard resolution under the mesh domain. +# `local=//` is omitted when forwarding: it cancels `server=/`. +publish_netbird_dns() { + command -v jq >/dev/null 2>&1 || return 0 + local status_json + status_json=$(netbird status --json 2>/dev/null) || return 0 + [[ -n "$status_json" ]] || return 0 + + local tmp_file peer_records have_server=false + tmp_file=$(mktemp) + peer_records=$(mktemp) + + # Nameserver forward line (best-effort; skipped when 0 nameservers configured). + local ns_ip + ns_ip=$(printf '%s' "$status_json" | jq -r ' + first( + (.dnsServers // .nameservers // .dns // [])[] + | select(.domains != null and (.domains | length) > 0) + | (.servers // .ips // [.ip // empty] // [])[] + | select(type == "string" and length > 0) + | split(":")[0] + ) // empty + ' 2>/dev/null) || true + if [[ -n "$ns_ip" ]] && [[ "$ns_ip" =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ || ( "$ns_ip" =~ ^[0-9a-fA-F:]+$ && "$ns_ip" == *:* ) ]]; then + printf 'server=/%s/%s\n' "$NETBIRD_DNS_DOMAIN" "$ns_ip" >> "$tmp_file" + have_server=true + fi + + # Per-peer address= records. + # NetBird >= 0.28 renamed the peer address field from `ip` to `netbirdIp` + # in `netbird status --json`. Accept either so DNS publishing keeps working + # across client versions; strip a trailing /prefix if present. + while IFS=$'\t' read -r fqdn ip; do + [[ -n "$fqdn" && -n "$ip" ]] || continue + netbird_dns_fqdn_allowed "$fqdn" || continue + [[ "$ip" =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]] || continue + # host-record= gives an exact FQDN mapping; address= also covers subdomains. + printf 'host-record=%s,%s\n' "$fqdn" "$ip" >> "$peer_records" + printf 'address=/%s/%s\n' "$fqdn" "$ip" >> "$peer_records" + # Alias without auto-generated four-octet IP suffix + # (myproject-proxy-100-64-0-5.netbird.selfhosted → myproject-proxy). + local hostname="${fqdn%.${NETBIRD_DNS_DOMAIN}}" + if [[ "$hostname" =~ ^(.+)-[0-9]{1,3}-[0-9]{1,3}-[0-9]{1,3}-[0-9]{1,3}$ ]]; then + local alias_fqdn="${BASH_REMATCH[1]}.${NETBIRD_DNS_DOMAIN}" + if netbird_dns_fqdn_allowed "$alias_fqdn"; then + printf 'host-record=%s,%s\n' "$alias_fqdn" "$ip" >> "$peer_records" + printf 'address=/%s/%s\n' "$alias_fqdn" "$ip" >> "$peer_records" + fi + fi + done < <(printf '%s' "$status_json" | jq -r ' + (.peers.details // [])[] + | . as $p + | ($p.netbirdIp // $p.ip // empty | tostring | split("/")[0]) as $ip + | select($p.fqdn != null and ($p.fqdn | tostring | length) > 0 and ($ip | test("^[0-9.]+$|^[0-9a-fA-F:]+$"))) + | [$p.fqdn, $ip] | @tsv + ' 2>/dev/null) + + if [[ -s "$peer_records" ]]; then + if [[ "$have_server" != true ]]; then + printf 'local=/%s/\n' "$NETBIRD_DNS_DOMAIN" >> "$tmp_file" + fi + cat "$peer_records" >> "$tmp_file" + fi + rm -f "$peer_records" + + # Atomically replace, including an empty file so vanished peers truncate. + if [[ -s "$tmp_file" || -e "$NETBIRD_DNS_CONF_PATH" ]]; then + if ! diff -q "$tmp_file" "$NETBIRD_DNS_CONF_PATH" >/dev/null 2>&1; then + cp "$tmp_file" "${NETBIRD_DNS_CONF_PATH}.tmp" + mv "${NETBIRD_DNS_CONF_PATH}.tmp" "$NETBIRD_DNS_CONF_PATH" + echo "[mitmproxy] Published NetBird DNS records to volume." >&2 + fi + fi + rm -f "$tmp_file" +} + +# Periodically publish NetBird peer DNS records to the shared volume. +supervise_netbird_dns_publish() { + while true; do + sleep 10 + publish_netbird_dns 2>/dev/null || true + done +} + +# Drop stale healthcheck sentinels before mitmweb or NetBird start. The +# mitmproxy-config volume persists across restarts; leaving dns.conf in place +# lets the healthcheck pass and wg-client read the previous run's upstream. +clear_mitmproxy_health_sentinels() { + mkdir -p "$MITMPROXY_PUBLIC" "$MITMPROXY_HOME" + chown -R mitmproxy:mitmproxy "$MITMPROXY_PUBLIC" 2>/dev/null || true + rm -f "$MITMPROXY_HOME/dns.conf" "$MITMPROXY_PUBLIC/mitmproxy-ca-cert.pem" +} + +# Copy the CA cert onto the agent-facing volume once mitmproxy writes it. +# Healthcheck gates on this file (same as the stock compose entrypoint). +publish_mitmproxy_ca_loop() { + ( + while [[ ! -f "$MITMPROXY_HOME/mitmproxy-ca-cert.pem" ]]; do + sleep 1 + done + cp "$MITMPROXY_HOME/mitmproxy-ca-cert.pem" "$MITMPROXY_PUBLIC/mitmproxy-ca-cert.pem.tmp" + mv "$MITMPROXY_PUBLIC/mitmproxy-ca-cert.pem.tmp" "$MITMPROXY_PUBLIC/mitmproxy-ca-cert.pem" + ) & +} + +# Extra CA bundles bind-mounted by apply_upstream_ca_bundles. enable_netbird +# deletes the compose entrypoint that used to install them, so the NetBird +# image entrypoint must do it before docker-entrypoint.sh drops privileges. +install_upstream_ca_bundles() { + local src="${UPSTREAM_CA_DIR:-/upstream-ca}" + local dest="${UPSTREAM_CA_INSTALL_DIR:-/usr/local/share/ca-certificates}" + local bundle="${UPSTREAM_CA_CERTIFI_BUNDLE:-}" + [[ -d "$src" ]] || return 0 + shopt -s nullglob + local certs=("$src"/*.crt) + ((${#certs[@]} > 0)) || return 0 + mkdir -p "$dest" + cp "$src"/*.crt "$dest/" || return 1 + if [[ "$dest" == "/usr/local/share/ca-certificates" ]] \ + && command -v update-ca-certificates >/dev/null 2>&1; then + update-ca-certificates >/dev/null || return 1 + fi + if [[ -z "$bundle" ]]; then + command -v python3 >/dev/null 2>&1 || return 1 + bundle=$(python3 -c 'import certifi; print(certifi.where())') || return 1 + fi + cat "$src"/*.crt >> "$bundle" +} + +# wt0 is in this netns and NetBird's default policy is all-to-all. mitmproxy +# is a mesh client here: allow only replies to connections it initiated. +# A port list would miss mitmproxy's userspace WireGuard (UDP 51820). +lockdown_wt0_ingress() { + local iface="${1:-$NETBIRD_IFACE}" + command -v iptables >/dev/null 2>&1 || return 0 + ip link show "$iface" >/dev/null 2>&1 || return 0 + iptables -C INPUT -i "$iface" -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT 2>/dev/null \ + || iptables -I INPUT -i "$iface" -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT + iptables -C INPUT -i "$iface" -j DROP 2>/dev/null \ + || iptables -A INPUT -i "$iface" -j DROP +} + +ensure_mitmweb_password() { + local pwfile="${MITMPROXY_WEB_PASSWORD_FILE}" + mkdir -p "$(dirname "$pwfile")" + if [[ ! -s "$pwfile" ]]; then + umask 077 + dd if=/dev/urandom bs=18 count=1 2>/dev/null | base64 | tr -d '/+\n=' >"$pwfile" + echo "[mitmproxy] generated mitmweb password; cat $pwfile" >&2 + fi + cat "$pwfile" +} + +# Mesh enrollment must not block the L7 proxy (healthcheck / wg-client). +maybe_start_netbird_mesh() { + export NETBIRD_AFTER_UP=lockdown_wt0_ingress + local _settings_file="/config/settings.json" + local _setup_key_from_compose=0 + [[ -n "${NB_SETUP_KEY:-}" ]] && _setup_key_from_compose=1 + if [[ -f "$_settings_file" ]] && command -v jq >/dev/null 2>&1; then + if [[ -z "${NB_MANAGEMENT_URL:-}" ]]; then + local _enrollment_url + _enrollment_url=$(jq -r '.netbird_enrollment_management_url // .netbird_management_url // empty' "$_settings_file" 2>/dev/null || true) + if [[ -n "$_enrollment_url" ]]; then + NB_MANAGEMENT_URL="$_enrollment_url" + export NB_MANAGEMENT_URL + fi + fi + fi + + if ! netbird_prepare_enroll_credentials; then + echo "[mitmproxy] Failed to prepare NetBird enroll credentials; starting L7 proxy without mesh." >&2 + return 0 + elif [[ -n "${NB_SETUP_KEY:-}" ]]; then + if [[ "$_setup_key_from_compose" -eq 0 ]]; then + echo "[mitmproxy] Loaded NB_SETUP_KEY from $_settings_file (compose did not pass it)." >&2 + fi + echo "[mitmproxy] NB_SETUP_KEY is set (${#NB_SETUP_KEY} chars); enrolling NetBird mesh." >&2 + if start_netbird "$NETBIRD_IFACE"; then + netbird_set_dns_label + publish_netbird_dns 2>/dev/null || true + supervise_netbird_daemon "$NETBIRD_IFACE" & + supervise_netbird_dns_publish & + else + echo "[mitmproxy] NetBird enrollment failed; starting L7 proxy without mesh." >&2 + echo "[mitmproxy] Check netbird_enrollment_key / NB_MANAGEMENT_URL, then recreate mitmproxy." >&2 + supervise_netbird_daemon "$NETBIRD_IFACE" & + supervise_netbird_dns_publish & + fi + else + echo "[mitmproxy] NB_SETUP_KEY not set; starting without NetBird mesh." >&2 + echo "[mitmproxy] Hint: ensure compose passes NB_SETUP_KEY (enable_netbird) and sandcat exports netbird_enrollment_key." >&2 + fi +} + +netbird_mitmproxy_cleanup() { + netbird_shutdown + if [[ -n "${MITMPROXY_CHILD_PID:-}" ]]; then + kill "$MITMPROXY_CHILD_PID" 2>/dev/null || true + fi + local job + for job in $(jobs -p); do + kill "$job" 2>/dev/null || true + done + if [[ -n "${MITMPROXY_CHILD_PID:-}" ]]; then + wait "$MITMPROXY_CHILD_PID" 2>/dev/null || true + fi +} + +main() { + # Stay PID 1 so SIGTERM can run netbird down. Enrollment stays in the + # background so NetBird cannot stall mitmweb / wg-client. + clear_mitmproxy_health_sentinels + install_upstream_ca_bundles || exit 1 + publish_mitmproxy_ca_loop + + local password + password=$(ensure_mitmweb_password) + local -a cmd_args=() + while [[ $# -gt 0 ]]; do + if [[ "$1" == "--set" && "${2-}" == web_password=* ]]; then + cmd_args+=(--set "web_password=${password}") + shift 2 + continue + fi + cmd_args+=("$1") + shift + done + + maybe_start_netbird_mesh & + + docker-entrypoint.sh "${cmd_args[@]}" & + MITMPROXY_CHILD_PID=$! + trap netbird_mitmproxy_cleanup TERM INT + wait "$MITMPROXY_CHILD_PID" + local status=$? + trap - TERM INT + netbird_shutdown + exit "$status" +} + +if [[ "${BASH_SOURCE[0]}" = "${0}" ]]; then + main "$@" +fi diff --git a/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py b/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py index b5a81ab9..ca8a3060 100644 --- a/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py +++ b/cli/templates/devcontainer/sandcat/scripts/mitmproxy_addon_common.py @@ -48,6 +48,7 @@ import shlex import subprocess import sys +import time from fnmatch import fnmatch from mitmproxy import ctx, dns, http @@ -199,6 +200,34 @@ def _pass_cli_session_is_pat(stdout: str) -> bool: return bool(_PAT_SESSION_MARKER.search(stdout or "")) +_ENABLED_TRUE = {"true", "1", "yes", "on"} +_ENABLED_FALSE = {"false", "0", "no", "off"} + + +def _parse_rule_enabled(raw) -> bool | None: + """Interpret a network rule's ``enabled`` value; None when unrecognized. + + JSON settings are hand-edited, so ``false``, ``"false"``, ``"0"`` and ``0`` + all show up in the wild and must all disable the rule. + """ + if isinstance(raw, bool): + return raw + if isinstance(raw, (int, float)): + return raw != 0 + if isinstance(raw, str): + value = raw.strip().lower() + if value in _ENABLED_FALSE: + return False + if value in _ENABLED_TRUE: + return True + return None + + +def _rule_enabled(rule: dict) -> bool: + """Whether a network rule applies. Unrecognized values keep the default (on).""" + return _parse_rule_enabled(rule.get("enabled", True)) is not False + + class SandcatAddon: """Base sandcat addon: network policy + secret substitution.""" @@ -241,6 +270,7 @@ def load(self, loader): self._pass_cli_logged_in = self._pass_cli_login_if_needed(has_pass_secrets) if has_pass_secrets and self._pass_cli_logged_in: self._verify_pat_auth_or_die() + self._pass_cli_warmup() self.env = merged["env"] self._load_secrets(merged["secrets"]) @@ -260,6 +290,10 @@ def _on_settings_merged(self, merged: dict): """Hook: subclasses may inspect merged settings (e.g., feature flags).""" pass + def done(self): + """mitmproxy shutdown hook.""" + pass + @staticmethod def _configure_op_token(token: str | None): """Set OP_SERVICE_ACCOUNT_TOKEN from settings if not already in the environment.""" @@ -414,6 +448,42 @@ def _deep_merge_dict(base: dict, overlay: dict) -> dict: result[key] = value return result + @staticmethod + def _pass_cli_warmup(): + """Prime the local vault cache before reading individual items. + + The first ``pass-cli item view`` after login triggers a full vault sync + from Proton's servers, which can take 30-60 s on a cold start and exceed + the per-item timeout. Running ``pass-cli vault list`` first forces that + sync in one place with a generous timeout so subsequent item reads hit the + local cache. Failures are non-fatal — item reads will attempt the sync + themselves and may fail if the network is truly unavailable. + """ + attempts = 3 + for attempt in range(1, attempts + 1): + try: + result = subprocess.run( + ["pass-cli", "vault", "list"], + capture_output=True, text=True, timeout=60, + ) + if result.returncode == 0: + ctx.log.info("pass-cli vault cache warmed up") + return + ctx.log.warn( + f"pass-cli vault list failed (attempt {attempt}/{attempts}): " + f"{result.stderr.strip()}" + ) + except subprocess.TimeoutExpired: + ctx.log.warn( + f"pass-cli vault list timed out (attempt {attempt}/{attempts})" + ) + if attempt < attempts: + time.sleep(5) + ctx.log.warn( + "pass-cli vault warmup failed after all attempts; " + "item reads will attempt their own sync" + ) + @staticmethod def _merge_settings(layers: list[dict]) -> dict: """Merge settings from multiple layers (lowest to highest precedence). @@ -610,20 +680,31 @@ def _resolve_secret_value(cls, name: str, entry: dict) -> str: raise ValueError( f"Secret {name!r}: 'pass' value must start with 'pass://', got {pass_ref!r}" ) - try: - result = subprocess.run( - ["pass-cli", "item", "view", pass_ref], - capture_output=True, text=True, timeout=30, - ) - except FileNotFoundError: - raise RuntimeError( - f"Secret {name!r}: 'pass-cli' not found. Install Proton Pass CLI to use pass:// references." - ) from None - if result.returncode != 0: - raise RuntimeError( - f"Secret {name!r}: 'pass-cli' read failed: {result.stderr.strip()}" - ) - return cls._normalize_secret_value(result.stdout.strip()) + attempts = 3 + last_error = None + for attempt in range(1, attempts + 1): + try: + result = subprocess.run( + ["pass-cli", "item", "view", pass_ref], + capture_output=True, text=True, timeout=60, + ) + except FileNotFoundError: + raise RuntimeError( + f"Secret {name!r}: 'pass-cli' not found. Install Proton Pass CLI to use pass:// references." + ) from None + except subprocess.TimeoutExpired: + last_error = f"timed out after 60 s (attempt {attempt}/{attempts})" + if attempt < attempts: + time.sleep(5) + continue + if result.returncode == 0: + return cls._normalize_secret_value(result.stdout.strip()) + last_error = result.stderr.strip() + if attempt < attempts: + time.sleep(5) + raise RuntimeError( + f"Secret {name!r}: 'pass-cli' read failed after {attempts} attempts: {last_error}" + ) @staticmethod def _normalize_secret_value(value) -> str: @@ -634,6 +715,15 @@ def _normalize_secret_value(value) -> str: def _load_network_rules(self, raw_rules: list): self.network_rules = self._expand_network_presets(raw_rules) + for rule in self.network_rules: + if not isinstance(rule, dict) or "enabled" not in rule: + continue + if _parse_rule_enabled(rule["enabled"]) is None: + ctx.log.warn( + f"Network rule for host {rule.get('host')!r}: unrecognized " + f"'enabled' value {rule['enabled']!r}; the rule stays enabled. " + "Use true/false to control it." + ) ctx.log.info(f"Loaded {len(self.network_rules)} network rule(s)") @classmethod @@ -799,6 +889,8 @@ def _write_extra_hosts(self, extra_hosts: dict[str, str]): def _find_matching_rule(self, method: str | None, host: str) -> dict | None: host = host.lower().rstrip(".") for rule in self.network_rules: + if not _rule_enabled(rule): + continue if not fnmatch(host, rule["host"].lower()): continue rule_method = rule.get("method") @@ -843,6 +935,15 @@ def _write_placeholders_env(self): lines.append(f"export {name}={shlex.quote(value)}") for name, entry in self.secrets.items(): lines.append(f"export {name}={shlex.quote(entry['placeholder'])}") + # Publish the NetBird mesh DNS domain so the agent can form FQDNs like + # .$SANDCAT_NETBIRD_DNS_DOMAIN without hard-coding the domain. + # Set on the mitmproxy container via NETBIRD_DNS_DOMAIN (injected by + # enable_netbird() in composefile.bash). Not emitted when NetBird is disabled. + netbird_dns_domain = os.environ.get("NETBIRD_DNS_DOMAIN", "") + if netbird_dns_domain: + lines.append( + f"export SANDCAT_NETBIRD_DNS_DOMAIN={shlex.quote(netbird_dns_domain)}" + ) self._atomic_write_text(SANDCAT_ENV_PATH, "\n".join(lines) + "\n") def _write_cursor_cli_config(self, merged: dict): diff --git a/cli/templates/devcontainer/sandcat/scripts/netbird-peer-lifecycle.sh b/cli/templates/devcontainer/sandcat/scripts/netbird-peer-lifecycle.sh new file mode 100755 index 00000000..28720e29 --- /dev/null +++ b/cli/templates/devcontainer/sandcat/scripts/netbird-peer-lifecycle.sh @@ -0,0 +1,588 @@ +#!/bin/bash + +netbird_peer_log() { + local prefix="${NETBIRD_PEER_LOG_PREFIX:-netbird}" + echo "[${prefix}] $*" >&2 +} + +NETBIRD_PASS_CLI_LOGGED_IN=0 +NETBIRD_PASS_CLI_WARMED=0 + +netbird_pass_cli_login_once() { + if [[ "${NETBIRD_PASS_CLI_LOGGED_IN}" -eq 1 ]]; then + return 0 + fi + pass-cli login || { + netbird_peer_log "pass-cli login failed" + return 1 + } + # Reject full-account credentials the same way the mitmproxy addon does: + # pass-cli info prints "Personal Access Token:" only for PAT sessions. + local info_out + info_out=$(pass-cli info 2>/dev/null) || { + netbird_peer_log "pass-cli info failed; cannot verify PAT session — logging out" + pass-cli logout >/dev/null 2>&1 || true + return 1 + } + if ! printf '%s' "$info_out" | grep -qiE '^\s*-?\s*personal\s+access\s+token\s*:'; then + netbird_peer_log "pass-cli session is not a Personal Access Token — logging out" + pass-cli logout >/dev/null 2>&1 || true + return 1 + fi + NETBIRD_PASS_CLI_LOGGED_IN=1 +} + +netbird_pass_cli_warmup_once() { + if [[ "${NETBIRD_PASS_CLI_WARMED}" -eq 1 ]]; then + return 0 + fi + # First item view after login can stall on a cold vault sync. Warm the + # cache once; failure is non-fatal because item view retries below. + timeout 60 pass-cli vault list >/dev/null 2>&1 || { + netbird_peer_log "pass-cli vault list warmup failed" + } + NETBIRD_PASS_CLI_WARMED=1 +} + +netbird_resolve_secret_ref() { + local value=${1-} + local output + if [[ "$value" == op://* ]]; then + output=$(timeout 60 op read "$value") || { + netbird_peer_log "op read failed for ${value}" + return 1 + } + printf '%s\n' "$output" + return 0 + fi + if [[ "$value" == pass://* ]]; then + netbird_pass_cli_login_once || return 1 + netbird_pass_cli_warmup_once + if output=$(timeout 60 pass-cli item view "$value"); then + printf '%s\n' "$output" + return 0 + fi + output=$(timeout 60 pass-cli item view "$value") || { + netbird_peer_log "pass-cli item view failed for ${value}" + return 1 + } + printf '%s\n' "$output" + return 0 + fi + printf '%s' "$value" +} + +netbird_json_field() { + local file=$1 key=$2 + [[ -f "$file" ]] || { printf 'null'; return 0; } + command -v jq >/dev/null 2>&1 || { printf 'null'; return 0; } + jq -c --arg k "$key" '.[$k] // null' "$file" +} + +netbird_flatten_secret_json() { + local json=$1 + [[ "$json" == "null" || -z "$json" ]] && { printf ''; return 0; } + command -v jq >/dev/null 2>&1 || { printf '%s' "$json"; return 0; } + local typ + typ=$(printf '%s' "$json" | jq -r 'type') + case "$typ" in + string) + printf '%s' "$(printf '%s' "$json" | jq -r '.')" + ;; + object) + local n + n=$(printf '%s' "$json" | jq '[.value, .op, .pass] | map(select(. != null)) | length') + if [[ "$n" != "1" ]]; then + netbird_peer_log "settings secret must specify exactly one of value, op, or pass" + return 1 + fi + printf '%s' "$(printf '%s' "$json" | jq -r '.value // .op // .pass')" + ;; + *) + netbird_peer_log "settings secret must be a string or object" + return 1 + ;; + esac +} + +netbird_prepare_enroll_credentials() { + local settings_path="${NETBIRD_SETTINGS_PATH:-/config/settings.json}" + local raw flat + if [[ -z "${NB_SETUP_KEY:-}" ]]; then + raw=$(netbird_json_field "$settings_path" netbird_enrollment_key) + flat=$(netbird_flatten_secret_json "$raw") || return 1 + [[ -n "$flat" ]] && export NB_SETUP_KEY="$flat" + fi + if [[ -z "${NB_API_TOKEN:-}" ]]; then + raw=$(netbird_json_field "$settings_path" netbird_api_token) + flat=$(netbird_flatten_secret_json "$raw") || return 1 + [[ -n "$flat" ]] && export NB_API_TOKEN="$flat" + fi + if [[ -n "${NB_SETUP_KEY:-}" ]]; then + NB_SETUP_KEY=$(netbird_resolve_secret_ref "$NB_SETUP_KEY") || return 1 + export NB_SETUP_KEY + fi + if [[ -n "${NB_API_TOKEN:-}" ]]; then + NB_API_TOKEN=$(netbird_resolve_secret_ref "$NB_API_TOKEN") || return 1 + export NB_API_TOKEN + fi +} + +netbird_daemon_status() { + local json status + json=$(netbird status --json 2>/dev/null) || return 1 + status=$(printf '%s' "$json" | jq -r '.status // .daemonStatus // .daemon.status // empty' 2>/dev/null) || return 1 + [[ -n "$status" ]] || return 1 + printf '%s' "$status" +} + +# 0.72 Start() creates /var/lib/netbird/default.json and stays NeedsLogin until +# `netbird up --setup-key`. That file is not enrolled identity. +netbird_profile_is_enrolled() { + local status + status=$(netbird_daemon_status) || return 1 + [[ "$status" != "NeedsLogin" && "$status" != "LoginFailed" && "$status" != "NeedsLoginSSO" ]] +} + +netbird_local_state_present() { + local state_root="${NETBIRD_STATE_ROOT:-/var/lib/netbird}" + + [[ -s "${state_root}/config.json" ]] && return 0 + [[ -s /etc/netbird/config.json ]] && return 0 + if [[ -s "${state_root}/default.json" ]] && netbird_profile_is_enrolled; then + return 0 + fi + return 1 +} + +# Go encoding/json for net/url.URL (NetBird 0.72 Config.ManagementURL). +netbird_go_url_json() { + local url=$1 + [[ "$url" =~ ^(https?)://([^/?#]+)([^?#]*) ]] || return 1 + jq -nc --arg Scheme "${BASH_REMATCH[1]}" --arg Host "${BASH_REMATCH[2]}" --arg Path "${BASH_REMATCH[3]}" \ + '{Scheme:$Scheme, Host:$Host, Path:$Path}' +} + +# NetBird 0.72 unmarshals ManagementURL/AdminURL as *url.URL. A JSON string +# (0.28 seed format) fatal's the daemon. Never create default.json; only +# coerce leftover string URLs and refresh ManagementURL from NB_MANAGEMENT_URL +# so a cloud enroll cannot inherit a stale self-hosted host from the volume. +netbird_prepare_local_management_profile() { + local mgmt_url="${NB_MANAGEMENT_URL:-https://api.netbird.io}" + local state_root="${NETBIRD_STATE_ROOT:-/var/lib/netbird}" + local profile_file="${state_root}/default.json" + local url_json tmp + + command -v jq >/dev/null 2>&1 || return 0 + [[ -f "$profile_file" ]] || return 0 + + tmp=$(mktemp) + local jq_rc=0 + if url_json=$(netbird_go_url_json "$mgmt_url"); then + if [[ -n "${NETBIRD_WG_PORT:-}" ]]; then + jq --argjson mgmt "$url_json" --arg iface "${NETBIRD_IFACE:-wt0}" --argjson port "${NETBIRD_WG_PORT}" \ + '.ManagementURL = $mgmt | .AdminURL = $mgmt | .WgIface = $iface | .WgPort = $port' \ + "$profile_file" >"$tmp" || jq_rc=$? + else + jq --argjson mgmt "$url_json" --arg iface "${NETBIRD_IFACE:-wt0}" \ + '.ManagementURL = $mgmt | .AdminURL = $mgmt | .WgIface = $iface' \ + "$profile_file" >"$tmp" || jq_rc=$? + fi + else + jq ' + def coerce: + if type == "string" then + (capture("^(?https?)://(?[^/?#]+)(?[^?#]*)") // .) + else . end; + .ManagementURL |= coerce + | .AdminURL |= coerce + ' "$profile_file" >"$tmp" || jq_rc=$? + fi + if [[ "$jq_rc" -ne 0 ]]; then + rm -f "$tmp" + return "$jq_rc" + fi + mv "$tmp" "$profile_file" +} + +netbird_resolve_api_token() { + local settings_path="${NETBIRD_SETTINGS_PATH:-/config/settings.json}" + local token raw flat + + if [[ -n "${NB_API_TOKEN:-}" ]]; then + token="$NB_API_TOKEN" + else + raw=$(netbird_json_field "$settings_path" netbird_api_token) + flat=$(netbird_flatten_secret_json "$raw") || return 1 + token="$flat" + fi + + [[ -n "$token" ]] || return 1 + token=$(netbird_resolve_secret_ref "$token") || return 1 + [[ -n "$token" ]] || return 1 + printf '%s\n' "$token" +} + +netbird_mgmt_find_peer_id_by_name() { + local peer_name=$1 + local management_url="${NB_MANAGEMENT_URL:?NB_MANAGEMENT_URL is required}" + local token peers matches match_count + token=$(netbird_resolve_api_token) || { + netbird_peer_log "netbird_api_token / NB_API_TOKEN required to query management peers" + return 1 + } + + peers=$(netbird_curl "$token" "${management_url%/}/api/peers") || return 1 + matches=$(printf '%s' "$peers" \ + | jq -c --arg name "$peer_name" \ + '[.[] | select( + ((.name // "") | ascii_downcase) == ($name | ascii_downcase) + or ((.hostname // "") | ascii_downcase) == ($name | ascii_downcase) + or ((.dns_label // "") | ascii_downcase) == ($name | ascii_downcase) + ) | .id]') || return 1 + match_count=$(printf '%s' "$matches" | jq 'length') || return 1 + if ((match_count > 1)); then + netbird_peer_log "multiple management peers match '${peer_name}'; refusing ambiguous replacement" + return 1 + fi + + printf '%s' "$matches" | jq -r '.[0] // empty' +} + +# curl against the management API without putting the PAT on argv. +# $1 is the token; remaining args are extra curl arguments (URL last). +netbird_curl() { + local token=$1 + shift + local header_file rc + local old_umask + old_umask=$(umask) + umask 077 + header_file=$(mktemp) + umask "$old_umask" + printf 'Authorization: Token %s\n' "$token" >"$header_file" + curl -sf --max-time 10 -H "@${header_file}" "$@" + rc=$? + rm -f "$header_file" + return "$rc" +} + +netbird_mgmt_delete_peer_by_id() { + local peer_id=$1 + local token + + token=$(netbird_resolve_api_token) || { + netbird_peer_log "netbird_api_token / NB_API_TOKEN required to delete management peer '${peer_id}'" + return 1 + } + + netbird_curl "$token" -X DELETE \ + "${NB_MANAGEMENT_URL%/}/api/peers/${peer_id}" >/dev/null +} + +netbird_mgmt_delete_peer_by_name() { + local peer_name=$1 + local peer_id + + netbird_resolve_api_token >/dev/null || { + netbird_peer_log "netbird_api_token / NB_API_TOKEN required to delete management peer '${peer_name}'" + return 1 + } + + peer_id=$(netbird_mgmt_find_peer_id_by_name "$peer_name") || return 1 + [[ -n "$peer_id" ]] || return 0 + + netbird_mgmt_delete_peer_by_id "$peer_id" +} + +netbird_replace_same_name_peer_if_needed() { + local peer_name="${NB_PEER_NAME:?NB_PEER_NAME is required}" + local peer_id + + if netbird_local_state_present; then + netbird_peer_log "local state present — will reconnect as '${peer_name}'" + return 0 + fi + + # Without a PAT we cannot query management; skip replace and let netbird up + # enroll fresh. FQDN may get an IP suffix but enrollment is not blocked. + if ! netbird_resolve_api_token >/dev/null 2>&1; then + netbird_peer_log "no API token — skipping same-name peer check; will enroll fresh" + return 0 + fi + + peer_id=$(netbird_mgmt_find_peer_id_by_name "$peer_name") || return 1 + if [[ -z "$peer_id" ]]; then + netbird_peer_log "no existing management peer named '${peer_name}' — will enroll fresh" + return 0 + fi + + netbird_peer_log "replace existing management peer '${peer_name}' (id=${peer_id}) before enroll" + netbird_mgmt_delete_peer_by_id "$peer_id" +} + +netbird_set_dns_label() { + local peer_name="${NB_PEER_NAME:?NB_PEER_NAME is required}" + local management_url="${NB_MANAGEMENT_URL:-}" + local token current_fqdn peer_id result new_fqdn payload + + [[ -n "$management_url" ]] || return 0 + command -v curl >/dev/null 2>&1 || return 0 + command -v jq >/dev/null 2>&1 || return 0 + + token=$(netbird_resolve_api_token) || { + netbird_peer_log "netbird_api_token not set; skipping dns_label update (FQDN will include IP suffix)." + return 0 + } + current_fqdn=$(netbird status --json 2>/dev/null \ + | jq -r '.fqdn // empty' 2>/dev/null || true) + if [[ -z "$current_fqdn" ]]; then + netbird_peer_log "could not read local peer FQDN; skipping dns_label update." + return 0 + fi + + peer_id=$(netbird_curl "$token" "${management_url%/}/api/peers" \ + | jq -r --arg fqdn "$current_fqdn" \ + 'first(.[] | select(.fqdn == $fqdn) | .id) // empty' 2>/dev/null || true) + if [[ -z "$peer_id" ]]; then + netbird_peer_log "could not find peer ID for FQDN ${current_fqdn}; skipping dns_label update." + return 0 + fi + + payload=$(jq -cn --arg dns_label "$peer_name" '{dns_label: $dns_label}') + result=$(netbird_curl "$token" -X PUT \ + -H "Content-Type: application/json" \ + -d "$payload" \ + "${management_url%/}/api/peers/${peer_id}" 2>/dev/null) || { + netbird_peer_log "dns_label update failed for peer ${peer_id}; FQDN may retain IP suffix." + return 0 + } + + new_fqdn=$(printf '%s' "$result" | jq -r '.fqdn // empty' 2>/dev/null || true) + netbird_peer_log "dns_label set → FQDN: ${new_fqdn:-${peer_name}.}" +} + +wait_until() { + local max="$1" delay="$2" msg="$3" + shift 3 + local attempt=0 + while ! "$@"; do + if [[ "$attempt" -ge "$max" ]]; then + echo "$msg" >&2 + return 1 + fi + sleep "$delay" + attempt=$((attempt + 1)) + done +} + +netbird_management_url_host() { + local url=$1 + [[ "$url" =~ ^https?://([^/:]+) ]] || return 1 + printf '%s' "${BASH_REMATCH[1]}" +} + +netbird_management_url_host_is_literal_ipv4() { + local host=$1 + [[ "$host" =~ ^([0-9]{1,3}\.){3}[0-9]{1,3}$ ]] +} + +netbird_management_url_port() { + local url=$1 + if [[ "$url" =~ :([0-9]+)(/|$|\?) ]]; then + printf '%s' "${BASH_REMATCH[1]}" + return 0 + fi + if [[ "$url" =~ ^https:// ]]; then + printf '443' + else + printf '80' + fi +} + +netbird_host_route_uses_gateway() { + local host_ip=$1 + local docker_gateway=$2 + [[ -n "$host_ip" && -n "$docker_gateway" ]] || return 1 + [[ "$host_ip" != "$docker_gateway" ]] +} + +# Allow enrollment against a self-hosted management server on the Docker host. +configure_netbird_host_management_access() { + local docker_gateway=$1 + local mgmt_url="${NB_MANAGEMENT_URL:-https://api.netbird.io}" + local host_ip port + + [[ -n "$docker_gateway" ]] || return 0 + + host_ip=$(netbird_management_url_host "$mgmt_url") || return 0 + netbird_management_url_host_is_literal_ipv4 "$host_ip" || return 0 + + port=$(netbird_management_url_port "$mgmt_url") + + if netbird_host_route_uses_gateway "$host_ip" "$docker_gateway"; then + netbird_peer_log "Routing NetBird management traffic to ${host_ip}:${port} via eth0 (via ${docker_gateway})." + ip -4 route add "${host_ip}/32" via "${docker_gateway}" dev eth0 2>/dev/null || true + else + netbird_peer_log "Routing NetBird management traffic to ${host_ip}:${port} via eth0." + ip -4 route add "${host_ip}/32" dev eth0 2>/dev/null || true + fi + + ip -4 rule add to "${host_ip}/32" lookup main priority 50 2>/dev/null || true + + iptables -C OUTPUT -o eth0 -d "$host_ip" -p tcp --dport "$port" -j ACCEPT 2>/dev/null \ + || iptables -I OUTPUT 1 -o eth0 -d "$host_ip" -p tcp --dport "$port" -j ACCEPT + iptables -C OUTPUT -o eth0 -d "$host_ip" -p udp --dport 3478 -j ACCEPT 2>/dev/null \ + || iptables -I OUTPUT 1 -o eth0 -d "$host_ip" -p udp --dport 3478 -j ACCEPT +} + +netbird_verify_host_management_reachable() { + local mgmt_url="${NB_MANAGEMENT_URL:-}" + local host_ip port check_url + + host_ip=$(netbird_management_url_host "$mgmt_url") || return 0 + netbird_management_url_host_is_literal_ipv4 "$host_ip" || return 0 + + port=$(netbird_management_url_port "$mgmt_url") + command -v curl >/dev/null 2>&1 || return 0 + + check_url="${mgmt_url%/}/api/instance" + wait_until 15 1 \ + "[${NETBIRD_PEER_LOG_PREFIX:-netbird}] Cannot reach NetBird management at ${mgmt_url}; is the server running on the host (port ${port})?" \ + curl -sf --max-time 5 "$check_url" >/dev/null +} + +netbird_export_service_env() { + local mgmt_url="${NB_MANAGEMENT_URL:-https://api.netbird.io}" + export NB_MANAGEMENT_URL="$mgmt_url" + if [[ -n "${NETBIRD_WG_PORT:-}" ]]; then + export NB_WIREGUARD_PORT="${NETBIRD_WG_PORT}" + fi + if netbird_management_url_host_is_literal_ipv4 "$(netbird_management_url_host "$mgmt_url" 2>/dev/null)"; then + export NB_USE_LEGACY_ROUTING=true + fi +} + +netbird_daemon_ready() { + netbird status >/dev/null 2>&1 +} + +ensure_netbird_service() { + netbird_prepare_local_management_profile + netbird_export_service_env + if netbird_daemon_ready; then + return 0 + fi + + netbird_peer_log "Starting NetBird service daemon ($(netbird version 2>/dev/null || echo unknown))." + netbird service run --log-file console & + + wait_until 30 1 \ + "[${NETBIRD_PEER_LOG_PREFIX:-netbird}] Timed out waiting for NetBird service daemon" \ + netbird_daemon_ready +} + +# Enroll without putting the setup key on argv. Bound by NETBIRD_UP_TIMEOUT. +netbird_up_enroll() { + local iface=$1 + local keyfile rc + local old_umask + local -a up_args + old_umask=$(umask) + umask 077 + keyfile=$(mktemp) + umask "$old_umask" + printf '%s' "${NB_SETUP_KEY}" >"$keyfile" + + up_args=( + up + --setup-key-file "$keyfile" + --management-url "${NB_MANAGEMENT_URL:-https://api.netbird.io}" + --hostname "${NB_PEER_NAME}" + --interface-name "${iface}" + ) + if [[ -n "${NETBIRD_WG_PORT:-}" ]]; then + up_args+=(--wireguard-port "${NETBIRD_WG_PORT}") + fi + + netbird_peer_log "Enrolling NetBird peer on ${iface} as '${NB_PEER_NAME}'${NETBIRD_WG_PORT:+ (WG port ${NETBIRD_WG_PORT})}." + timeout "${NETBIRD_UP_TIMEOUT:-60}" netbird "${up_args[@]}" + rc=$? + rm -f "$keyfile" + if [[ "$rc" -ne 0 ]]; then + netbird_peer_log "netbird up failed for ${iface}." + return "$rc" + fi + return 0 +} + +netbird_start() { + local iface="${1:-wt0}" + local docker_gateway + + docker_gateway=$(ip -4 route show default dev eth0 2>/dev/null | awk '{print $3}') + + ensure_netbird_service + configure_netbird_host_management_access "$docker_gateway" + netbird_verify_host_management_reachable + netbird_prepare_local_management_profile + if [[ -f /var/lib/netbird/default.json ]] \ + && grep -qE 'localhost|127\.0\.0\.1|\[::1\]' /var/lib/netbird/default.json 2>/dev/null; then + netbird down 2>/dev/null || true + fi + netbird_export_service_env + + netbird_prepare_enroll_credentials || return 1 + netbird_replace_same_name_peer_if_needed || return 1 + netbird_up_enroll "$iface" || return 1 + + if ! wait_until 30 1 \ + "[${NETBIRD_PEER_LOG_PREFIX:-netbird}] Timed out waiting for NetBird to bring up ${iface}" \ + ip link show "${iface}" >/dev/null 2>&1; then + return 1 + fi + if [[ -n "${NETBIRD_AFTER_UP:-}" ]]; then + "${NETBIRD_AFTER_UP}" "$iface" + fi +} + +start_netbird() { + netbird_start "$@" +} + +netbird_shutdown() { + netbird down >/dev/null 2>&1 || true +} + +netbird_supervise_daemon() { + local iface="${1:-wt0}" + local backoff="${NETBIRD_SUPERVISE_INTERVAL:-10}" + local max_backoff="${NETBIRD_SUPERVISE_MAX_BACKOFF:-60}" + + while true; do + sleep "$backoff" + if ! netbird_daemon_ready; then + netbird_peer_log "NetBird service daemon not responding; restarting." + netbird_export_service_env + netbird service run --log-file console & + wait_until 15 1 \ + "[${NETBIRD_PEER_LOG_PREFIX:-netbird}] Timed out waiting for NetBird service daemon" \ + netbird_daemon_ready || true + fi + if ! ip link show "${iface}" >/dev/null 2>&1; then + netbird_peer_log "NetBird interface ${iface} down; re-enrolling." + if netbird_start "$iface"; then + backoff="${NETBIRD_SUPERVISE_INTERVAL:-10}" + else + backoff=$((backoff * 2)) + if ((backoff > max_backoff)); then + backoff=$max_backoff + fi + fi + fi + done +} + +supervise_netbird_daemon() { + netbird_supervise_daemon "$@" +} diff --git a/cli/templates/devcontainer/sandcat/scripts/prepare-agent-sandcat-mount.sh b/cli/templates/devcontainer/sandcat/scripts/prepare-agent-sandcat-mount.sh new file mode 100755 index 00000000..69ec6d68 --- /dev/null +++ b/cli/templates/devcontainer/sandcat/scripts/prepare-agent-sandcat-mount.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +# Host-side helper for Dev Containers initializeCommand. +# Prepares the filtered .sandcat copy and writes .devcontainer/.env so +# compose can interpolate SANDCAT_AGENT_SANDCAT without mounting the live +# project directory. +set -euo pipefail + +root=$(cd "$(dirname "$0")/../../.." && pwd) +cd "$root" + +if [[ -z "${SCT_LIBDIR:-}" ]]; then + _sandcat_bin=$(command -v sandcat 2>/dev/null || true) + if [[ -z "$_sandcat_bin" && -x "${HOME}/.local/bin/sandcat" ]]; then + _sandcat_bin="${HOME}/.local/bin/sandcat" + fi + if [[ -z "$_sandcat_bin" ]]; then + echo "prepare-agent-sandcat-mount: sandcat CLI not found on PATH" >&2 + exit 1 + fi + SCT_LIBDIR=$(cd "$(dirname "$_sandcat_bin")/../lib" && pwd) + export SCT_LIBDIR +fi + +# shellcheck source=/dev/null +source "$SCT_LIBDIR/netbird.bash" +export_agent_sandcat_mount diff --git a/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh b/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh index 820b6036..fb99751a 100644 --- a/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh +++ b/cli/templates/devcontainer/sandcat/scripts/wg-client-init.sh @@ -20,6 +20,10 @@ DNSMASQ_CONF="/etc/dnsmasq-sandcat.conf" # inherit the resolv.conf this script writes for wg-client. We publish it here # for sibling app-init scripts to copy into their own /etc/resolv.conf. SHARED_RESOLV_CONF="/run/sandcat/resolv.conf" +# NetBird peer DNS records published by mitmproxy-init.sh into the shared volume. +# Format: dnsmasq-compatible address= and server= lines, one per line. +NETBIRD_PEERS_CONF="/mitmproxy-config/netbird-peers.conf" +DNSMASQ_PID_FILE="/run/sandcat/dnsmasq.pid" # Poll a command until it returns success or a timeout is hit. # @@ -130,6 +134,161 @@ write_resolv_conf() { } > "$resolv_conf" } +# Merge NetBird peer DNS records published by mitmproxy-init.sh into the +# dnsmasq config and restart dnsmasq so the running process picks them up. +# +# mitmproxy-init.sh writes dnsmasq-compatible local=, host-record=, address=, +# and server= lines to $NETBIRD_PEERS_CONF in the shared mitmproxy-config +# volume whenever peer state changes. This function replaces the marked +# NetBird block (and any legacy unmarked records for the same names) so a +# new mesh IP is not appended behind a stale one. SIGHUP is not enough: +# dnsmasq often keeps serving stale data after address=/host-record= changes. +# +# Idempotent and safe to call repeatedly. Returns 0 if the source file does +# not exist yet (NetBird disabled or mitmproxy still enrolling). +# +# Args: +# $1 - path to the dnsmasq config file to update +restart_dnsmasq() { + local conf="$1" + local killed=false + + if [[ -f "$DNSMASQ_PID_FILE" ]]; then + local dnsmasq_pid + dnsmasq_pid=$(tr -d '[:space:]' <"$DNSMASQ_PID_FILE" 2>/dev/null) || true + if [[ -n "$dnsmasq_pid" ]] && kill -0 "$dnsmasq_pid" 2>/dev/null; then + kill "$dnsmasq_pid" 2>/dev/null || true + killed=true + local attempt=0 + while dnsmasq-ready && [[ "$attempt" -lt 25 ]]; do + sleep 0.2 + attempt=$((attempt + 1)) + done + fi + rm -f "$DNSMASQ_PID_FILE" + fi + + if dnsmasq-ready 2>/dev/null; then + if [[ "$killed" == true ]]; then + echo "wg-client: previous dnsmasq still listening; not starting a second process" >&2 + return 1 + fi + return 0 + fi + + mkdir -p "$(dirname "$DNSMASQ_PID_FILE")" + dnsmasq --conf-file="$conf" --pid-file="$DNSMASQ_PID_FILE" + if ! wait_until 25 0.2 \ + "dnsmasq did not start after NetBird DNS restart" \ + dnsmasq-ready; then + return 1 + fi + echo "wg-client: dnsmasq restarted for NetBird DNS records." >&2 +} + +# Prefix that identifies a dnsmasq record regardless of its rdata (IP). +# Used to drop stale host-record=/address= lines when a peer's mesh IP changes. +netbird_dnsmasq_record_prefix() { + local line=$1 + case "$line" in + host-record=*,*) + printf '%s,' "${line%%,*}" + ;; + address=/*) + local rest="${line#address=/}" + printf 'address=/%s/' "${rest%%/*}" + ;; + local=/*) + local rest="${line#local=/}" + printf 'local=/%s/' "${rest%%/*}" + ;; + server=/*) + local rest="${line#server=/}" + printf 'server=/%s/' "${rest%%/*}" + ;; + *) + return 1 + ;; + esac +} + +patch_dnsmasq_from_netbird_volume() { + local conf="$1" + local peers_conf="$NETBIRD_PEERS_CONF" + local peers_stamp="${conf}.netbird-peers.stamp" + + [[ -f "$peers_conf" ]] || return 0 + + local peers_mtime=0 + peers_mtime=$(stat -c %Y "$peers_conf" 2>/dev/null || echo 0) + local recorded_mtime=0 + [[ -f "$peers_stamp" ]] && recorded_mtime=$(tr -d '[:space:]' <"$peers_stamp" 2>/dev/null || echo 0) + + # Unchanged source: skip the merge. The 5s supervisor would otherwise + # rewrite dnsmasq.conf on every tick. + if [[ -f "$peers_stamp" && "$peers_mtime" == "$recorded_mtime" ]]; then + return 0 + fi + + local begin="# BEGIN SANDCAT-NETBIRD-DNS" + local end="# END SANDCAT-NETBIRD-DNS" + local records tmp line prefix existing + records=$(mktemp) + tmp=$(mktemp) + { + echo "$begin" + while IFS= read -r line || [[ -n "$line" ]]; do + [[ -n "$line" ]] || continue + [[ "$line" =~ ^(local|host-record|address|server)= ]] || continue + printf '%s\n' "$line" + done < "$peers_conf" + echo "$end" + } > "$records" + + existing="" + if grep -qF "$begin" "$conf" 2>/dev/null; then + existing=$(awk -v b="$begin" -v e="$end" ' + $0==b {p=1} + p {print} + $0==e {p=0} + ' "$conf") + fi + + local reload_needed=false + if [[ "$existing" != "$(cat "$records")" ]]; then + awk -v b="$begin" -v e="$end" ' + $0==b {skip=1; next} + $0==e {skip=0; next} + !skip {print} + ' "$conf" > "$tmp" + mv "$tmp" "$conf" + tmp=$(mktemp) + + while IFS= read -r line || [[ -n "$line" ]]; do + prefix=$(netbird_dnsmasq_record_prefix "$line") || continue + [[ -n "$prefix" ]] || continue + awk -v p="$prefix" 'index($0, p) != 1 {print}' "$conf" > "$tmp" + mv "$tmp" "$conf" + tmp=$(mktemp) + done < "$records" + + cat "$records" >> "$conf" + reload_needed=true + echo "wg-client: updated dnsmasq records from NetBird volume." >&2 + fi + rm -f "$records" "$tmp" + + # Restart only if dnsmasq is already listening. Boot calls this *before* + # the first start so a persisted netbird-peers.conf would otherwise spawn + # dnsmasq here and then again in main() → EADDRINUSE on 127.0.0.1:53. + if [[ "$reload_needed" == true ]] || [[ "$peers_mtime" != "$recorded_mtime" ]]; then + if dnsmasq-ready 2>/dev/null; then + restart_dnsmasq "$conf" + fi + printf '%s\n' "$peers_mtime" > "$peers_stamp" + fi +} + main() { # Production behavior is errexit; kept inside main() so sourcing the file # (e.g. from bats tests) doesn't enable errexit in the caller's shell. @@ -260,7 +419,11 @@ main() { # - 127.0.0.11 traffic uses the lo interface (ACCEPT-ed), # - upstream queries go via wg0 (ACCEPT-ed). write_dnsmasq_conf "$DNS_CONF" "$DNSMASQ_CONF" "${search_domains[@]}" - dnsmasq --conf-file="$DNSMASQ_CONF" + # Merge any NetBird records already on the volume before the first dnsmasq + # start so boot-time resolution works even when mitmproxy enrolled first. + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" 2>/dev/null || true + mkdir -p "$(dirname "$DNSMASQ_PID_FILE")" + dnsmasq --conf-file="$DNSMASQ_CONF" --pid-file="$DNSMASQ_PID_FILE" # dnsmasq daemonizes after parsing its config; verify it actually bound # to 127.0.0.1:53 before we point /etc/resolv.conf at it. Without this @@ -307,6 +470,10 @@ main() { } >> /etc/hosts fi + # Apply any NetBird peer DNS records already published by mitmproxy. + # Best-effort: file may not exist yet if mitmproxy is still enrolling. + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" 2>/dev/null || true + # Signal readiness to containers waiting on the healthcheck. touch /tmp/wg-ready @@ -319,14 +486,19 @@ main() { # leave the agent attached to a now-destroyed namespace. Supervising dnsmasq # locally — instead of letting the container exit and rely on Docker's # restart policy — keeps the namespace intact across dnsmasq crashes. +# +# Also picks up NetBird peer DNS records published by mitmproxy on each +# iteration so that newly enrolled peers resolve without restarting wg-client. supervise_dnsmasq() { local conf="$1" while true; do sleep 5 if ! dnsmasq-ready; then echo "[wg-client] dnsmasq not listening; restarting" >&2 - dnsmasq --conf-file="$conf" || true + mkdir -p "$(dirname "$DNSMASQ_PID_FILE")" + dnsmasq --conf-file="$conf" --pid-file="$DNSMASQ_PID_FILE" || true fi + patch_dnsmasq_from_netbird_volume "$conf" 2>/dev/null || true done } diff --git a/cli/test/composefile/composefile.bats b/cli/test/composefile/composefile.bats index a34dacf6..e9ddf68c 100644 --- a/cli/test/composefile/composefile.bats +++ b/cli/test/composefile/composefile.bats @@ -21,16 +21,46 @@ services: cap_add: - SOME_CAPABILITY # need at least one entry so that we can add foot comments YAML + + # mitmproxy lives in the file compose-all.yml includes, never in + # compose-all.yml itself — Compose rejects a service declared in both. + PROXY_COMPOSE_FILE="$BATS_TEST_TMPDIR/sandcat/compose-proxy.yml" + mkdir -p "$BATS_TEST_TMPDIR/sandcat" + + cat >"$PROXY_COMPOSE_FILE" <<'YAML' +services: + mitmproxy: + image: mitmproxy/mitmproxy:latest + volumes: + - ~/.config/sandcat/settings.json:/config/settings.json:ro +YAML } teardown() { unstub_all } +# Count in-place yq writes without needing a real binary. Non-inplace +# queries (e.g. array length) print 1 so foot-comment helpers proceed. +stub_yq_inplace_counter() { + YQ_INPLACE_COUNT=0 + require() { return 0; } + yq() { + local a + for a in "$@"; do + if [[ "$a" == "-i" || "$a" == "--inplace" ]]; then + YQ_INPLACE_COUNT=$((YQ_INPLACE_COUNT + 1)) + return 0 + fi + done + echo 1 + } +} + @test "add_settings_volume adds settings mount to proxy service" { - add_settings_volume "$COMPOSE_FILE" ".sandcat/settings.json" + add_settings_volume "$PROXY_COMPOSE_FILE" ".sandcat/settings.json" - yq -e '.services.mitmproxy.volumes[] | select(. == ".sandcat:/config/project:ro")' "$COMPOSE_FILE" + yq -e '.services.mitmproxy.volumes[] | select(. == ".sandcat:/config/project:ro")' "$PROXY_COMPOSE_FILE" } @test "add_claude_config_volumes adds CLAUDE.md and settings.json" { @@ -273,8 +303,10 @@ assert_jetbrains_capabilities() { assert_customize_compose_file_common() { local compose_file=$1 - # Verify settings volume on proxy - yq -e '.services.mitmproxy.volumes[] | select(. == ".sandcat:/config/project:ro")' "$compose_file" + # Verify settings volume on proxy, re-based on the included file's directory + yq -e '.services.mitmproxy.volumes[] | select(. == "../.sandcat:/config/project:ro")' "$PROXY_COMPOSE_FILE" + run yq '.services | has("mitmproxy")' "$compose_file" + assert_output "false" # shellcheck disable=SC2016 yq -e '.services.agent.volumes[] | select(. == "${HOME}/.claude/CLAUDE.md:/home/vscode/.claude/CLAUDE.md:ro")' "$compose_file" @@ -352,6 +384,42 @@ EOF EOF } +@test "add_volume_entry with comment uses one in-place yq" { + stub_yq_inplace_counter + add_volume_entry "$COMPOSE_FILE" "../test:/workspace/test:ro" "true" "Test volume" + assert_equal "$YQ_INPLACE_COUNT" 1 +} + +@test "add_cursor_config_volumes uses one in-place yq for active mounts" { + stub_yq_inplace_counter + add_cursor_config_volumes "$COMPOSE_FILE" true "test-project" + assert_equal "$YQ_INPLACE_COUNT" 1 +} + +@test "add_cursor_config_volumes uses one in-place yq for inactive mounts" { + stub_yq_inplace_counter + add_cursor_config_volumes "$COMPOSE_FILE" false "test-project" + assert_equal "$YQ_INPLACE_COUNT" 1 +} + +@test "add_claude_config_volumes uses one in-place yq" { + stub_yq_inplace_counter + add_claude_config_volumes "$COMPOSE_FILE" + assert_equal "$YQ_INPLACE_COUNT" 1 +} + +@test "set_workspace uses one in-place yq" { + stub_yq_inplace_counter + set_workspace "$COMPOSE_FILE" "my-project" + assert_equal "$YQ_INPLACE_COUNT" 1 +} + +@test "add_jetbrains_capabilities uses one in-place yq" { + stub_yq_inplace_counter + add_jetbrains_capabilities "$COMPOSE_FILE" + assert_equal "$YQ_INPLACE_COUNT" 1 +} + @test "set_workspace adds working_dir and workspace volumes" { set_workspace "$COMPOSE_FILE" "my-project" @@ -360,7 +428,11 @@ EOF yq -e '.services.agent.volumes[] | select(. == "..:/workspaces/my-project")' "$COMPOSE_FILE" yq -e '.services.agent.volumes[] | select(. == "../.devcontainer:/workspaces/my-project/.devcontainer:ro")' "$COMPOSE_FILE" - yq -e '.services.agent.volumes[] | select(. == "../.sandcat:/workspaces/my-project/.sandcat:ro")' "$COMPOSE_FILE" + # Unset SANDCAT_AGENT_SANDCAT must not fall back to the live project .sandcat. + # shellcheck disable=SC2016 + yq -e '.services.agent.volumes[] | select(. == "${SANDCAT_AGENT_SANDCAT:?}:/workspaces/my-project/.sandcat:ro")' "$COMPOSE_FILE" + run yq -r '.services.agent.volumes[] | select(test(".sandcat:ro"))' "$COMPOSE_FILE" + refute_output --partial ':-' } # shellcheck disable=SC2016 @@ -392,7 +464,7 @@ EOF customize_compose_file "$SETTINGS_FILE" "$COMPOSE_FILE" "claude" "jetbrains" "test-project" # Verify settings volume on proxy - yq -e '.services.mitmproxy.volumes[] | select(. == ".sandcat:/config/project:ro")' "$COMPOSE_FILE" + yq -e '.services.mitmproxy.volumes[] | select(. == "../.sandcat:/config/project:ro")' "$PROXY_COMPOSE_FILE" # Verify .idea volume is active yq -e '.services.agent.volumes[] | select(. == "../.idea:/workspace/.idea:ro")' "$COMPOSE_FILE" @@ -599,6 +671,63 @@ YAML yq -e '.services.agent.volumes[] | select(. == "${HOME}/.copilot/mcp-config.json:/home/vscode/.copilot/mcp-config.json:ro")' "$COMPOSE_FILE" } +@test "apply_secret_provider appends token without wiping existing env" { + local proxy_compose="$BATS_TEST_TMPDIR/compose-proxy-preserve.yml" + cat >"$proxy_compose" <<'YAML' +services: + mitmproxy: + image: mitmproxy/mitmproxy:latest + environment: + - NB_SETUP_KEY + - NB_MANAGEMENT_URL=http://192.168.5.2:33073 +YAML + + apply_secret_provider "$proxy_compose" "protonpass" + + yq -e '.services.mitmproxy.environment[] | select(. == "NB_SETUP_KEY")' "$proxy_compose" + yq -e '.services.mitmproxy.environment[] | select(. == "NB_MANAGEMENT_URL=http://192.168.5.2:33073")' "$proxy_compose" + yq -e '.services.mitmproxy.environment[] | select(. == "PROTON_PASS_PERSONAL_ACCESS_TOKEN")' "$proxy_compose" +} + +@test "apply_secret_provider does not clobber Dockerfile.mitmproxy build" { + local proxy_compose="$BATS_TEST_TMPDIR/compose-proxy-netbird.yml" + cat >"$proxy_compose" <<'YAML' +services: + mitmproxy: + build: + context: . + dockerfile: Dockerfile.mitmproxy + environment: + - NB_SETUP_KEY +YAML + + apply_secret_provider "$proxy_compose" "protonpass" + + yq -e '.services.mitmproxy.build.dockerfile == "Dockerfile.mitmproxy"' "$proxy_compose" + run yq '.services.mitmproxy | has("image")' "$proxy_compose" + assert_output "false" + yq -e '.services.mitmproxy.environment[] | select(. == "NB_SETUP_KEY")' "$proxy_compose" + yq -e '.services.mitmproxy.environment[] | select(. == "PROTON_PASS_PERSONAL_ACCESS_TOKEN")' "$proxy_compose" +} + +@test "apply_secret_provider passes provider image as BASE_IMAGE to Dockerfile.mitmproxy" { + local proxy_compose="$BATS_TEST_TMPDIR/compose-proxy-base-image.yml" + cat >"$proxy_compose" <<'YAML' +services: + mitmproxy: + build: + context: . + dockerfile: Dockerfile.mitmproxy +YAML + + apply_secret_provider "$proxy_compose" "protonpass" + + # image: would be ignored next to build:, leaving the built image without + # pass-cli and every pass:// reference unresolvable. + run yq -r '.services.mitmproxy.build.args.BASE_IMAGE' "$proxy_compose" + assert_output "ghcr.io/virtuslab/sandcat-mitmproxy-pass:${SCT_MITMPROXY_VERSION}" +} + @test "apply_secret_provider leaves default image for none" { local proxy_compose="$BATS_TEST_TMPDIR/compose-proxy-none.yml" cat >"$proxy_compose" <<'YAML' diff --git a/cli/test/composefile/netbird.bats b/cli/test/composefile/netbird.bats new file mode 100644 index 00000000..ae47cee1 --- /dev/null +++ b/cli/test/composefile/netbird.bats @@ -0,0 +1,348 @@ +#!/usr/bin/env bats + +setup() { + load test_helper + source "$SCT_LIBDIR/composefile.bash" + # shellcheck source=netbird.bash + source "$SCT_LIBDIR/netbird.bash" + + export HOME="$BATS_TEST_TMPDIR/home" + mkdir -p "$HOME/.config/sandcat" + + COMPOSE_FILE="$BATS_TEST_TMPDIR/compose-proxy.yml" + cp "$SCT_TEMPLATEDIR/devcontainer/sandcat/netbird.env" "$BATS_TEST_TMPDIR/netbird.env" + # Minimal template that mirrors the real compose-proxy.yml: mitmproxy uses + # image: initially; enable_netbird() switches it to build:. + cat >"$COMPOSE_FILE" <<'YAML' +services: + wg-client: + build: + context: . + dockerfile: Dockerfile.wg-client + cap_add: + - NET_ADMIN + # Real compose-proxy.yml also sets this on wg-client; enable_netbird must + # still add it to mitmproxy (file-wide grep would false-positive here). + sysctls: + - net.ipv4.conf.all.src_valid_mark=1 + mitmproxy: + image: mitmproxy/mitmproxy:latest + entrypoint: ["sh", "-c", "rm -f dns.conf && exec docker-entrypoint.sh \"$@\"", "sh"] + command: mitmweb --mode wireguard +YAML +} + +teardown() { + unstub_all +} + +@test "enable_netbird adds NB_SETUP_KEY to mitmproxy environment" { + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + yq -e '.services.mitmproxy.environment[] | select(. == "NB_SETUP_KEY")' "$COMPOSE_FILE" +} + +@test "enable_netbird keeps secret-provider token when adding NB_SETUP_KEY" { + apply_secret_provider "$COMPOSE_FILE" "protonpass" + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + yq -e '.services.mitmproxy.environment[] | select(. == "NB_SETUP_KEY")' "$COMPOSE_FILE" + yq -e '.services.mitmproxy.environment[] | select(. == "PROTON_PASS_PERSONAL_ACCESS_TOKEN")' "$COMPOSE_FILE" + run yq -r '.services.mitmproxy.build.dockerfile' "$COMPOSE_FILE" + assert_output "Dockerfile.mitmproxy" +} + +@test "enable_netbird keeps the secret-provider image as the build base" { + # init order: apply_secret_provider pins image:, then enable_netbird + # converts to build:. Dropping the image here would silently remove + # pass-cli from the NetBird-enabled proxy. + apply_secret_provider "$COMPOSE_FILE" "protonpass" + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + run yq -r '.services.mitmproxy.build.args.BASE_IMAGE' "$COMPOSE_FILE" + assert_output "ghcr.io/virtuslab/sandcat-mitmproxy-pass:${SCT_MITMPROXY_VERSION}" +} + +@test "enable_netbird keeps NetBird build args alongside BASE_IMAGE" { + apply_secret_provider "$COMPOSE_FILE" "1password" + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + run yq -r '.services.mitmproxy.build.args.BASE_IMAGE' "$COMPOSE_FILE" + assert_output "ghcr.io/virtuslab/sandcat-mitmproxy-op:${SCT_MITMPROXY_VERSION}" + yq -e '.services.mitmproxy.build.args | has("NETBIRD_VERSION")' "$COMPOSE_FILE" +} + +@test "enable_netbird carries the stock image as BASE_IMAGE without a provider" { + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + run yq -r '.services.mitmproxy.build.args.BASE_IMAGE' "$COMPOSE_FILE" + assert_output "mitmproxy/mitmproxy:latest" +} + +@test "enable_netbird removes stale NB_SETUP_KEY from wg-client" { + yq -i ' + .services."wg-client".environment = [ + "NB_SETUP_KEY", + "NB_MANAGEMENT_URL=http://192.168.5.2:33073", + "NB_USE_LEGACY_ROUTING=true" + ] + ' "$COMPOSE_FILE" + + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + run yq '[.services."wg-client".environment[]? | select(. == "NB_SETUP_KEY" or test("^NB_MANAGEMENT_URL=") or test("^NB_USE_LEGACY_ROUTING="))] | length' "$COMPOSE_FILE" + assert_output "0" + yq -e '.services.mitmproxy.environment[] | select(. == "NB_SETUP_KEY")' "$COMPOSE_FILE" +} + +@test "enable_netbird is idempotent" { + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + run yq '[.services.mitmproxy.environment[] | select(. == "NB_SETUP_KEY")] | length' "$COMPOSE_FILE" + assert_output "1" + run yq '[.services.mitmproxy.extra_hosts[] | select(. == "host.docker.internal:172.17.0.1")] | length' "$COMPOSE_FILE" + assert_output "1" +} + +@test "enable_netbird replaces host-gateway extra_hosts with docker0" { + yq -i '.services.mitmproxy.extra_hosts = ["host.docker.internal:host-gateway"]' "$COMPOSE_FILE" + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + run yq '[.services.mitmproxy.extra_hosts[] | select(. == "host.docker.internal:host-gateway")] | length' "$COMPOSE_FILE" + assert_output "0" + run yq '[.services.mitmproxy.extra_hosts[] | select(. == "host.docker.internal:172.17.0.1")] | length' "$COMPOSE_FILE" + assert_output "1" +} + +@test "enable_netbird switches mitmproxy from image to build" { + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + run yq -r '.services.mitmproxy.build.dockerfile' "$COMPOSE_FILE" + assert_output "Dockerfile.mitmproxy" + + run yq '(.services.mitmproxy.image // "null")' "$COMPOSE_FILE" + assert_output "null" +} + +@test "enable_netbird removes mitmproxy entrypoint override" { + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + run yq '(.services.mitmproxy.entrypoint // "null")' "$COMPOSE_FILE" + assert_output "null" +} + +@test "enable_netbird adds NET_ADMIN to mitmproxy" { + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + yq -e '.services.mitmproxy.cap_add[] | select(. == "NET_ADMIN")' "$COMPOSE_FILE" +} + +@test "enable_netbird adds src_valid_mark sysctl to mitmproxy" { + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + yq -e '.services.mitmproxy.sysctls[] | select(. == "net.ipv4.conf.all.src_valid_mark=1")' "$COMPOSE_FILE" +} + +@test "enable_netbird adds src_valid_mark even when wg-client already has it" { + # setup() already puts src_valid_mark on wg-client; this asserts the + # regression that a file-wide grep would skip mitmproxy. + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + run yq '[.services.mitmproxy.sysctls[] | select(test("src_valid_mark"))] | length' "$COMPOSE_FILE" + assert_output "1" +} + +@test "enable_netbird does not modify wg-client" { + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + run yq '(.services."wg-client".environment // "null")' "$COMPOSE_FILE" + assert_output "null" +} + +@test "enable_netbird adds NB_MANAGEMENT_URL when provided" { + local management_url="https://netbird.internal" + enable_netbird "$COMPOSE_FILE" "$management_url" "test-proxy" + + run yq -r '.services.mitmproxy.environment[] | select(test("^NB_MANAGEMENT_URL="))' "$COMPOSE_FILE" + assert_output "NB_MANAGEMENT_URL=$management_url" +} + +@test "enable_netbird with management URL is idempotent" { + local management_url="https://netbird.internal" + enable_netbird "$COMPOSE_FILE" "$management_url" "test-proxy" + enable_netbird "$COMPOSE_FILE" "$management_url" "test-proxy" + + run yq '[.services.mitmproxy.environment[] | select(. == "NB_SETUP_KEY")] | length' "$COMPOSE_FILE" + assert_output "1" + + run yq '[.services.mitmproxy.environment[] | select(test("^NB_MANAGEMENT_URL="))] | length' "$COMPOSE_FILE" + assert_output "1" +} + +@test "enable_netbird updates existing NB_MANAGEMENT_URL when provided" { + enable_netbird "$COMPOSE_FILE" "https://old.example.com" "test-proxy" + enable_netbird "$COMPOSE_FILE" "https://new.example.com" "test-proxy" + + run yq -r '.services.mitmproxy.environment[] | select(test("^NB_MANAGEMENT_URL="))' "$COMPOSE_FILE" + assert_output "NB_MANAGEMENT_URL=https://new.example.com" +} + +@test "enable_netbird with empty management URL does not add NB_MANAGEMENT_URL" { + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + run yq '[.services.mitmproxy.environment[] | select(test("^NB_MANAGEMENT_URL="))] | length' "$COMPOSE_FILE" + assert_output "0" +} + +@test "enable_netbird warns when localhost has no container-reachable enrollment URL" { + run enable_netbird "$COMPOSE_FILE" "http://localhost:33073" "test-proxy" + + assert_success + assert_output --partial "no NB_MANAGEMENT_URL" + assert_output --partial "netbird_enrollment_management_url" +} + +@test "enable_netbird omits NB_MANAGEMENT_URL for localhost without enrollment URL" { + enable_netbird "$COMPOSE_FILE" "http://localhost:33073" "test-proxy" + + run yq '[.services.mitmproxy.environment[] | select(test("^NB_MANAGEMENT_URL="))] | length' "$COMPOSE_FILE" + assert_output "0" +} + +@test "enable_netbird uses explicit enrollment URL for local self-hosted" { + export HOME="$BATS_TEST_TMPDIR/home" + mkdir -p "$HOME/.config/sandcat" + echo '{"netbird_enrollment_management_url": "http://192.168.5.2:33073"}' > "$HOME/.config/sandcat/settings.json" + + enable_netbird "$COMPOSE_FILE" "http://localhost:33073" "test-proxy" + + run yq -r '.services.mitmproxy.environment[] | select(test("^NB_MANAGEMENT_URL="))' "$COMPOSE_FILE" + assert_output "NB_MANAGEMENT_URL=http://192.168.5.2:33073" +} + +@test "enable_netbird sets NB_USE_LEGACY_ROUTING for host IP enrollment URL" { + export HOME="$BATS_TEST_TMPDIR/home-enrollment" + mkdir -p "$HOME/.config/sandcat" + echo '{"netbird_enrollment_management_url": "http://192.168.5.2:33073"}' > "$HOME/.config/sandcat/settings.json" + + enable_netbird "$COMPOSE_FILE" "http://localhost:33073" "test-proxy" + + run yq -r '.services.mitmproxy.environment[] | select(. == "NB_USE_LEGACY_ROUTING=true")' "$COMPOSE_FILE" + assert_output "NB_USE_LEGACY_ROUTING=true" +} + +@test "enable_netbird does not rewrite remote management URL" { + enable_netbird "$COMPOSE_FILE" "https://netbird.example.com" "test-proxy" + + run yq -r '.services.mitmproxy.environment[] | select(test("^NB_MANAGEMENT_URL="))' "$COMPOSE_FILE" + assert_output "NB_MANAGEMENT_URL=https://netbird.example.com" + + run yq '[.services.mitmproxy.environment[] | select(. == "NB_USE_LEGACY_ROUTING=true")] | length' "$COMPOSE_FILE" + assert_output "0" +} + +@test "enable_netbird injects NetBird build args into mitmproxy" { + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + # shellcheck disable=SC1091 + source "$BATS_TEST_TMPDIR/netbird.env" + run yq -r '.services.mitmproxy.build.args.NETBIRD_VERSION' "$COMPOSE_FILE" + assert_output "$NETBIRD_VERSION" + run yq -r '.services.mitmproxy.build.args.NETBIRD_SHA256_AMD64' "$COMPOSE_FILE" + assert_output "$NETBIRD_SHA256_AMD64" +} + +@test "enable_netbird adds NETBIRD_DNS_DOMAIN to mitmproxy environment" { + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + yq -e '.services.mitmproxy.environment[] | select(test("^NETBIRD_DNS_DOMAIN="))' "$COMPOSE_FILE" +} + +@test "enable_netbird NETBIRD_DNS_DOMAIN defaults to netbird.selfhosted" { + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + run yq -r '.services.mitmproxy.environment[] | select(test("^NETBIRD_DNS_DOMAIN="))' "$COMPOSE_FILE" + assert_output "NETBIRD_DNS_DOMAIN=netbird.selfhosted" +} + +@test "enable_netbird NETBIRD_DNS_DOMAIN is idempotent" { + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + enable_netbird "$COMPOSE_FILE" "" "test-proxy" + + run yq '[.services.mitmproxy.environment[] | select(test("^NETBIRD_DNS_DOMAIN="))] | length' "$COMPOSE_FILE" + assert_output "1" +} + +@test "enable_netbird sets NB_PEER_NAME on mitmproxy from argument" { + enable_netbird "$COMPOSE_FILE" "" "myapp-sandbox-proxy" + + run yq -r '.services.mitmproxy.environment[] | select(test("^NB_PEER_NAME="))' "$COMPOSE_FILE" + assert_output "NB_PEER_NAME=myapp-sandbox-proxy" +} + +@test "enable_netbird adds NB_API_TOKEN passthrough to mitmproxy" { + enable_netbird "$COMPOSE_FILE" "" "myapp-sandbox-proxy" + + yq -e '.services.mitmproxy.environment[] | select(. == "NB_API_TOKEN")' "$COMPOSE_FILE" +} + +@test "enable_netbird mounts named volume on /var/lib/netbird" { + enable_netbird "$COMPOSE_FILE" "" "myapp-sandbox-proxy" + + yq -e '.services.mitmproxy.volumes[] | select(. == "netbird-mitmproxy-state:/var/lib/netbird")' "$COMPOSE_FILE" + yq -e '.volumes | has("netbird-mitmproxy-state")' "$COMPOSE_FILE" +} + +@test "enable_netbird copies the peer lifecycle script into the build context" { + mkdir -p "$BATS_TEST_TMPDIR/scripts" + + enable_netbird "$COMPOSE_FILE" "" "myapp-sandbox-proxy" + + run cmp \ + "$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/netbird-peer-lifecycle.sh" \ + "$BATS_TEST_TMPDIR/scripts/netbird-peer-lifecycle.sh" + assert_success +} + +@test "enable_netbird fails when peer name argument is empty" { + run enable_netbird "$COMPOSE_FILE" "" "" + assert_failure + assert_output --partial "peer name" +} + +@test "enable_netbird keeps upstream CA volume mounts" { + local ca="$BATS_TEST_TMPDIR/company-ca.pem" + printf -- '-----BEGIN CERTIFICATE-----\nABC\n-----END CERTIFICATE-----\n' > "$ca" + mkdir -p "$HOME/.config/sandcat" + cat > "$HOME/.config/sandcat/settings.json" </dev/null \ + || stat -f '%OLp' "$WORKDIR/config.local.yaml") + assert_equal "$mode" "600" +} + +@test "start.sh --prepare-only reuses existing env secrets" { + sed -i.bak \ + -e 's/^NETBIRD_RELAY_AUTH_SECRET=.*/NETBIRD_RELAY_AUTH_SECRET=relay-test-secret/' \ + -e 's/^NETBIRD_ENCRYPTION_KEY=.*/NETBIRD_ENCRYPTION_KEY=encrypt-test-secret/' \ + "$WORKDIR/netbird-server.env" + + run bash "$WORKDIR/start.sh" --prepare-only + assert_success + assert_output --partial "reusing secrets" + + run grep -F 'NETBIRD_RELAY_AUTH_SECRET=relay-test-secret' "$WORKDIR/netbird-server.env" + assert_success + run grep -F 'authSecret: "relay-test-secret"' "$WORKDIR/config.local.yaml" + assert_success + run grep -F 'encryptionKey: "encrypt-test-secret"' "$WORKDIR/config.local.yaml" + assert_success +} + +@test "start.sh --secrets-from injects YAML keys without writing env" { + cat >"$WORKDIR/existing.yaml" <<'YAML' +server: + authSecret: "from-existing-relay" + store: + encryptionKey: "from-existing-store" +YAML + + run bash "$WORKDIR/start.sh" --prepare-only --secrets-from "$WORKDIR/existing.yaml" + assert_success + assert_output --partial "not written to netbird-server.env" + + run grep -F 'authSecret: "from-existing-relay"' "$WORKDIR/config.local.yaml" + assert_success + run grep -F 'encryptionKey: "from-existing-store"' "$WORKDIR/config.local.yaml" + assert_success + + run grep -F 'from-existing-relay' "$WORKDIR/netbird-server.env" + assert_failure + run grep -E '^NETBIRD_RELAY_AUTH_SECRET=$' "$WORKDIR/netbird-server.env" + assert_success +} + +@test "compose mounts config.local.yaml not tracked config.yaml placeholders" { + run grep -F './config.local.yaml:/etc/netbird/config.yaml' "$EXAMPLE/docker-compose.yml" + assert_success + run grep -F './config.yaml:/etc/netbird/config.yaml' "$EXAMPLE/docker-compose.yml" + assert_failure +} + +@test "example gitignores config.local.yaml" { + run grep -F 'docs/examples/netbird-server/config.local.yaml' "$SCT_ROOT/../.gitignore" + assert_success +} + +@test "start.sh expands empty COMPOSE_ARGS without set -u failure" { + run grep -F 'COMPOSE_ARGS[@]+"${COMPOSE_ARGS[@]}"' "$EXAMPLE/start.sh" + assert_success +} diff --git a/cli/test/examples/proxy_peer_compose.bats b/cli/test/examples/proxy_peer_compose.bats new file mode 100644 index 00000000..7be2767c --- /dev/null +++ b/cli/test/examples/proxy_peer_compose.bats @@ -0,0 +1,58 @@ +#!/usr/bin/env bats + +setup() { + load test_helper + EXAMPLE="$SCT_ROOT/../docs/examples/proxy-peer" +} + +@test "compose-proxy-peer interpolates NB_SETUP_KEY" { + run grep -F 'NB_SETUP_KEY=${NB_SETUP_KEY}' "$EXAMPLE/compose-proxy-peer.yml" + assert_success +} + +@test "compose-proxy-peer has no hardcoded nbp_ token" { + run grep -E 'nbp_|[0-9A-F]{8}-[0-9A-F]{4}' "$EXAMPLE/compose-proxy-peer.yml" + assert_failure +} + +@test "compose-proxy-peer does not mount sandcat user settings" { + run grep -F 'settings.json' "$EXAMPLE/compose-proxy-peer.yml" + assert_failure +} + +@test "example ships .env.example and gitignores .env" { + [[ -f "$EXAMPLE/.env.example" ]] + run grep -F 'docs/examples/proxy-peer/.env' "$SCT_ROOT/../.gitignore" + assert_success +} + +@test ".env.example leaves NB_SETUP_KEY NB_MANAGEMENT_URL and NB_API_TOKEN empty" { + run grep -E '^NB_SETUP_KEY=.' "$EXAMPLE/.env.example" + assert_failure + run grep -E '^NB_MANAGEMENT_URL=.' "$EXAMPLE/.env.example" + assert_failure + run grep -E '^NB_API_TOKEN=.' "$EXAMPLE/.env.example" + assert_failure + run grep -F 'NB_MANAGEMENT_URL=' "$EXAMPLE/.env.example" + assert_success +} + +@test "example has no capability-catalog.json" { + [[ ! -f "$EXAMPLE/capability-catalog.json" ]] +} + +@test "example does not duplicate netbird-peer-lifecycle.sh" { + [[ ! -f "$EXAMPLE/scripts/netbird-peer-lifecycle.sh" ]] +} + +@test "compose additional_contexts points at template scripts" { + run grep -F 'sandcat-scripts: ../../../cli/templates/devcontainer/sandcat/scripts' \ + "$EXAMPLE/compose-proxy-peer.yml" + assert_success +} + +@test "example Dockerfile copies lifecycle from sandcat templates context" { + run grep -F 'COPY --from=sandcat-scripts netbird-peer-lifecycle.sh /usr/local/lib/netbird-peer-lifecycle.sh' \ + "$EXAMPLE/Dockerfile.proxy-peer" + assert_success +} diff --git a/cli/test/examples/test_helper.bash b/cli/test/examples/test_helper.bash new file mode 100644 index 00000000..69cd0113 --- /dev/null +++ b/cli/test/examples/test_helper.bash @@ -0,0 +1,19 @@ +#!/bin/bash +bats_require_minimum_version 1.5.0 +if shopt -s compat32 2>/dev/null; then + export BASH_COMPAT=3.2 +fi +set -uo pipefail +export SHELLOPTS + +SCT_ROOT="$BATS_TEST_DIRNAME/../.." +BATS_LIB_PATH="$SCT_ROOT/support":${BATS_LIB_PATH-} + +bats_load_library bats-ext +bats_load_library bats-support +bats_load_library bats-assert +bats_load_library bats-mock-ext + +export SCT_ROOT SCT_LIBDIR="$SCT_ROOT/lib" +export SCT_LIBEXECDIR="$SCT_ROOT/libexec" +export SCT_TEMPLATEDIR="$SCT_ROOT/templates" diff --git a/cli/test/init/devcontainer.bats b/cli/test/init/devcontainer.bats index 7b6da7a8..142bccbf 100644 --- a/cli/test/init/devcontainer.bats +++ b/cli/test/init/devcontainer.bats @@ -55,6 +55,14 @@ teardown() { assert_success } +@test "devcontainer.json prepares a filtered .sandcat copy before compose" { + run grep -F '"initializeCommand":' "$DEVCONTAINER_JSON" + assert_success + run grep -F 'prepare-agent-sandcat-mount.sh' "$DEVCONTAINER_JSON" + assert_success + [[ -f "$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/prepare-agent-sandcat-mount.sh" ]] +} + @test "customize_devcontainer_json keeps customizations.vscode when ide is vscode" { customize_devcontainer_json "$DEVCONTAINER_JSON" "my-project" "vscode" diff --git a/cli/test/init/extensions.bats b/cli/test/init/extensions.bats index 014c3a2e..3aa17f48 100644 --- a/cli/test/init/extensions.bats +++ b/cli/test/init/extensions.bats @@ -341,8 +341,8 @@ teardown() { } @test "compose-agent.yml mounts agent from mitmproxy-public (not mitmproxy-config)" { - # The agent's constant volumes live in sandcat/compose-agent.yml since the - # #22 split; compose-all.yml only carries user-editable overrides. + # The agent's volumes live in sandcat/compose-agent.yml; compose-all.yml + # must not redeclare services.agent (Compose include rejects that). yq -e '.services.agent.volumes[] | select(. == "mitmproxy-public:/mitmproxy-config:ro")' \ "$SCT_TEMPLATEDIR/devcontainer/sandcat/compose-agent.yml" @@ -350,6 +350,11 @@ teardown() { run yq -e '.services.agent.volumes[] | select(. == "mitmproxy-config:/mitmproxy-config:ro")' \ "$SCT_TEMPLATEDIR/devcontainer/sandcat/compose-agent.yml" [ "$status" -ne 0 ] + + # compose-all.yml must stay free of services.agent. + run yq -e '.services | has("agent")' \ + "$SCT_TEMPLATEDIR/devcontainer/compose-all.yml" + assert_failure } # --------------------------------------------------- upstream CA bundles diff --git a/cli/test/init/include_contract.bats b/cli/test/init/include_contract.bats new file mode 100644 index 00000000..50ea92e5 --- /dev/null +++ b/cli/test/init/include_contract.bats @@ -0,0 +1,126 @@ +#!/usr/bin/env bats +# shellcheck disable=SC2030,SC2031 + +# Compose's `include` copies resources into the model, it never merges them: +# a service declared in both compose-all.yml and an included file aborts the +# whole project with "conflicts with imported resource". These tests pin that +# invariant on the generated tree so it fails here rather than at `sandcat up`. + +setup() { + load test_helper + # shellcheck source=../../libexec/init/init + source "$SCT_LIBEXECDIR/init/init" + # shellcheck source=../../libexec/init/devcontainer + source "$SCT_LIBEXECDIR/init/devcontainer" + + PROJECT_DIR="$BATS_TEST_TMPDIR/project" + mkdir -p "$PROJECT_DIR/.sandcat" + touch "$PROJECT_DIR/.sandcat/settings.json" + + SCT_HOME_DIR="$BATS_TEST_TMPDIR/config/sandcat" + mkdir -p "$SCT_HOME_DIR" + sct_home() { echo "$SCT_HOME_DIR"; } + export -f sct_home + + export HOME="$BATS_TEST_TMPDIR/home" + mkdir -p "$HOME" +} + +teardown() { + unstub_all +} + +assert_no_include_conflicts() { + local devcontainer_dir=$1 + + local -a included=() + local include_path service + while read -r include_path + do + [[ -n "$include_path" ]] || continue + while read -r service + do + included+=("$service") + done < <(yq -r '.services // {} | keys | .[]' "$devcontainer_dir/$include_path") + done < <(yq -r '.include[]?.path' "$devcontainer_dir/compose-all.yml") + + local imported + while read -r service + do + [[ -n "$service" ]] || continue + for imported in "${included[@]}" + do + if [[ "$service" == "$imported" ]] + then + fail "service '$service' is declared in compose-all.yml and in an included file" + fi + done + done < <(yq -r '.services // {} | keys | .[]' "$devcontainer_dir/compose-all.yml") +} + +@test "init keeps compose-all.yml services disjoint from its includes" { + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" \ + --stacks "" --proxy web --features "" --secret-provider none + assert_success + + assert_no_include_conflicts "$PROJECT_DIR/.devcontainer" +} + +@test "init keeps services disjoint with netbird enabled" { + stub settings "$PROJECT_DIR/.sandcat/settings.json cursor : :" + + run init --agent cursor --ide none --name cv-sandbox --path "$PROJECT_DIR" \ + --stacks python --proxy web --features tui --secret-provider protonpass \ + --netbird + assert_success + + assert_no_include_conflicts "$PROJECT_DIR/.devcontainer" +} + +@test "init --netbird does not write proxy-peer leftovers" { + stub settings "$PROJECT_DIR/.sandcat/settings.json cursor : :" + + run init --agent cursor --ide none --name cv-sandbox --path "$PROJECT_DIR" \ + --stacks python --proxy web --features tui --secret-provider protonpass \ + --netbird + assert_success + + run yq '.netbird_peer_name_proxy_peer' "$PROJECT_DIR/.sandcat/settings.json" + assert_output "null" + [[ ! -f "$PROJECT_DIR/.devcontainer/sandcat/compose-proxy-peer.yml" ]] + [[ ! -f "$PROJECT_DIR/.sandcat/settings.proxy-peer.example.json" ]] + run yq '[.include[]? | select(.path == "sandcat/compose-proxy-peer.yml")] | length' \ + "$PROJECT_DIR/.devcontainer/compose-all.yml" + assert_output "0" +} + +@test "init --netbird derives NB_PEER_NAME from the project and ignores settings" { + printf '%s\n' '{"netbird_peer_name_proxy":"victim-peer"}' >"$PROJECT_DIR/.sandcat/settings.json" + stub settings "$PROJECT_DIR/.sandcat/settings.json cursor : :" + + run init --agent cursor --ide none --name myapp-sandbox --path "$PROJECT_DIR" \ + --stacks "" --proxy web --features "" --secret-provider none \ + --netbird + assert_success + + run yq -r '.services.mitmproxy.environment[] | select(test("^NB_PEER_NAME="))' \ + "$PROJECT_DIR/.devcontainer/sandcat/compose-proxy.yml" + assert_output "NB_PEER_NAME=myapp-sandbox-proxy" + run yq -r '.netbird_peer_name_proxy' "$PROJECT_DIR/.sandcat/settings.json" + assert_output "myapp-sandbox-proxy" +} + +@test "init mounts project settings on the imported mitmproxy service" { + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" \ + --stacks "" --proxy web --features "" --secret-provider none + assert_success + + # Two levels up, because relative paths in an included file resolve against + # that file's own directory (.devcontainer/sandcat), not the project root. + yq -e '.services.mitmproxy.volumes[] | select(. == "../../.sandcat:/config/project:ro")' \ + "$PROJECT_DIR/.devcontainer/sandcat/compose-proxy.yml" +} diff --git a/cli/test/init/init.bats b/cli/test/init/init.bats index 315a8e39..450589f5 100644 --- a/cli/test/init/init.bats +++ b/cli/test/init/init.bats @@ -332,6 +332,138 @@ EOF [[ ! -e "$HOME/.cursor/mcp.json" ]] } +@test "init --netbird passes netbird flag to devcontainer" { + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird : :" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird + assert_success +} + +@test "init --netbird seeds netbird_enrollment_key in user settings" { + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer ":" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird + assert_success + run yq '.netbird_enrollment_key' "$SCT_HOME_DIR/settings.json" + assert_output '""' + run yq '.netbird_api_token' "$SCT_HOME_DIR/settings.json" + assert_output '""' + run yq '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output '""' +} + +@test "init rejects --netbird-management-url without --netbird" { + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird-management-url https://netbird.example.com + assert_failure + assert_output --partial "--netbird-management-url requires --netbird" +} + +@test "init treats --netbird-server as an unknown option" { + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird --netbird-server cloud + assert_failure + assert_output --partial "Unknown option: --netbird-server" +} + +@test "init --netbird-management-url persists management server immediately" { + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird-management-url https://management.example.com --netbird : :" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird --netbird-management-url https://management.example.com + assert_success + run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "https://management.example.com" +} + +@test "init forwards --netbird-management-url to devcontainer args" { + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird-management-url https://selected.example.com --netbird : :" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird --netbird-management-url https://selected.example.com + assert_success +} + +@test "init --netbird without management URL uses cloud summary" { + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird : :" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --secret-provider none --netbird + assert_success + assert_output --partial "Management server: cloud (https://api.netbird.io)" + assert_output --partial "docs/examples/netbird-server/" + run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "" +} + +@test "init interactive netbird existing re-prompts for non-empty URL" { + unset -f read_line + unset -f select_option + + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird-management-url https://management.example.com --netbird : :" + stub select_option \ + "'Select secret provider:' none 1password protonpass : echo none" + stub read_line \ + "'>' : echo '2'" \ + "'Management URL:' : echo ''" \ + "'Management URL:' : echo 'https://management.example.com'" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --netbird + assert_success + assert_output --partial "URL is required" + run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "https://management.example.com" +} + +@test "init interactive netbird existing accepts non-empty URL without format restriction" { + unset -f read_line + unset -f select_option + + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird-management-url management.example.com --netbird : :" + stub select_option \ + "'Select secret provider:' none 1password protonpass : echo none" + stub read_line \ + "'>' : echo existing" \ + "'Management URL:' : echo 'management.example.com'" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --netbird + assert_success + run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "management.example.com" +} + +@test "init interactive netbird existing localhost persists enrollment URL" { + unset -f read_line + unset -f select_option + unset -f netbird_detect_docker_host_ip + + stub settings "$PROJECT_DIR/.sandcat/settings.json claude vscode : :" + stub devcontainer \ + "--settings-file .sandcat/settings.json --project-path * --agent claude --ide vscode --name test --stacks * --proxy web --secret-provider none --netbird-management-url http://localhost:33073 --netbird : :" + stub select_option \ + "'Select secret provider:' none 1password protonpass : echo none" + stub netbird_detect_docker_host_ip "echo 192.168.1.50" + stub read_line \ + "'>' : echo 2" \ + "'Management URL:' : echo 'http://localhost:33073'" \ + "'Enrollment URL [http://192.168.1.50:33073]:' : echo ''" + + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" --stacks "" --proxy web --features "" --netbird + assert_success + run yq -r '.netbird_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "http://localhost:33073" + run yq -r '.netbird_enrollment_management_url' "$SCT_HOME_DIR/settings.json" + assert_output "http://192.168.1.50:33073" +} + @test "init interactive flow (devcontainer mode)" { unset -f read_line unset -f select_option @@ -411,6 +543,21 @@ EOF assert_output --partial "no-rtk" } +@test "init rejects --capability as unknown" { + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" \ + --stacks "" --proxy web --features "" --secret-provider none --capability + assert_failure + assert_output --partial "Unknown option: --capability" +} + +@test "init rejects --capability even with --netbird" { + run init --agent claude --ide vscode --name test --path "$PROJECT_DIR" \ + --stacks "" --proxy web --features "" --secret-provider none \ + --netbird --capability + assert_failure + assert_output --partial "Unknown option: --capability" +} + @test "init interactive feature selection applies tui from full labels" { unset -f read_line unset -f select_option diff --git a/cli/test/init/regression.bats b/cli/test/init/regression.bats index f57f0168..e1cd5bf0 100644 --- a/cli/test/init/regression.bats +++ b/cli/test/init/regression.bats @@ -11,6 +11,9 @@ setup() { PROJECT_DIR="$BATS_TEST_TMPDIR/project" mkdir -p "$PROJECT_DIR/$SCT_PROJECT_DIR" + export HOME="$BATS_TEST_TMPDIR/home" + mkdir -p "$HOME" + SETTINGS_FILE="$SCT_PROJECT_DIR/settings.json" touch "$PROJECT_DIR/$SETTINGS_FILE" } @@ -24,6 +27,19 @@ assert_proxy_service() { yq -e '.services.mitmproxy.image == "mitmproxy/mitmproxy:'"$SCT_MITMPROXY_VERSION"'"' "$compose_file" + # Project settings are mounted from the included sandcat/compose-proxy.yml, + # where the relative path resolves against that file's own directory. This + # asserts on the effective config so an off-by-one ".." shows up here. + PROJECT_DIR="$PROJECT_DIR" yq -e ' + .services.mitmproxy.volumes[] | + select( + .type == "bind" and + .source == (env(PROJECT_DIR) + "/.sandcat") and + .target == "/config/project" and + .read_only == true + ) + ' "$compose_file" + # FIXME vscode startup fails with capabilities dropped # yq -e '.services.mitmproxy.cap_drop[] | select(. == "ALL")' "$compose_file" } @@ -61,6 +77,7 @@ assert_cursor_environment_vars() { assert_common_volumes() { local compose_file=$1 + local filtered # Bind: Project root PROJECT_DIR="$PROJECT_DIR" yq -e ' @@ -68,16 +85,20 @@ assert_common_volumes() { select(.type == "bind" and .source == env(PROJECT_DIR) and .target == "/workspaces/project-sandbox") ' "$compose_file" - # Bind: .sandcat (read-only) - PROJECT_DIR="$PROJECT_DIR" yq -e " + # Agent .sandcat is the filtered copy, never the live project dir. + filtered=$(grep '^SANDCAT_AGENT_SANDCAT=' "$PROJECT_DIR/.devcontainer/.env") + filtered="${filtered#SANDCAT_AGENT_SANDCAT=}" + [[ -n "$filtered" ]] + [[ "$filtered" != "$PROJECT_DIR/.sandcat" ]] + FILTERED="$filtered" yq -e ' .services.agent.volumes[] | select( - .type == \"bind\" and - .source == (env(PROJECT_DIR) + \"/.sandcat\") and - .target == \"/workspaces/project-sandbox/.sandcat\" and + .type == "bind" and + .source == env(FILTERED) and + .target == "/workspaces/project-sandbox/.sandcat" and .read_only == true ) - " "$compose_file" + ' "$compose_file" # Volume: agent-home yq -e ' @@ -339,6 +360,23 @@ cursor_agent_compose_file_has_expected_content() { assert_customization_volumes_core "$compose_file" } +# `run devcontainer` drops exported vars. Compose interpolates from the +# process environment and `.env` in --project-directory, not from a +# sibling of -f. Load the file init wrote, then render the merged model. +render_effective_compose() { + local out=$1 + + [[ -f "$PROJECT_DIR/.devcontainer/.env" ]] + set -a + # shellcheck disable=SC1091 + source "$PROJECT_DIR/.devcontainer/.env" + set +a + [[ -n "${SANDCAT_AGENT_SANDCAT:-}" ]] + + docker compose --project-directory "$PROJECT_DIR/.devcontainer" \ + -f "$PROJECT_DIR/.devcontainer/compose-all.yml" config > "$out" +} + @test "devcontainer end-to-end: creates devcontainer config for claude agent" { export SANDCAT_MOUNT_CLAUDE_CONFIG="true" export SANDCAT_ENABLE_DOTFILES="true" @@ -353,9 +391,8 @@ cursor_agent_compose_file_has_expected_content() { assert_success assert_output --partial "Devcontainer dir created at .devcontainer" - # Use docker compose config to get the effective merged configuration local effective_file="$BATS_TEST_TMPDIR/effective-compose.yml" - docker compose -f "$PROJECT_DIR/.devcontainer/compose-all.yml" config > "$effective_file" + render_effective_compose "$effective_file" yq -e '.name == "project-sandbox"' "$effective_file" @@ -386,7 +423,7 @@ cursor_agent_compose_file_has_expected_content() { assert_output --partial "Devcontainer dir created at .devcontainer" local effective_file="$BATS_TEST_TMPDIR/effective-compose-cursor.yml" - docker compose -f "$PROJECT_DIR/.devcontainer/compose-all.yml" config > "$effective_file" + render_effective_compose "$effective_file" yq -e '.name == "project-sandbox"' "$effective_file" diff --git a/cli/test/installer/installer.bats b/cli/test/installer/installer.bats index 0e1c0477..d63351c4 100644 --- a/cli/test/installer/installer.bats +++ b/cli/test/installer/installer.bats @@ -65,35 +65,19 @@ FAKE } @test "install.sh accepts curl-based fetch when curl available" { - # Prereq check should not complain about HTTP tool when curl exists. - # The install still fails later (task 3 not implemented), but the - # error should be from the install path, not the prereq step. - local fake_bin="$BATS_TEST_TMPDIR/nobin" - mkdir -p "$fake_bin" - ln -s "$(command -v curl)" "$fake_bin/curl" - ln -s "$(command -v tar)" "$fake_bin/tar" - ln -s "$(command -v bash)" "$fake_bin/bash" - ln -s "$(command -v sh)" "$fake_bin/sh" - ln -s "$(command -v mktemp)" "$fake_bin/mktemp" - ln -s "$(command -v cat)" "$fake_bin/cat" - ln -s "$(command -v rm)" "$fake_bin/rm" - ln -s "$(command -v grep)" "$fake_bin/grep" - ln -s "$(command -v uname)" "$fake_bin/uname" - cat > "$fake_bin/yq" <<'FAKE' -#!/bin/bash -if [[ "${1-}" == "--version" ]]; then - echo "yq (https://github.com/mikefarah/yq) version v4.44.3" - exit 0 -fi -exit 0 -FAKE - chmod +x "$fake_bin/yq" + # Prereq check must accept curl + mikefarah yq and proceed into the + # install path (not fail on the yq/HTTP gate). + _stub_curl_with_fixture master - PATH="$fake_bin" run bash "$INSTALL_SH" - # Task 3 not implemented yet → fails with the stub error, NOT the yq one. - assert_failure + SANDCAT_HOME="$BATS_TEST_TMPDIR/home/.local/share/sandcat" \ + SANDCAT_BIN_DIR="$BATS_TEST_TMPDIR/home/.local/bin" \ + SANDCAT_REF=master \ + SANDCAT_NON_INTERACTIVE=true \ + PATH="$FAKE_BIN:$PATH" run bash "$INSTALL_SH" + + assert_success + refute_output --partial "Neither curl nor wget" refute_output --partial "mikefarah" - refute_output --partial "yq" } # Helper: builds a fake sandcat tarball (structure matches what @@ -248,16 +232,13 @@ FAKEYQ @test "install.sh fetches + extracts tarball into TMP_DIR" { _stub_curl_with_fixture master - # For this task we cover fetch + extract only; the install-files step - # is Task 4 and will still fail (the stub echo). Assert output shows - # the fetch + extraction succeeded before the stub error. SANDCAT_HOME="$BATS_TEST_TMPDIR/home/.local/share/sandcat" \ SANDCAT_BIN_DIR="$BATS_TEST_TMPDIR/home/.local/bin" \ SANDCAT_REF=master \ - PATH="$FAKE_BIN" run bash "$INSTALL_SH" + SANDCAT_NON_INTERACTIVE=true \ + PATH="$FAKE_BIN:$PATH" run bash "$INSTALL_SH" - # fetch + extract stages should have logged their [INFO]s before we hit - # the "install-files not implemented" stub from Task 3 flow. + assert_success assert_output --partial "Fetching" assert_output --partial "Extracting" } diff --git a/cli/test/mitmproxy/netbird_dns.bats b/cli/test/mitmproxy/netbird_dns.bats new file mode 100644 index 00000000..c836ff24 --- /dev/null +++ b/cli/test/mitmproxy/netbird_dns.bats @@ -0,0 +1,282 @@ +#!/usr/bin/env bats +# Tests for mitmproxy-init.sh NetBird DNS publishing into the shared volume. + +setup() { + load "$BATS_TEST_DIRNAME/../wg-client/test_helper" + export NB_PEER_NAME="test-proxy" + export NETBIRD_PEER_LIFECYCLE_PATH="$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/netbird-peer-lifecycle.sh" + # Re-source mitmproxy-init after wg-client helper (which sources wg-client-init). + # shellcheck source=../../templates/devcontainer/sandcat/scripts/mitmproxy-init.sh + source "$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/mitmproxy-init.sh" + + NETBIRD_DNS_DOMAIN="netbird.selfhosted" + NETBIRD_DNS_CONF_PATH="$BATS_TEST_TMPDIR/netbird-peers.conf" + export NETBIRD_DNS_DOMAIN NETBIRD_DNS_CONF_PATH + rm -f "$NETBIRD_DNS_CONF_PATH" + + STATUS_JSON="$BATS_TEST_TMPDIR/netbird-status.json" +} + +teardown() { + unstub_all +} + +@test "publish_netbird_dns writes address= records from netbirdIp (NetBird >= 0.28)" { + # Real NetBird status --json shape: peers.details[].netbirdIp (0.28+), not .ip. + cat >"$STATUS_JSON" <<'JSON' +{ + "peers": { + "total": 1, + "connected": 1, + "details": [ + { + "fqdn": "test-proxy-peer.netbird.selfhosted", + "netbirdIp": "100.79.176.190", + "status": "Connected" + } + ] + } +} +JSON + stub netbird "status --json : cat '$STATUS_JSON'" + + publish_netbird_dns + + run cat "$NETBIRD_DNS_CONF_PATH" + assert_success + assert_output --partial "local=/netbird.selfhosted/" + assert_output --partial "host-record=test-proxy-peer.netbird.selfhosted,100.79.176.190" + assert_output --partial "address=/test-proxy-peer.netbird.selfhosted/100.79.176.190" +} + +@test "publish_netbird_dns still accepts legacy peers.details[].ip" { + cat >"$STATUS_JSON" <<'JSON' +{ + "peers": { + "details": [ + { + "fqdn": "test-proxy-peer.netbird.selfhosted", + "ip": "100.64.0.5" + } + ] + } +} +JSON + stub netbird "status --json : cat '$STATUS_JSON'" + + publish_netbird_dns + + run cat "$NETBIRD_DNS_CONF_PATH" + assert_success + assert_output --partial "address=/test-proxy-peer.netbird.selfhosted/100.64.0.5" +} + +@test "publish_netbird_dns strips CIDR suffix from netbirdIp" { + cat >"$STATUS_JSON" <<'JSON' +{ + "peers": { + "details": [ + { + "fqdn": "test-proxy-peer.netbird.selfhosted", + "netbirdIp": "100.79.176.190/16" + } + ] + } +} +JSON + stub netbird "status --json : cat '$STATUS_JSON'" + + publish_netbird_dns + + run cat "$NETBIRD_DNS_CONF_PATH" + assert_success + assert_output --partial "address=/test-proxy-peer.netbird.selfhosted/100.79.176.190" + run grep -F '100.79.176.190/16' "$NETBIRD_DNS_CONF_PATH" + assert_failure +} + +@test "publish_netbird_dns truncates stale records when no peers remain" { + cat >"$STATUS_JSON" <<'JSON' +{ + "peers": { + "details": [ + { + "fqdn": "test-proxy-peer.netbird.selfhosted", + "status": "Connected" + } + ] + } +} +JSON + printf 'address=/stale.netbird.selfhosted/100.64.0.1\n' >"$NETBIRD_DNS_CONF_PATH" + stub netbird "status --json : cat '$STATUS_JSON'" + + publish_netbird_dns + + [[ -f "$NETBIRD_DNS_CONF_PATH" ]] + run grep -F 'stale.netbird.selfhosted' "$NETBIRD_DNS_CONF_PATH" + assert_failure +} + +@test "publish_netbird_dns rejects a substring domain match" { + cat >"$STATUS_JSON" <<'JSON' +{ + "peers": { + "details": [ + { + "fqdn": "evil.netbird.selfhosted.attacker.example", + "netbirdIp": "100.64.0.9" + } + ] + } +} +JSON + stub netbird "status --json : cat '$STATUS_JSON'" + + publish_netbird_dns + + [[ ! -s "$NETBIRD_DNS_CONF_PATH" ]] +} + +@test "publish_netbird_dns rejects an FQDN with a slash" { + cat >"$STATUS_JSON" <<'JSON' +{ + "peers": { + "details": [ + { + "fqdn": "foo/bar.netbird.selfhosted", + "netbirdIp": "100.64.0.9" + } + ] + } +} +JSON + stub netbird "status --json : cat '$STATUS_JSON'" + + publish_netbird_dns + + [[ ! -s "$NETBIRD_DNS_CONF_PATH" ]] +} + +@test "publish_netbird_dns strips a four-octet IP suffix alias" { + cat >"$STATUS_JSON" <<'JSON' +{ + "peers": { + "details": [ + { + "fqdn": "myapp-proxy-100-64-0-5.netbird.selfhosted", + "netbirdIp": "100.64.0.5" + } + ] + } +} +JSON + stub netbird "status --json : cat '$STATUS_JSON'" + + publish_netbird_dns + + run cat "$NETBIRD_DNS_CONF_PATH" + assert_success + assert_output --partial "host-record=myapp-proxy.netbird.selfhosted,100.64.0.5" + assert_output --partial "host-record=myapp-proxy-100-64-0-5.netbird.selfhosted,100.64.0.5" + run grep -F 'host-record=myapp-proxy-100-64.netbird.selfhosted' "$NETBIRD_DNS_CONF_PATH" + assert_failure +} + +@test "publish_netbird_dns does not emit local= when forwarding to a nameserver" { + cat >"$STATUS_JSON" <<'JSON' +{ + "dnsServers": [ + { + "domains": ["netbird.selfhosted"], + "servers": ["100.64.0.1:53"] + } + ], + "peers": { + "details": [ + { + "fqdn": "test-proxy-peer.netbird.selfhosted", + "netbirdIp": "100.64.0.5" + } + ] + } +} +JSON + stub netbird "status --json : cat '$STATUS_JSON'" + + publish_netbird_dns + + run cat "$NETBIRD_DNS_CONF_PATH" + assert_success + assert_output --partial "server=/netbird.selfhosted/100.64.0.1" + assert_output --partial "address=/test-proxy-peer.netbird.selfhosted/100.64.0.5" + run grep -F 'local=/netbird.selfhosted/' "$NETBIRD_DNS_CONF_PATH" + assert_failure +} + +@test "clear_mitmproxy_health_sentinels deletes stale dns.conf and published CA first" { + MITMPROXY_HOME="$BATS_TEST_TMPDIR/mitm-home" + MITMPROXY_PUBLIC="$BATS_TEST_TMPDIR/mitm-public" + mkdir -p "$MITMPROXY_HOME" "$MITMPROXY_PUBLIC" + printf 'stale\n' >"$MITMPROXY_HOME/dns.conf" + printf 'old-ca\n' >"$MITMPROXY_PUBLIC/mitmproxy-ca-cert.pem" + + clear_mitmproxy_health_sentinels + + [[ ! -e "$MITMPROXY_HOME/dns.conf" ]] + [[ ! -e "$MITMPROXY_PUBLIC/mitmproxy-ca-cert.pem" ]] +} + +@test "ensure_mitmweb_password persists a generated password" { + MITMPROXY_HOME="$BATS_TEST_TMPDIR/mitm-home" + MITMPROXY_WEB_PASSWORD_FILE="$MITMPROXY_HOME/web_password" + mkdir -p "$MITMPROXY_HOME" + + ensure_mitmweb_password >/dev/null + local pw + pw=$(cat "$MITMPROXY_WEB_PASSWORD_FILE") + [[ ${#pw} -ge 16 ]] + ensure_mitmweb_password >/dev/null + [[ "$(cat "$MITMPROXY_WEB_PASSWORD_FILE")" == "$pw" ]] +} + +@test "install_upstream_ca_bundles copies PEMs into the trust store" { + UPSTREAM_CA_DIR="$BATS_TEST_TMPDIR/upstream-ca" + UPSTREAM_CA_INSTALL_DIR="$BATS_TEST_TMPDIR/ca-certificates" + UPSTREAM_CA_CERTIFI_BUNDLE="$BATS_TEST_TMPDIR/certifi.pem" + mkdir -p "$UPSTREAM_CA_DIR" + printf '%s\n' '-----BEGIN CERTIFICATE-----' 'ABC' '-----END CERTIFICATE-----' \ + >"$UPSTREAM_CA_DIR/000-company.crt" + : >"$UPSTREAM_CA_CERTIFI_BUNDLE" + + install_upstream_ca_bundles + + [[ -f "$UPSTREAM_CA_INSTALL_DIR/000-company.crt" ]] + run grep -F 'BEGIN CERTIFICATE' "$UPSTREAM_CA_CERTIFI_BUNDLE" + assert_success +} + +@test "lockdown_wt0_ingress default-denies new INPUT on wt0" { + local log="$BATS_TEST_TMPDIR/iptables.log" + : >"$log" + mkdir -p "$BATS_TEST_TMPDIR/bin" + cat >"$BATS_TEST_TMPDIR/bin/iptables" <>"$log" +case "\$*" in + *-C*) exit 1 ;; +esac +exit 0 +EOF + cat >"$BATS_TEST_TMPDIR/bin/ip" <<'EOF' +#!/bin/sh +exit 0 +EOF + chmod +x "$BATS_TEST_TMPDIR/bin/iptables" "$BATS_TEST_TMPDIR/bin/ip" + PATH="$BATS_TEST_TMPDIR/bin:$PATH" lockdown_wt0_ingress wt0 + + run grep -F 'INPUT -i wt0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT' "$log" + assert_success + run grep -F 'INPUT -i wt0 -j DROP' "$log" + assert_success +} diff --git a/cli/test/mitmproxy/netbird_peer_lifecycle.bats b/cli/test/mitmproxy/netbird_peer_lifecycle.bats new file mode 100644 index 00000000..aa1f23bb --- /dev/null +++ b/cli/test/mitmproxy/netbird_peer_lifecycle.bats @@ -0,0 +1,404 @@ +#!/usr/bin/env bats + +setup() { + load "$BATS_TEST_DIRNAME/../composefile/test_helper" + + SCRIPT="$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/netbird-peer-lifecycle.sh" + # shellcheck source=/dev/null + source "$SCRIPT" + + export NB_PEER_NAME="myapp-sandbox-proxy" + export NB_MANAGEMENT_URL="http://mgmt.test:33073" + export NETBIRD_STATE_ROOT="$BATS_TEST_TMPDIR/var-lib-netbird" + export NETBIRD_SETTINGS_PATH="$BATS_TEST_TMPDIR/settings.json" + unset NB_API_TOKEN +} + +teardown() { + unstub_all +} + +@test "local state causes reconnect without a management request" { + mkdir -p "$NETBIRD_STATE_ROOT" + printf '{}\n' >"$NETBIRD_STATE_ROOT/config.json" + + run netbird_replace_same_name_peer_if_needed + + assert_success + assert_output --partial "reconnect" +} + +@test "0.72 enrolled default.json reconnects without a management request" { + mkdir -p "$NETBIRD_STATE_ROOT" + printf '{}\n' >"$NETBIRD_STATE_ROOT/default.json" + stub netbird "status --json : echo '{\"status\":\"Connected\"}'" + + run netbird_replace_same_name_peer_if_needed + + assert_success + assert_output --partial "reconnect" +} + +@test "0.72 NeedsLogin default.json is not enrolled local state" { + mkdir -p "$NETBIRD_STATE_ROOT" + printf '{}\n' >"$NETBIRD_STATE_ROOT/default.json" + export NB_API_TOKEN="tok" + stub netbird "status --json : echo '{\"status\":\"NeedsLogin\"}'" + stub curl \ + "-sf --max-time 10 -H * http://mgmt.test:33073/api/peers : echo '[{\"id\":\"abc\",\"name\":\"myapp-sandbox-proxy\"}]'" \ + "-sf --max-time 10 -H * -X DELETE http://mgmt.test:33073/api/peers/abc : :" + + run netbird_replace_same_name_peer_if_needed + + assert_success + assert_output --partial "replace" +} + +@test "missing local state replaces an existing peer" { + export NB_API_TOKEN="tok" + stub curl \ + "-sf --max-time 10 -H * http://mgmt.test:33073/api/peers : echo '[{\"id\":\"abc\",\"name\":\"myapp-sandbox-proxy\"}]'" \ + "-sf --max-time 10 -H * -X DELETE http://mgmt.test:33073/api/peers/abc : :" + + run netbird_replace_same_name_peer_if_needed + + assert_success + assert_output --partial "replace" +} + +@test "replacement skips when the API token is missing" { + run netbird_replace_same_name_peer_if_needed + + assert_success + assert_output --partial "skipping same-name peer check" +} + +@test "netbird_mgmt_delete_peer_by_name deletes matched peer by id" { + export NB_API_TOKEN="tok" + stub curl \ + "-sf --max-time 10 -H * http://mgmt.test:33073/api/peers : echo '[{\"id\":\"abc\",\"name\":\"myapp-sandbox-proxy\"}]'" \ + "-sf --max-time 10 -H * -X DELETE http://mgmt.test:33073/api/peers/abc : :" + + run netbird_mgmt_delete_peer_by_name "myapp-sandbox-proxy" + + assert_success +} + +@test "replacement fails when multiple management peers match the name" { + export NB_API_TOKEN="tok" + stub curl \ + "-sf --max-time 10 -H * http://mgmt.test:33073/api/peers : echo '[{\"id\":\"abc\",\"name\":\"myapp-sandbox-proxy\"},{\"id\":\"def\",\"hostname\":\"MYAPP-SANDBOX-PROXY\"}]'" + + run netbird_replace_same_name_peer_if_needed + + assert_failure + assert_output --partial "multiple management peers" +} + +@test "peer lookup matches name hostname and dns_label case-insensitively" { + export NB_API_TOKEN="tok" + stub curl \ + "-sf --max-time 10 -H * http://mgmt.test:33073/api/peers : echo '[{\"id\":\"by-name\",\"name\":\"NAME-TARGET\"},{\"id\":\"by-hostname\",\"hostname\":\"Hostname-Target\"},{\"id\":\"by-label\",\"dns_label\":\"label-target\"}]'" \ + "-sf --max-time 10 -H * http://mgmt.test:33073/api/peers : echo '[{\"id\":\"by-name\",\"name\":\"NAME-TARGET\"},{\"id\":\"by-hostname\",\"hostname\":\"Hostname-Target\"},{\"id\":\"by-label\",\"dns_label\":\"label-target\"}]'" \ + "-sf --max-time 10 -H * http://mgmt.test:33073/api/peers : echo '[{\"id\":\"by-name\",\"name\":\"NAME-TARGET\"},{\"id\":\"by-hostname\",\"hostname\":\"Hostname-Target\"},{\"id\":\"by-label\",\"dns_label\":\"label-target\"}]'" + + lookup_all_peer_names() { + netbird_mgmt_find_peer_id_by_name "name-target" + netbird_mgmt_find_peer_id_by_name "hostname-target" + netbird_mgmt_find_peer_id_by_name "LABEL-TARGET" + } + run lookup_all_peer_names + + assert_success + assert_output $'by-name\nby-hostname\nby-label' +} + +@test "dns_label update uses the local peer FQDN" { + export NB_API_TOKEN="tok" + stub netbird \ + "status --json : echo '{\"fqdn\":\"myapp-sandbox-proxy-100-64-0-5.netbird.selfhosted\"}'" + stub curl \ + "-sf --max-time 10 -H * http://mgmt.test:33073/api/peers : echo '[{\"id\":\"abc\",\"fqdn\":\"myapp-sandbox-proxy-100-64-0-5.netbird.selfhosted\"}]'" \ + "-sf --max-time 10 -H * -X PUT -H 'Content-Type: application/json' -d '{\"dns_label\":\"myapp-sandbox-proxy\"}' http://mgmt.test:33073/api/peers/abc : echo '{\"fqdn\":\"myapp-sandbox-proxy.netbird.selfhosted\"}'" + + run netbird_set_dns_label + + assert_success + assert_output --partial "dns_label set" +} + +@test "dns_label update soft-skips without an API token" { + run netbird_set_dns_label + + assert_success + assert_output --partial "skipping dns_label" +} + +@test "dns_label update requires an explicit peer name" { + unset NB_PEER_NAME + + run netbird_set_dns_label + + assert_failure + assert_output --partial "NB_PEER_NAME" +} + +@test "netbird_resolve_secret_ref leaves plaintext unchanged" { + run netbird_resolve_secret_ref "nbp_plain" + assert_success + assert_output "nbp_plain" +} + +@test "netbird_resolve_secret_ref uses op read for op:// refs" { + stub timeout "60 op read op://Vault/x/credential : echo resolved-token" + run netbird_resolve_secret_ref "op://Vault/x/credential" + assert_success + assert_output "resolved-token" +} + +@test "netbird_resolve_secret_ref logs into pass-cli once for two pass:// refs" { + stub pass-cli \ + "login : :" \ + "info : echo ' - Personal Access Token: pst_test'" + stub timeout \ + "60 pass-cli vault list : :" \ + "60 pass-cli item view pass://Vault/Item/password : echo secret-a" \ + "60 pass-cli item view pass://Vault/Other/password : echo secret-b" + run bash -c ' + source "$1" + netbird_resolve_secret_ref "pass://Vault/Item/password" + echo --- + netbird_resolve_secret_ref "pass://Vault/Other/password" + ' _ "$SCRIPT" + assert_success + assert_output $'secret-a\n---\nsecret-b' +} + +@test "netbird_resolve_secret_ref retries pass-cli item view after warmup" { + stub pass-cli \ + "login : :" \ + "info : echo ' - Personal Access Token: pst_test'" + stub timeout \ + "60 pass-cli vault list : :" \ + "60 pass-cli item view pass://Vault/Item/password : printf PARTIAL; exit 1" \ + "60 pass-cli item view pass://Vault/Item/password : echo secret-a" + run netbird_resolve_secret_ref "pass://Vault/Item/password" + assert_success + assert_output "secret-a" + refute_output --partial "PARTIAL" +} + +@test "pass-cli login rejects non-PAT session" { + stub pass-cli \ + "login : :" \ + "info : echo 'ID: user@example.com'" \ + "logout : :" + run netbird_pass_cli_login_once + assert_failure + assert_output --partial "not a Personal Access Token" +} + +@test "netbird_prepare_enroll_credentials resolves NB_API_TOKEN before replace" { + export NB_API_TOKEN="op://Vault/x/credential" + export NB_SETUP_KEY="setup-literal" + stub timeout "60 op read op://Vault/x/credential : echo tok" + stub curl \ + "-sf --max-time 10 -H * http://mgmt.test:33073/api/peers : echo '[]'" + netbird_prepare_enroll_credentials + run netbird_replace_same_name_peer_if_needed + assert_success +} + +@test "netbird_prepare_enroll_credentials flattens object enrollment key from settings" { + unset NB_SETUP_KEY + unset NB_API_TOKEN + printf '%s\n' '{"netbird_enrollment_key":{"op":"op://Vault/x/credential"}}' >"$NETBIRD_SETTINGS_PATH" + stub timeout "60 op read op://Vault/x/credential : echo nbp_resolved" + netbird_prepare_enroll_credentials + [[ "$NB_SETUP_KEY" == "nbp_resolved" ]] +} + +@test "netbird_resolve_api_token flattens object-shaped token from settings" { + unset NB_API_TOKEN + printf '%s\n' '{"netbird_api_token":{"value":"nbp_from_settings"}}' >"$NETBIRD_SETTINGS_PATH" + run netbird_resolve_api_token + assert_success + assert_output "nbp_from_settings" +} + +@test "netbird_resolve_api_token resolves op:// from object-shaped settings" { + unset NB_API_TOKEN + printf '%s\n' '{"netbird_api_token":{"op":"op://Vault/x/credential"}}' >"$NETBIRD_SETTINGS_PATH" + stub timeout "60 op read op://Vault/x/credential : echo tok" + run netbird_resolve_api_token + assert_success + assert_output "tok" +} + +@test "replacement skips when settings token is an empty object" { + unset NB_API_TOKEN + printf '%s\n' '{"netbird_api_token":{}}' >"$NETBIRD_SETTINGS_PATH" + run netbird_replace_same_name_peer_if_needed + assert_success + assert_output --partial "skipping same-name peer check" +} + +@test "mitmproxy-init aborts start_netbird when same-name replace fails" { + local lifecycle="$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/netbird-peer-lifecycle.sh" + run awk '/^netbird_start\(\)/,/^}/' "$lifecycle" + assert_success + assert_output --partial "netbird_replace_same_name_peer_if_needed || return 1" + refute_output --partial "continuing with netbird up" +} + +@test "proxy-peer-init uses the shared lifecycle start path" { + local init="$SCT_ROOT/../docs/examples/proxy-peer/scripts/proxy-peer-init.sh" + run grep -F 'netbird_start' "$init" + assert_success + run grep -F -- '--setup-key "${NB_SETUP_KEY}"' "$init" + assert_failure +} + +@test "example does not keep a second lifecycle script" { + [[ ! -f "$SCT_ROOT/../docs/examples/proxy-peer/scripts/netbird-peer-lifecycle.sh" ]] +} + +@test "supervise_netbird_daemon re-enroll prepares credentials" { + local lifecycle="$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/netbird-peer-lifecycle.sh" + run awk '/^netbird_supervise_daemon\(\)/,/^}/' "$lifecycle" + assert_success + assert_output --partial "netbird_start" + refute_output --partial "netbird_replace_same_name_peer_if_needed || true" +} + +@test "proxy-peer supervise uses the shared lifecycle supervisor" { + local init="$SCT_ROOT/../docs/examples/proxy-peer/scripts/proxy-peer-init.sh" + run grep -F 'netbird_supervise_daemon' "$init" + assert_success +} + +@test "netbird_up_enroll uses a setup-key file and a timeout" { + run awk '/^netbird_up_enroll\(\)/,/^}/' "$SCRIPT" + assert_success + assert_output --partial "--setup-key-file" + assert_output --partial "timeout" + refute_output --partial '--setup-key "${NB_SETUP_KEY}"' +} + +@test "mitmproxy-init does not copy enrollment key into NB_SETUP_KEY with raw jq" { + local init="$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/mitmproxy-init.sh" + # Raw jq dumps object JSON into env and skips prepare's flatten-if-empty path. + run grep -F "NB_SETUP_KEY=\$(jq -r '.netbird_enrollment_key" "$init" + assert_failure +} + +@test "prepare profile converts string ManagementURL to a Go url.URL object" { + # NetBird 0.72 Config.ManagementURL is *url.URL; a JSON string crashes the + # daemon with "cannot unmarshal string into Go struct field Config.ManagementURL". + mkdir -p "$NETBIRD_STATE_ROOT" + printf '%s\n' '{"ManagementURL":"http://192.0.2.1:33073","AdminURL":"http://192.0.2.1:33073","PrivateKey":"keep-me"}' \ + >"$NETBIRD_STATE_ROOT/default.json" + export NB_MANAGEMENT_URL="http://192.0.2.1:33073" + export NETBIRD_IFACE="wt0" + + netbird_prepare_local_management_profile + + run jq -c '{scheme:.ManagementURL.Scheme, host:.ManagementURL.Host, key:.PrivateKey, admin:.AdminURL.Scheme}' \ + "$NETBIRD_STATE_ROOT/default.json" + assert_success + assert_output '{"scheme":"http","host":"192.0.2.1:33073","key":"keep-me","admin":"http"}' +} + +@test "prepare profile does not seed a missing default.json" { + # 0.72 writes native url.URL objects; seeding a string profile is what broke + # the daemon. Leave a missing file to the client. + export NB_MANAGEMENT_URL="http://192.0.2.1:33073" + + netbird_prepare_local_management_profile + + [[ ! -e "$NETBIRD_STATE_ROOT/default.json" ]] +} + +@test "init scripts do not seed ManagementURL as a JSON string" { + run grep -F '"ManagementURL": "$mgmt_url"' \ + "$SCT_TEMPLATEDIR/devcontainer/sandcat/scripts/mitmproxy-init.sh" \ + "$SCT_ROOT/../docs/examples/proxy-peer/scripts/proxy-peer-init.sh" + assert_failure +} + +@test "prepare profile coerces leftover hostname string URLs without rewriting" { + mkdir -p "$NETBIRD_STATE_ROOT" + printf '%s\n' '{"ManagementURL":"http://host.docker.internal:33073","AdminURL":"http://host.docker.internal:33073","PrivateKey":"keep-me"}' \ + >"$NETBIRD_STATE_ROOT/default.json" + export NB_MANAGEMENT_URL="http://host.docker.internal:33073" + + netbird_prepare_local_management_profile + + run jq -c '{host:.ManagementURL.Host, key:.PrivateKey}' "$NETBIRD_STATE_ROOT/default.json" + assert_success + assert_output '{"host":"host.docker.internal:33073","key":"keep-me"}' +} + +@test "prepare profile leaves default.json untouched when jq fails" { + mkdir -p "$NETBIRD_STATE_ROOT" + printf 'not-json\n' >"$NETBIRD_STATE_ROOT/default.json" + export NB_MANAGEMENT_URL="http://192.0.2.1:33073" + + run netbird_prepare_local_management_profile + + assert_failure + run cat "$NETBIRD_STATE_ROOT/default.json" + assert_output "not-json" +} + +@test "prepare profile rewrites a stale self-hosted URL for cloud enrollment" { + mkdir -p "$NETBIRD_STATE_ROOT" + printf '%s\n' '{"ManagementURL":{"Scheme":"http","Host":"192.0.2.8:33073","Path":""},"AdminURL":{"Scheme":"http","Host":"192.0.2.8:33073","Path":""},"PrivateKey":"keep-me"}' \ + >"$NETBIRD_STATE_ROOT/default.json" + export NB_MANAGEMENT_URL="https://api.netbird.io" + + netbird_prepare_local_management_profile + + run jq -c '{scheme:.ManagementURL.Scheme, host:.ManagementURL.Host, key:.PrivateKey}' \ + "$NETBIRD_STATE_ROOT/default.json" + assert_success + assert_output '{"scheme":"https","host":"api.netbird.io","key":"keep-me"}' +} + +@test "replacement fails when the management GET fails" { + export NB_API_TOKEN="tok" + stub curl \ + "-sf --max-time 10 -H * http://mgmt.test:33073/api/peers : exit 22" + + run netbird_replace_same_name_peer_if_needed + + assert_failure +} + +@test "replacement fails when the management DELETE fails" { + export NB_API_TOKEN="tok" + stub curl \ + "-sf --max-time 10 -H * http://mgmt.test:33073/api/peers : echo '[{\"id\":\"abc\",\"name\":\"myapp-sandbox-proxy\"}]'" \ + "-sf --max-time 10 -H * -X DELETE http://mgmt.test:33073/api/peers/abc : exit 22" + + run netbird_replace_same_name_peer_if_needed + + assert_failure + assert_output --partial "replace" +} + +@test "prepare profile refreshes an existing object URL from NB_MANAGEMENT_URL" { + mkdir -p "$NETBIRD_STATE_ROOT" + printf '%s\n' '{"ManagementURL":{"Scheme":"http","Host":"192.0.2.8:33073","Path":""},"AdminURL":{"Scheme":"http","Host":"192.0.2.8:33073","Path":""},"WgIface":"wt0","PrivateKey":"keep-me"}' \ + >"$NETBIRD_STATE_ROOT/default.json" + export NB_MANAGEMENT_URL="http://192.0.2.1:33073" + export NETBIRD_IFACE="wt0" + export NETBIRD_WG_PORT="51821" + + netbird_prepare_local_management_profile + + run jq -c '{host:.ManagementURL.Host, port:.WgPort, iface:.WgIface, key:.PrivateKey}' \ + "$NETBIRD_STATE_ROOT/default.json" + assert_success + assert_output '{"host":"192.0.2.1:33073","port":51821,"iface":"wt0","key":"keep-me"}' +} diff --git a/cli/test/mitmproxy/test_mitmproxy_addon.py b/cli/test/mitmproxy/test_mitmproxy_addon.py index fd152596..0f41a0ff 100644 --- a/cli/test/mitmproxy/test_mitmproxy_addon.py +++ b/cli/test/mitmproxy/test_mitmproxy_addon.py @@ -281,6 +281,60 @@ def test_dns_trailing_dot_stripped(self, addon_cls): assert addon._is_request_allowed(None, "api.github.com.") is True assert addon._is_request_allowed("GET", "api.github.com.") is True + def test_enabled_false_skips_rule(self, addon_cls): + addon = addon_cls() + addon.network_rules = [ + {"action": "allow", "host": "api.example.com", "enabled": False}, + {"action": "allow", "host": "*"}, + ] + # First rule skipped → falls through to allow * + assert addon._is_request_allowed("GET", "api.example.com") is True + addon.network_rules = [ + {"action": "allow", "host": "api.example.com", "enabled": False}, + ] + assert addon._is_request_allowed("GET", "api.example.com") is False + + def test_enabled_true_matches_as_today(self, addon_cls): + addon = addon_cls() + addon.network_rules = [ + {"action": "allow", "host": "api.example.com", "enabled": True}, + ] + assert addon._is_request_allowed("GET", "api.example.com") is True + + def test_enabled_absent_defaults_true(self, addon_cls): + addon = addon_cls() + addon.network_rules = [ + {"action": "allow", "host": "api.example.com"}, + ] + assert addon._is_request_allowed("GET", "api.example.com") is True + + @pytest.mark.parametrize("disabled", [False, "false", "False", "0", 0, "off", "no"]) + def test_falsey_enabled_values_disable_rule(self, addon_cls, disabled): + addon = addon_cls() + addon.network_rules = [ + {"action": "allow", "host": "api.example.com", "enabled": disabled}, + ] + assert addon._is_request_allowed("GET", "api.example.com") is False + + @pytest.mark.parametrize("truthy", [True, "true", "True", "1", 1, "on", "yes"]) + def test_truthy_enabled_values_keep_rule(self, addon_cls, truthy): + addon = addon_cls() + addon.network_rules = [ + {"action": "allow", "host": "api.example.com", "enabled": truthy}, + ] + assert addon._is_request_allowed("GET", "api.example.com") is True + + def test_unrecognized_enabled_value_keeps_rule_and_warns(self, addon_cls): + addon = addon_cls() + rules = [{"action": "allow", "host": "api.example.com", "enabled": "maybe"}] + + with patch.object(common.ctx, "log") as log: + addon._load_network_rules(rules) + + assert addon._is_request_allowed("GET", "api.example.com") is True + warning = " ".join(str(call) for call in log.warn.call_args_list) + assert "maybe" in warning + # --------------------------------------------------------------------------- # Network presets — {"preset": ""} expansion (issue #2). @@ -1011,6 +1065,51 @@ def test_missing_env_section_omits_vars(self, addon_cls, tmp_path): assert "# names: K" in content assert "export K=SANDCAT_PLACEHOLDER_K" in content + def test_netbird_dns_domain_emitted_when_env_var_set( + self, addon_cls, tmp_path, monkeypatch + ): + monkeypatch.setenv("NETBIRD_DNS_DOMAIN", "netbird.selfhosted") + settings = {"env": {}} + p = tmp_path / "settings.json" + p.write_text(json.dumps(settings)) + env_path = tmp_path / "sandcat.env" + addon = addon_cls() + with patch(f"{_COMMON}.SETTINGS_PATHS", [str(p)]), \ + patch(f"{_COMMON}.SANDCAT_ENV_PATH", str(env_path)): + addon.load(MagicMock()) + content = env_path.read_text() + assert "export SANDCAT_NETBIRD_DNS_DOMAIN=netbird.selfhosted" in content + + def test_netbird_dns_domain_not_emitted_when_env_var_absent( + self, addon_cls, tmp_path, monkeypatch + ): + monkeypatch.delenv("NETBIRD_DNS_DOMAIN", raising=False) + settings = {"env": {}} + p = tmp_path / "settings.json" + p.write_text(json.dumps(settings)) + env_path = tmp_path / "sandcat.env" + addon = addon_cls() + with patch(f"{_COMMON}.SETTINGS_PATHS", [str(p)]), \ + patch(f"{_COMMON}.SANDCAT_ENV_PATH", str(env_path)): + addon.load(MagicMock()) + content = env_path.read_text() + assert "SANDCAT_NETBIRD_DNS_DOMAIN" not in content + + def test_netbird_dns_domain_custom_value( + self, addon_cls, tmp_path, monkeypatch + ): + monkeypatch.setenv("NETBIRD_DNS_DOMAIN", "nb.corp.example.com") + settings = {"env": {}} + p = tmp_path / "settings.json" + p.write_text(json.dumps(settings)) + env_path = tmp_path / "sandcat.env" + addon = addon_cls() + with patch(f"{_COMMON}.SETTINGS_PATHS", [str(p)]), \ + patch(f"{_COMMON}.SANDCAT_ENV_PATH", str(env_path)): + addon.load(MagicMock()) + content = env_path.read_text() + assert "export SANDCAT_NETBIRD_DNS_DOMAIN=nb.corp.example.com" in content + # --------------------------------------------------------------------------- # Env value quoting — applies regardless of variant. @@ -1247,7 +1346,7 @@ def test_pass_reference_resolved_via_subprocess(self, addon_cls): assert value == "secret-value" mock_run.assert_called_once_with( ["pass-cli", "item", "view", "pass://vault/item/field"], - capture_output=True, text=True, timeout=30, + capture_output=True, text=True, timeout=60, ) def test_pass_without_prefix_raises(self, addon_cls): @@ -1263,7 +1362,8 @@ def test_pass_cli_not_found_raises(self, addon_cls): def test_pass_cli_failure_raises(self, addon_cls): entry = {"pass": "pass://vault/item/field", "hosts": []} - with patch(f"{_COMMON}.subprocess.run") as mock_run: + with patch(f"{_COMMON}.subprocess.run") as mock_run, \ + patch(f"{_COMMON}.time.sleep"): mock_run.return_value = MagicMock( returncode=1, stdout="", stderr="unauthorized" ) diff --git a/cli/test/netbird/netbird_peer_names.bats b/cli/test/netbird/netbird_peer_names.bats new file mode 100644 index 00000000..cc6c106f --- /dev/null +++ b/cli/test/netbird/netbird_peer_names.bats @@ -0,0 +1,54 @@ +#!/usr/bin/env bats + +setup() { + load test_helper + source "$SCT_LIBDIR/netbird.bash" + export HOME="$BATS_TEST_TMPDIR/home" + mkdir -p "$HOME/.config/sandcat" + PROJECT_DIR="$BATS_TEST_TMPDIR/project" + mkdir -p "$PROJECT_DIR/.sandcat" + cd "$PROJECT_DIR" || return 1 + SETTINGS="$PROJECT_DIR/.sandcat/settings.json" +} + +teardown() { + unstub_all +} + +@test "netbird_ensure_peer_name_settings writes proxy default when key absent" { + echo '{}' >"$SETTINGS" + netbird_ensure_peer_name_settings "$SETTINGS" "myapp-sandbox" + + run yq -r '.netbird_peer_name_proxy' "$SETTINGS" + assert_output "myapp-sandbox-proxy" + run grep -F 'netbird_peer_name_proxy_peer' "$SETTINGS" + assert_failure +} + +@test "netbird_ensure_peer_name_settings refills empty string keys" { + cat >"$SETTINGS" <<'JSON' +{"netbird_peer_name_proxy": ""} +JSON + netbird_ensure_peer_name_settings "$SETTINGS" "myapp-sandbox" + + run yq -r '.netbird_peer_name_proxy' "$SETTINGS" + assert_output "myapp-sandbox-proxy" +} + +@test "netbird_ensure_peer_name_settings overwrites non-empty committed names" { + cat >"$SETTINGS" <<'JSON' +{"netbird_peer_name_proxy": "victim-peer"} +JSON + netbird_ensure_peer_name_settings "$SETTINGS" "myapp-sandbox" + + run yq -r '.netbird_peer_name_proxy' "$SETTINGS" + assert_output "myapp-sandbox-proxy" +} + +@test "netbird_ensure_peer_name_settings creates settings file when missing" { + rm -f "$SETTINGS" + netbird_ensure_peer_name_settings "$SETTINGS" "myapp-sandbox" + [[ -f "$SETTINGS" ]] + run yq -r '.netbird_peer_name_proxy' "$SETTINGS" + assert_output "myapp-sandbox-proxy" +} diff --git a/cli/test/netbird/netbird_secret_flatten.bats b/cli/test/netbird/netbird_secret_flatten.bats new file mode 100644 index 00000000..e60aa5aa --- /dev/null +++ b/cli/test/netbird/netbird_secret_flatten.bats @@ -0,0 +1,137 @@ +#!/usr/bin/env bats + +setup() { + load test_helper + source "$SCT_LIBDIR/netbird.bash" + export HOME="$BATS_TEST_TMPDIR/home" + mkdir -p "$HOME/.config/sandcat" + PROJECT_DIR="$BATS_TEST_TMPDIR/project" + mkdir -p "$PROJECT_DIR/.sandcat" + cd "$PROJECT_DIR" || return 1 +} + +teardown() { + unstub_all +} + +@test "netbird_flatten_secret_setting returns empty for empty and null" { + run netbird_flatten_secret_setting "" + assert_success + assert_output "" + run netbird_flatten_secret_setting "null" + assert_success + assert_output "" +} + +@test "netbird_flatten_secret_setting returns JSON strings as-is" { + run netbird_flatten_secret_setting '"nbp_x"' + assert_success + assert_output "nbp_x" +} + +@test "netbird_flatten_secret_setting keeps digit-only JSON strings" { + run netbird_flatten_secret_setting '"0123456789"' + assert_success + assert_output "0123456789" +} + +@test "netbird_flatten_secret_setting keeps boolean-looking JSON strings" { + run netbird_flatten_secret_setting '"true"' + assert_success + assert_output "true" +} + +@test "export_netbird_compose_env keeps a digit-only API token from user settings" { + printf '%s\n' '{"netbird_api_token":"0123456789"}' >"$HOME/.config/sandcat/settings.json" + unset NB_API_TOKEN + export_netbird_compose_env + [[ "$NB_API_TOKEN" == "0123456789" ]] +} + +@test "netbird_flatten_secret_setting unwraps op object" { + run netbird_flatten_secret_setting '{"op":"op://Vault/Item/credential"}' + assert_success + assert_output "op://Vault/Item/credential" +} + +@test "netbird_flatten_secret_setting unwraps pass object" { + run netbird_flatten_secret_setting '{"pass":"pass://Vault/Item/password"}' + assert_success + assert_output "pass://Vault/Item/password" +} + +@test "netbird_flatten_secret_setting unwraps value object" { + run netbird_flatten_secret_setting '{"value":"nbp_plain"}' + assert_success + assert_output "nbp_plain" +} + +@test "netbird_flatten_secret_setting fails when both value and op are set" { + run netbird_flatten_secret_setting '{"value":"k","op":"op://x"}' + assert_failure + assert_output --partial "exactly one of" +} + +@test "export_netbird_compose_env flattens object api token to NB_API_TOKEN" { + echo '{"netbird_api_token":{"op":"op://Vault/Item/credential"}}' > "$HOME/.config/sandcat/settings.json" + unset NB_API_TOKEN + export_netbird_compose_env + [[ "$NB_API_TOKEN" == "op://Vault/Item/credential" ]] +} + +@test "export_netbird_compose_env flattens object enrollment key to NB_SETUP_KEY" { + echo '{"netbird_enrollment_key":{"pass":"pass://V/I/password"}}' > "$HOME/.config/sandcat/settings.json" + unset NB_SETUP_KEY + export_netbird_compose_env + [[ "$NB_SETUP_KEY" == "pass://V/I/password" ]] +} + +@test "export_netbird_compose_env still exports literal string tokens" { + echo '{"netbird_api_token":"api-token-456"}' > "$HOME/.config/sandcat/settings.json" + unset NB_API_TOKEN + export_netbird_compose_env + [[ "$NB_API_TOKEN" == "api-token-456" ]] +} + +@test "export_netbird_compose_env does not invoke op" { + echo '{"netbird_api_token":{"op":"op://Vault/Item/credential"}}' > "$HOME/.config/sandcat/settings.json" + unset NB_API_TOKEN + # require.bash defines a yq() function, so `command -v yq` is the function + # name — use type -P for the real binary. Keep that dir on PATH while + # dropping any `op` so a mistaken resolve would fail loudly. + local yq_bin yq_dir + yq_bin=$(type -P yq) + [[ -n "$yq_bin" ]] + yq_dir=$(dirname "$yq_bin") + PATH="$yq_dir:/usr/bin:/bin" + export_netbird_compose_env + [[ "$NB_API_TOKEN" == "op://Vault/Item/credential" ]] +} + +@test "export_netbird_compose_env ignores a project-only API token" { + printf '%s\n' '{}' >"$HOME/.config/sandcat/settings.json" + printf '%s\n' '{"netbird_api_token":"project-token"}' >"$PROJECT_DIR/.sandcat/settings.json" + unset NB_API_TOKEN + export_netbird_compose_env + [[ -z "${NB_API_TOKEN:-}" ]] +} + +@test "prepare_agent_sandcat_mount strips NetBird secrets and keeps other keys" { + mkdir -p "$PROJECT_DIR/.sandcat" + printf '%s\n' '{"netbird_api_token":"tok","netbird_enrollment_key":"key","network":[{"action":"allow"}]}' \ + >"$PROJECT_DIR/.sandcat/settings.json" + printf '%s\n' '{"netbird_api_token":"local-tok","env":{"FOO":"bar"}}' \ + >"$PROJECT_DIR/.sandcat/settings.local.json" + local dest="$BATS_TEST_TMPDIR/filtered" + prepare_agent_sandcat_mount "$PROJECT_DIR/.sandcat" "$dest" + run yq -r '.netbird_api_token' "$dest/settings.json" + assert_output "null" + run yq -r '.netbird_enrollment_key' "$dest/settings.json" + assert_output "null" + run yq -r '.network[0].action' "$dest/settings.json" + assert_output "allow" + run yq -r '.netbird_api_token' "$dest/settings.local.json" + assert_output "null" + run yq -r '.env.FOO' "$dest/settings.local.json" + assert_output "bar" +} diff --git a/cli/test/netbird/netbird_settings.bats b/cli/test/netbird/netbird_settings.bats new file mode 100644 index 00000000..ac8208f3 --- /dev/null +++ b/cli/test/netbird/netbird_settings.bats @@ -0,0 +1,238 @@ +#!/usr/bin/env bats + +setup() { + load test_helper + source "$SCT_LIBDIR/netbird.bash" + + export HOME="$BATS_TEST_TMPDIR/home" + mkdir -p "$HOME/.config/sandcat" + PROJECT_DIR="$BATS_TEST_TMPDIR/project" + mkdir -p "$PROJECT_DIR/.sandcat" + cd "$PROJECT_DIR" || return 1 +} + +teardown() { + unstub_all +} + +# CI ships mikefarah yq in /usr/bin (often usr-merged with /bin), so +# PATH=/usr/bin:/bin still finds it — same trap as require.bats. Keep the +# utilities these tests need, omit yq. +path_without_yq() { + local fake_bin="$BATS_TEST_TMPDIR/no-yq-bin" + mkdir -p "$fake_bin" + local cmd + for cmd in bash sh mkdir rm cp mv cat grep mktemp chmod ln stat; do + if command -v "$cmd" >/dev/null 2>&1 && [[ ! -e "$fake_bin/$cmd" ]]; then + ln -s "$(command -v "$cmd")" "$fake_bin/$cmd" + fi + done + printf '%s' "$fake_bin" +} + +@test "netbird_read_setting returns empty when no settings exist" { + run netbird_read_setting netbird_api_token + assert_success + assert_output "" +} + +@test "netbird_read_setting does not require yq when the key is absent" { + echo '{"unrelated": true}' > "$HOME/.config/sandcat/settings.json" + PATH="$(path_without_yq)" run netbird_read_setting netbird_api_token + assert_success + assert_output "" +} + +@test "export_netbird_compose_env does not require yq when NetBird keys are absent" { + echo '{"unrelated": true}' > "$HOME/.config/sandcat/settings.json" + unset NB_SETUP_KEY NB_API_TOKEN + PATH="$(path_without_yq)" run export_netbird_compose_env + assert_success + refute_output --partial "yq required" +} + +@test "prepare_agent_sandcat_mount does not require yq when settings files are empty" { + : > "$PROJECT_DIR/.sandcat/settings.json" + local dest="$BATS_TEST_TMPDIR/agent-sandcat" + PATH="$(path_without_yq)" run prepare_agent_sandcat_mount "$PROJECT_DIR/.sandcat" "$dest" + assert_success + [[ -f "$dest/settings.json" ]] +} + +@test "prepare_agent_sandcat_mount fails closed when yq is missing and settings exist" { + echo '{"netbird_api_token":"tok"}' > "$PROJECT_DIR/.sandcat/settings.json" + local dest="$BATS_TEST_TMPDIR/agent-sandcat" + PATH="$(path_without_yq)" run prepare_agent_sandcat_mount "$PROJECT_DIR/.sandcat" "$dest" + assert_failure + refute_output --partial "tok" + [[ ! -e "$dest" ]] +} + +@test "export_agent_sandcat_mount does not export dest when prepare fails" { + echo '{"netbird_enrollment_key":"key"}' > "$PROJECT_DIR/.sandcat/settings.json" + local dest + dest="$(sct_home)/agent-sandcat/${PROJECT_DIR//\//_}" + PATH="$(path_without_yq)" run export_agent_sandcat_mount + assert_failure + [[ ! -e "$dest" ]] +} + +@test "export_agent_sandcat_mount writes SANDCAT_AGENT_SANDCAT into .devcontainer/.env" { + mkdir -p "$PROJECT_DIR/.devcontainer" + export_agent_sandcat_mount + [[ -n "${SANDCAT_AGENT_SANDCAT:-}" ]] + run grep -F "SANDCAT_AGENT_SANDCAT=$SANDCAT_AGENT_SANDCAT" "$PROJECT_DIR/.devcontainer/.env" + assert_success + local mode + mode=$(stat -c '%a' "$PROJECT_DIR/.devcontainer/.env" 2>/dev/null \ + || stat -f '%OLp' "$PROJECT_DIR/.devcontainer/.env") + assert_equal "$mode" "600" +} + +@test "netbird_read_setting reads netbird_api_token from user settings" { + echo '{"netbird_api_token": "user-token"}' > "$HOME/.config/sandcat/settings.json" + + run netbird_read_setting netbird_api_token + assert_output "user-token" +} + +@test "netbird_read_setting ignores project-layer API token" { + echo '{"netbird_api_token": "user-token"}' > "$HOME/.config/sandcat/settings.json" + echo '{"netbird_api_token": "project-token"}' > "$PROJECT_DIR/.sandcat/settings.json" + + run netbird_read_setting netbird_api_token + assert_output "user-token" +} + +@test "netbird_read_setting ignores local project API token" { + echo '{"netbird_api_token": "user-token"}' > "$HOME/.config/sandcat/settings.json" + echo '{"netbird_api_token": "project-token"}' > "$PROJECT_DIR/.sandcat/settings.json" + echo '{"netbird_api_token": "local-token"}' > "$PROJECT_DIR/.sandcat/settings.local.json" + + run netbird_read_setting netbird_api_token + assert_output "user-token" +} + +@test "netbird_read_setting still prefers project settings for management URL" { + echo '{"netbird_management_url": "https://user.example.com"}' > "$HOME/.config/sandcat/settings.json" + echo '{"netbird_management_url": "https://project.example.com"}' > "$PROJECT_DIR/.sandcat/settings.json" + + run netbird_read_setting netbird_management_url + assert_output "https://project.example.com" +} + +@test "export_netbird_compose_env exports enrollment key from user settings" { + echo '{"netbird_enrollment_key": "setup-key-123"}' > "$HOME/.config/sandcat/settings.json" + unset NB_SETUP_KEY + + export_netbird_compose_env + + [[ "$NB_SETUP_KEY" == "setup-key-123" ]] +} + +@test "export_netbird_compose_env does not override existing NB_SETUP_KEY" { + echo '{"netbird_enrollment_key": "from-settings"}' > "$HOME/.config/sandcat/settings.json" + export NB_SETUP_KEY="from-env" + + export_netbird_compose_env + + [[ "$NB_SETUP_KEY" == "from-env" ]] +} + +@test "export_netbird_compose_env exports API token from user settings" { + echo '{"netbird_api_token": "api-token-456"}' > "$HOME/.config/sandcat/settings.json" + unset NB_API_TOKEN + + export_netbird_compose_env + + [[ "$NB_API_TOKEN" == "api-token-456" ]] +} + +@test "export_netbird_compose_env does not override existing NB_API_TOKEN" { + echo '{"netbird_api_token": "from-settings"}' > "$HOME/.config/sandcat/settings.json" + export NB_API_TOKEN="from-env" + + export_netbird_compose_env + + [[ "$NB_API_TOKEN" == "from-env" ]] +} + +@test "export_netbird_management_url exports management URL from user settings" { + echo '{"netbird_management_url": "https://management.example.com"}' > "$HOME/.config/sandcat/settings.json" + unset NB_MANAGEMENT_URL + + export_netbird_management_url + + [[ "$NB_MANAGEMENT_URL" == "https://management.example.com" ]] +} + +@test "export_netbird_management_url does not override existing NB_MANAGEMENT_URL" { + echo '{"netbird_management_url": "https://from-settings.example.com"}' > "$HOME/.config/sandcat/settings.json" + export NB_MANAGEMENT_URL="https://from-env.example.com" + + export_netbird_management_url + + [[ "$NB_MANAGEMENT_URL" == "https://from-env.example.com" ]] +} + +@test "netbird_enrollment_management_url_from returns empty for localhost without explicit enrollment URL" { + run netbird_enrollment_management_url_from "http://localhost:33073" + assert_output "" +} + +@test "netbird_enrollment_management_url_from returns empty for 127.0.0.1 without explicit enrollment URL" { + run netbird_enrollment_management_url_from "http://127.0.0.1:33073" + assert_output "" +} + +@test "netbird_enrollment_management_url_from leaves remote URLs unchanged" { + run netbird_enrollment_management_url_from "https://netbird.example.com" + assert_output "https://netbird.example.com" +} + +@test "netbird_enrollment_management_url_from prefers netbird_enrollment_management_url setting" { + echo '{"netbird_enrollment_management_url": "http://192.168.5.2:33073"}' > "$HOME/.config/sandcat/settings.json" + + run netbird_enrollment_management_url_from "http://localhost:33073" + assert_output "http://192.168.5.2:33073" +} + +@test "netbird_detect_docker_host_ip extracts the host LAN address on Linux" { + stub uname "-s : echo Linux" + stub ip "-4 route get 1.1.1.1 : echo '1.1.1.1 via 192.168.1.1 dev eth0 src 192.168.1.50 uid 1000'" + + run netbird_detect_docker_host_ip + assert_success + assert_output "192.168.1.50" +} + +@test "netbird_detect_docker_host_ip prints nothing when no IPv4 is found" { + stub uname "-s : echo Linux" + stub ip "-4 route get 1.1.1.1 : echo ''" + + run netbird_detect_docker_host_ip + assert_success + assert_output "" +} + +@test "netbird_enrollment_url_uses_host_bypass for literal IPv4 enrollment URL" { + run netbird_enrollment_url_uses_host_bypass "http://192.168.5.2:33073" + assert_success +} + +@test "netbird_enrollment_url_uses_host_bypass is false for hostname enrollment URL" { + run netbird_enrollment_url_uses_host_bypass "https://netbird.example.com" + assert_failure +} + +@test "netbird_read_setting uses SANDCAT_PROJECT_ROOT instead of PWD" { + local other="$BATS_TEST_TMPDIR/other-project" + mkdir -p "$other/.sandcat" "$HOME/.config/sandcat" + echo '{"netbird_management_url": "https://pwd.example.com"}' > "$PROJECT_DIR/.sandcat/settings.json" + echo '{"netbird_management_url": "https://path.example.com"}' > "$other/.sandcat/settings.json" + echo '{}' > "$HOME/.config/sandcat/settings.json" + + cd "$PROJECT_DIR" + SANDCAT_PROJECT_ROOT="$other" run netbird_read_setting netbird_management_url + assert_output "https://path.example.com" +} diff --git a/cli/test/netbird/test_helper.bash b/cli/test/netbird/test_helper.bash new file mode 100644 index 00000000..69cd0113 --- /dev/null +++ b/cli/test/netbird/test_helper.bash @@ -0,0 +1,19 @@ +#!/bin/bash +bats_require_minimum_version 1.5.0 +if shopt -s compat32 2>/dev/null; then + export BASH_COMPAT=3.2 +fi +set -uo pipefail +export SHELLOPTS + +SCT_ROOT="$BATS_TEST_DIRNAME/../.." +BATS_LIB_PATH="$SCT_ROOT/support":${BATS_LIB_PATH-} + +bats_load_library bats-ext +bats_load_library bats-support +bats_load_library bats-assert +bats_load_library bats-mock-ext + +export SCT_ROOT SCT_LIBDIR="$SCT_ROOT/lib" +export SCT_LIBEXECDIR="$SCT_ROOT/libexec" +export SCT_TEMPLATEDIR="$SCT_ROOT/templates" diff --git a/cli/test/require/require.bats b/cli/test/require/require.bats new file mode 100644 index 00000000..18323d5d --- /dev/null +++ b/cli/test/require/require.bats @@ -0,0 +1,122 @@ +#!/usr/bin/env bats +# shellcheck disable=SC2030,SC2031 + +setup() { + load test_helper + + # Not BATS_MOCK_BINDIR ($BATS_TEST_TMPDIR/bin): unstub_all treats every + # file there as a bats-mock stub. + YQ_BIN="$BATS_TEST_TMPDIR/yq-bin" + mkdir -p "$YQ_BIN" + export YQ_LOG="$BATS_TEST_TMPDIR/yq.log" + : >"$YQ_LOG" + # Prepend so type -P yq finds the fixture, not a host binary. + export PATH="$YQ_BIN:$PATH" +} + +teardown() { + unstub_all +} + +write_yq() { + local body=$1 + # Fixture inherits exported nounset; never assume "$1" is set. + printf '%s\n' '#!/bin/bash' "$body" >"$YQ_BIN/yq" + chmod +x "$YQ_BIN/yq" +} + +@test "require yq probes PATH yq --version, not the shell wrapper" { + write_yq ' +echo "called $*" >> "${YQ_LOG:?}" +if [[ "${1-}" == "--version" ]]; then + echo "yq (https://github.com/mikefarah/yq/) version v4.44.3" + exit 0 +fi +exit 1 +' + # shellcheck source=../../lib/require.bash + source "$SCT_LIBDIR/require.bash" + + # The retry wrapper would report this as the version string and fail + # the mikefarah check. require must call `command yq`. + yq() { echo "python yq 3.0.0"; } + + require yq +} + +@test "require yq checks mikefarah version only once" { + write_yq ' +echo "called $*" >> "${YQ_LOG:?}" +if [[ "${1-}" == "--version" ]]; then + echo "yq (https://github.com/mikefarah/yq/) version v4.44.3" + exit 0 +fi +exit 1 +' + # shellcheck source=../../lib/require.bash + source "$SCT_LIBDIR/require.bash" + + require yq + require yq + require yq + + run grep -c . "$YQ_LOG" + assert_output "1" + run cat "$YQ_LOG" + assert_output "called --version" +} + +@test "require yq cache survives re-sourcing require.bash" { + write_yq ' +echo "called $*" >> "${YQ_LOG:?}" +if [[ "${1-}" == "--version" ]]; then + echo "yq (https://github.com/mikefarah/yq/) version v4.44.3" + exit 0 +fi +exit 1 +' + # shellcheck source=../../lib/require.bash + source "$SCT_LIBDIR/require.bash" + require yq + # shellcheck source=../../lib/require.bash + source "$SCT_LIBDIR/require.bash" + require yq + + run grep -c . "$YQ_LOG" + assert_output "1" +} + +@test "require yq does not cache a failed mikefarah check" { + write_yq ' +echo "called $*" >> "${YQ_LOG:?}" +echo "yq 3.0.0" +' + # shellcheck source=../../lib/require.bash + source "$SCT_LIBDIR/require.bash" + + status1=0 + require yq || status1=$? + status2=0 + require yq || status2=$? + assert_equal "$status1" "$exitcode_expectation_failed" + assert_equal "$status2" "$exitcode_expectation_failed" + + run grep -c . "$YQ_LOG" + assert_output "2" +} + +@test "require yq fails when no binary is on PATH" { + # shellcheck source=../../lib/require.bash + source "$SCT_LIBDIR/require.bash" + # CI installs mikefarah yq in /usr/bin (often usr-merged with /bin), so + # PATH=/usr/bin:/bin still finds it. Isolate to an empty dir. Prefix + # PATH on the require invocation only — bats `run` itself needs mktemp. + local empty_path="$BATS_TEST_TMPDIR/empty-path" + mkdir -p "$empty_path" + + status=0 + PATH="$empty_path" require yq 2>"$BATS_TEST_TMPDIR/require.err" || status=$? + assert_equal "$status" "$exitcode_expectation_failed" + run cat "$BATS_TEST_TMPDIR/require.err" + assert_output --partial "yq required" +} diff --git a/cli/test/require/test_helper.bash b/cli/test/require/test_helper.bash new file mode 100644 index 00000000..b2b9ad1a --- /dev/null +++ b/cli/test/require/test_helper.bash @@ -0,0 +1,20 @@ +#!/bin/bash +bats_require_minimum_version 1.5.0 + +# Enable Bash 3.2 compat mode when running on Bash 4.4+ +# On actual Bash 3.2 (macOS default), these options don't exist and aren't needed. +if shopt -s compat32 2>/dev/null; then + export BASH_COMPAT=3.2 +fi +set -uo pipefail +export SHELLOPTS + +SCT_ROOT="$BATS_TEST_DIRNAME/../.." +BATS_LIB_PATH="$SCT_ROOT/support":${BATS_LIB_PATH-} + +bats_load_library bats-ext +bats_load_library bats-support +bats_load_library bats-assert +bats_load_library bats-mock-ext + +export SCT_ROOT SCT_LIBDIR="$SCT_ROOT/lib" diff --git a/cli/test/restart/restart.bats b/cli/test/restart/restart.bats index 068c7ff7..fe22deed 100644 --- a/cli/test/restart/restart.bats +++ b/cli/test/restart/restart.bats @@ -57,3 +57,17 @@ teardown() { assert_success assert_output --partial "not running" } + +@test "restart exports management URL from settings when env is unset" { + export HOME="$BATS_TEST_TMPDIR/home" + mkdir -p "$HOME/.config/sandcat" + echo '{"netbird_management_url": "https://management.settings.example.com"}' > "$HOME/.config/sandcat/settings.json" + unset NB_MANAGEMENT_URL + + stub docker \ + "compose -f $COMPOSE_FILE ps mitmproxy --status running --quiet : :" + + cd "$BATS_TEST_TMPDIR" + restart + [[ "$NB_MANAGEMENT_URL" == "https://management.settings.example.com" ]] +} diff --git a/cli/test/run/run.bats b/cli/test/run/run.bats index 55097fb4..4876cb43 100755 --- a/cli/test/run/run.bats +++ b/cli/test/run/run.bats @@ -122,6 +122,21 @@ teardown() { refute_output --partial "rebuilt" } +@test "warning when image is much newer than volume with a numeric offset" { + if ! date -d "2024-01-15T10:00:00-07:00" +%s &>/dev/null; then + skip "requires GNU date with numeric offsets" + fi + + stub docker \ + "volume inspect myproject-sandbox_agent-home : :" \ + "volume inspect --format {{.CreatedAt}} myproject-sandbox_agent-home : echo '2024-01-15T10:00:00-07:00'" \ + "image inspect --format {{.Created}} myproject-sandbox-agent : echo '2024-06-20T14:30:00.123456789-07:00'" + + run --separate-stderr warn_stale_home_volume "$COMPOSE_FILE" + assert_success + assert_stderr --partial "agent image was rebuilt" +} + @test "no warning when compose file has no project name" { cat > "$COMPOSE_FILE" <<-'EOF' services: @@ -134,3 +149,15 @@ teardown() { assert_success refute_output --partial "volume" } + +@test "_volume_timestamp_epoch parses a trailing Z" { + run _volume_timestamp_epoch "2024-01-15T10:00:00Z" + assert_success + assert_output "1705312800" +} + +@test "_volume_timestamp_epoch parses fractional seconds and a numeric offset" { + run _volume_timestamp_epoch "2024-06-20T14:30:00.123456789-07:00" + assert_success + assert_output "1718919000" +} diff --git a/cli/test/wg-client/netbird_dns.bats b/cli/test/wg-client/netbird_dns.bats new file mode 100644 index 00000000..de16d056 --- /dev/null +++ b/cli/test/wg-client/netbird_dns.bats @@ -0,0 +1,182 @@ +#!/usr/bin/env bats +# Tests for volume-based NetBird DNS bridge in wg-client-init.sh. +# +# The old netbird_dns_nameserver_ip and patch_dnsmasq_for_netbird functions +# (which called `netbird status` directly on wg-client) have been replaced by +# patch_dnsmasq_from_netbird_volume, which reads records published by +# mitmproxy-init.sh into the shared mitmproxy-config volume. + +setup() { + load test_helper + DNSMASQ_CONF="$BATS_TEST_TMPDIR/dnsmasq.conf" + NETBIRD_PEERS_CONF="$BATS_TEST_TMPDIR/netbird-peers.conf" + DNSMASQ_PID_FILE="$BATS_TEST_TMPDIR/dnsmasq.pid" + export NETBIRD_PEERS_CONF DNSMASQ_PID_FILE + printf '# existing config\nserver=1.1.1.1\n' > "$DNSMASQ_CONF" + + mkdir -p "$BATS_TEST_TMPDIR/bin" + printf '%s\n' '#!/bin/sh' 'exit 0' > "$BATS_TEST_TMPDIR/bin/dnsmasq" + printf '%s\n' '#!/bin/sh' 'exit 0' > "$BATS_TEST_TMPDIR/bin/dnsmasq-ready" + chmod +x "$BATS_TEST_TMPDIR/bin/dnsmasq" "$BATS_TEST_TMPDIR/bin/dnsmasq-ready" + export PATH="$BATS_TEST_TMPDIR/bin:$PATH" +} + +teardown() { + unstub_all +} + +# ── patch_dnsmasq_from_netbird_volume ──────────────────────────────────────── + +@test "patch_dnsmasq_from_netbird_volume is a no-op when peers file does not exist" { + rm -f "$NETBIRD_PEERS_CONF" + local before + before=$(cat "$DNSMASQ_CONF") + + run patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" + assert_success + + assert_equal "$(cat "$DNSMASQ_CONF")" "$before" +} + +@test "patch_dnsmasq_from_netbird_volume appends address= records from volume" { + printf 'address=/test-proxy-peer.netbird.selfhosted/100.64.0.5\n' > "$NETBIRD_PEERS_CONF" + + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" + + run grep -c "address=/test-proxy-peer.netbird.selfhosted/100.64.0.5" "$DNSMASQ_CONF" + assert_output "1" +} + +@test "patch_dnsmasq_from_netbird_volume appends server= forward lines from volume" { + printf 'server=/netbird.selfhosted/100.64.0.1\n' > "$NETBIRD_PEERS_CONF" + + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" + + run grep -c "server=/netbird.selfhosted/100.64.0.1" "$DNSMASQ_CONF" + assert_output "1" +} + +@test "patch_dnsmasq_from_netbird_volume is idempotent — does not duplicate records" { + printf 'address=/test-proxy-peer.netbird.selfhosted/100.64.0.5\n' > "$NETBIRD_PEERS_CONF" + + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" + + run grep -c "address=/test-proxy-peer.netbird.selfhosted/100.64.0.5" "$DNSMASQ_CONF" + assert_output "1" +} + +@test "patch_dnsmasq_from_netbird_volume replaces stale host-record IPs" { + # Recreating proxy-peer assigns a new 100.79.x; append-only left the old + # record first, so the agent connected to a dead mesh IP (EHOSTUNREACH/502) + # while curl inside mitmproxy used live NetBird DNS and succeeded. + printf 'host-record=proxy-peer.netbird.selfhosted,100.64.0.5\n' > "$NETBIRD_PEERS_CONF" + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" + printf 'host-record=proxy-peer.netbird.selfhosted,100.64.0.9\n' > "$NETBIRD_PEERS_CONF" + rm -f "$DNSMASQ_CONF.netbird-peers.stamp" + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" + + run grep -c "100.64.0.5" "$DNSMASQ_CONF" + assert_output "0" + run grep -c "host-record=proxy-peer.netbird.selfhosted,100.64.0.9" "$DNSMASQ_CONF" + assert_output "1" +} + +@test "patch_dnsmasq_from_netbird_volume replaces stale address= IPs" { + printf 'address=/proxy-peer.netbird.selfhosted/100.64.0.5\n' > "$NETBIRD_PEERS_CONF" + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" + printf 'address=/proxy-peer.netbird.selfhosted/100.64.0.9\n' > "$NETBIRD_PEERS_CONF" + rm -f "$DNSMASQ_CONF.netbird-peers.stamp" + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" + + run grep -c "address=/proxy-peer.netbird.selfhosted/100.64.0.5" "$DNSMASQ_CONF" + assert_output "0" + run grep -c "address=/proxy-peer.netbird.selfhosted/100.64.0.9" "$DNSMASQ_CONF" + assert_output "1" +} + +@test "patch_dnsmasq_from_netbird_volume skips lines that are not dnsmasq directives" { + printf 'host-record=peer.netbird.selfhosted,100.64.0.5\nsome-random-line\n# comment\n' > "$NETBIRD_PEERS_CONF" + + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" + + run grep -c "some-random-line" "$DNSMASQ_CONF" + assert_output "0" + + run grep -c "# comment" "$DNSMASQ_CONF" + assert_output "0" +} + +@test "patch_dnsmasq_from_netbird_volume handles multiple records" { + { + printf 'server=/netbird.selfhosted/100.64.0.1\n' + printf 'address=/peer-a.netbird.selfhosted/100.64.0.5\n' + printf 'address=/peer-b.netbird.selfhosted/100.64.0.6\n' + } > "$NETBIRD_PEERS_CONF" + + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" + + run grep -c "address=" "$DNSMASQ_CONF" + assert_output "2" + + run grep -c "server=/netbird.selfhosted" "$DNSMASQ_CONF" + assert_output "1" +} + +@test "patch_dnsmasq_from_netbird_volume does not start dnsmasq before it is listening" { + printf 'address=/test-proxy-peer.netbird.selfhosted/100.64.0.5\n' > "$NETBIRD_PEERS_CONF" + local restarts="$BATS_TEST_TMPDIR/dnsmasq.restarts" + : >"$restarts" + dnsmasq-ready() { return 1; } + restart_dnsmasq() { echo restart >>"$restarts"; } + + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" + + run grep -c "address=/test-proxy-peer.netbird.selfhosted/100.64.0.5" "$DNSMASQ_CONF" + assert_output "1" + assert_equal "$(cat "$restarts")" "" +} + +@test "wg-client-init restarts dnsmasq when NetBird peers volume changes" { + run grep -F 'peers_mtime' "$WG_CLIENT_INIT" + assert_success + run grep -F 'restart_dnsmasq "$conf"' "$WG_CLIENT_INIT" + assert_success +} + +@test "wg-client-init restarts dnsmasq instead of SIGHUP reload" { + run grep -F 'restart_dnsmasq' "$WG_CLIENT_INIT" + assert_success + run grep -F 'reload_dnsmasq_sighup' "$WG_CLIENT_INIT" + assert_failure + run grep -F 'pgrep -x dnsmasq' "$WG_CLIENT_INIT" + assert_failure + run grep -F -- '--pid-file="$DNSMASQ_PID_FILE"' "$WG_CLIENT_INIT" + assert_success +} + +@test "patch_dnsmasq_from_netbird_volume skips merge when peers mtime is unchanged" { + printf 'address=/test-proxy-peer.netbird.selfhosted/100.64.0.5\n' > "$NETBIRD_PEERS_CONF" + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" + local before + before=$(cat "$DNSMASQ_CONF") + printf '# existing config\nserver=1.1.1.1\n' > "$DNSMASQ_CONF" + + patch_dnsmasq_from_netbird_volume "$DNSMASQ_CONF" + + assert_equal "$(cat "$DNSMASQ_CONF")" "# existing config +server=1.1.1.1" + [[ "$before" != "$(cat "$DNSMASQ_CONF")" ]] +} + +@test "restart_dnsmasq does not start a second process if the old one is still listening" { + rm -f "$DNSMASQ_PID_FILE" + dnsmasq-ready() { return 0; } + local started="$BATS_TEST_TMPDIR/dnsmasq.started" + : >"$started" + dnsmasq() { echo started >>"$started"; } + + run restart_dnsmasq "$DNSMASQ_CONF" + assert_success + assert_equal "$(cat "$started")" "" +} diff --git a/cli/yq b/cli/yq new file mode 100755 index 00000000..90ce623c Binary files /dev/null and b/cli/yq differ diff --git a/docs/conf.py b/docs/conf.py index 24660acb..7b5f9c4d 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -74,6 +74,8 @@ "**/site-packages/**", "**/node_modules/**", "_templates", + "examples/**", + "superpowers/**", "requirements.txt", "README.md", ] diff --git a/docs/examples/netbird-server/README.md b/docs/examples/netbird-server/README.md new file mode 100644 index 00000000..1bb47ddd --- /dev/null +++ b/docs/examples/netbird-server/README.md @@ -0,0 +1,182 @@ +# Self-hosted NetBird management server + +Sandcat does not create or start a NetBird management server. Run this +compose stack yourself (or use NetBird's official installer), then point +sandcat at the URL. + +This layout matches NetBird's +[getting-started.sh](https://docs.netbird.io/selfhosted/selfhosted-quickstart#installation-script) +exposed-ports mode, tuned for localhost. + +For a VM with a public domain, prefer the official installer instead of this +compose file: + +```bash +curl -fsSL https://github.com/netbirdio/netbird/releases/latest/download/getting-started.sh | bash +``` + +Docs: https://docs.netbird.io/selfhosted/selfhosted-quickstart#installation-script + +## 1. Start the stack + +You cannot omit `server.authSecret`: combined 0.72 fails with +`authSecret is required when running local relay`. `store.encryptionKey` is +not a “off” switch either — leave it empty and recreates can mint a new key +and make existing setup keys / PATs unreadable. Generate both on the fly and +keep them: + +```bash +cd docs/examples/netbird-server +chmod +x start.sh +./start.sh --secrets-from "$HOME/.config/sandcat/netbird-server/config.yaml" +``` + +That copies `authSecret` and `encryptionKey` into gitignored `config.local.yaml` at +start time and does **not** write them to `netbird-server.env`. Recreate: + +```bash +./start.sh --secrets-from "$HOME/.config/sandcat/netbird-server/config.yaml" \ + --force-recreate netbird-server +``` + +Or `NETBIRD_SECRETS_CONFIG=/path/to/config.yaml ./start.sh`. + +Without `--secrets-from`, `start.sh` still generates empty keys into +`netbird-server.env` (needed if you have no existing config). `--prepare-only` +stops after writing `config.local.yaml` (no Docker). + +Edit `exposedAddress` in tracked `config.yaml`; `start.sh` copies it into +`config.local.yaml` on each run. Combined `netbird-server` advertises that +host for management and signal (TCP). `localhost` is fine for host tools +and the dashboard; mitmproxy inside Docker cannot use it. + +Do **not** put sandcat or proxy-peer on the netbird-server compose network. + +On Colima, `host-gateway` / `192.168.5.2` work for TCP but UDP 3478 hairpins. +Keep embedded STUN (`stunPorts` only — do **not** set `server.stuns`, that +stops the listener). Map the advertised name to docker0 in peer compose: + +```yaml +# config.yaml — embedded STUN stays on +server: + exposedAddress: "http://host.docker.internal:33073" + stunPorts: [3478] + +# mitmproxy / proxy-peer +extra_hosts: + - "host.docker.internal:172.17.0.1" +``` + +`netbird_enrollment_management_url` may stay on the LAN IP (TCP still works). +`exposedAddress` is what peers use for STUN; it does not have to match the +enrollment URL, but it must not be `localhost`. + +## 2. Verify the API + +```bash +curl -s http://localhost:33073/api/instance +``` + +Expect `"setup_required": true` before bootstrap. Use **http** (not https) and +the **/api/** prefix. + +## 3. Bootstrap the first admin + +```bash +curl -fsS -X POST "http://localhost:33073/api/setup" \ + -H "Content-Type: application/json" \ + -d '{ + "email": "admin@example.com", + "name": "Admin", + "password": "choose-a-long-random-password", + "create_pat": true, + "pat_expire_in": 7 + }' +``` + +Save the returned `personal_access_token` for `netbird_api_token` in +`~/.config/sandcat/settings.json`. See +[NetBird automated setup](https://docs.netbird.io/selfhosted/automated-setup). + +## 4. Open the dashboard + +```text +http://localhost:8080 +``` + +Or the setup wizard: `http://localhost:8080/setup` (before step 4). Create a +**Setup Key** for peer enrollment. + +## 5. Point sandcat at the server + +```bash +sandcat init --agent cursor --ide vscode --netbird \ + --netbird-management-url http://localhost:33073 --name myproject +``` + +Host tools and the browser use `localhost`. mitmproxy needs the Docker host IP +in `netbird_enrollment_management_url`: + +```json +"netbird_management_url": "http://localhost:33073", +"netbird_enrollment_management_url": "http://192.168.5.2:33073", +"netbird_enrollment_key": "", +"netbird_api_token": "" +``` + +Replace `192.168.5.2` with your Docker host IP (`colima status -j` on Colima) +for **enrollment TCP**. Set `server.exposedAddress` to +`http://host.docker.internal:33073` so STUN uses the same published ports. +If `exposedAddress` is `localhost`, peers dial `localhost` / `[::1]` inside +the container. Recreate the server after changing it: + +```bash +cd docs/examples/netbird-server +./start.sh --secrets-from "$HOME/.config/sandcat/netbird-server/config.yaml" \ + --force-recreate netbird-server +``` + +Then recreate mitmproxy (`sandcat run --force-recreate mitmproxy`). + +## Troubleshooting + +### Peers dial `[::1]:33073` after enrollment + +The management server is still advertising `http://localhost:33073` in +`config.yaml` `server.exposedAddress`. Set it to +`http://host.docker.internal:33073`, recreate netbird-server, then recreate +mitmproxy (needs `extra_hosts: ["host.docker.internal:172.17.0.1"]`). + +### STUN `context deadline exceeded` / peers stuck `Connecting` + +`server.stuns` in config.yaml **disables** the embedded UDP 3478 listener — +every STUN probe then times out, including `172.17.0.1`. Leave `stunPorts` +only. On Colima, `host.docker.internal` via `host-gateway` is `192.168.5.2` +(TCP ok, UDP hairpin). Set peer `extra_hosts` to +`host.docker.internal:172.17.0.1`, recreate server **without** `stuns:`, then +recreate mitmproxy and proxy-peer. Probe from mitmproxy: resolve +`host.docker.internal` to `172.17.0.1` and STUN that IP. + +### Port clash + +Default ports **8080** (dashboard) and **33073** (management API) may already +be in use. Change `NETBIRD_DASHBOARD_HTTP_PORT` and `NETBIRD_MGMT_API_PORT` in +`netbird-server.env`, then update `config.yaml` and `dashboard.env` URLs to +match. + +### `curl: (52) Empty reply` or SSL errors on port 33073 + +- Use **`http://localhost:33073/api/...`** — not `https://` and not `/setup` + (dashboard route; API is `/api/setup`). +- Ensure compose maps **`33073:80`** to `netbird-server`, not `33073:443`. +- Recreate after config changes: `./start.sh --force-recreate netbird-server` + +### `/api/setup` returns "not authenticated" + +Embedded IdP must be enabled in `config.yaml` (`server.auth.issuer`). If +needed, reset data: + +```bash +docker compose --env-file netbird-server.env down -v +./start.sh +``` diff --git a/docs/examples/netbird-server/config.yaml b/docs/examples/netbird-server/config.yaml new file mode 100644 index 00000000..ffe48371 --- /dev/null +++ b/docs/examples/netbird-server/config.yaml @@ -0,0 +1,30 @@ +# Combined NetBird server — localhost defaults. Fill secrets via start.sh. +# +# Do NOT set server.stuns: that disables the embedded STUN listener (UDP 3478). +# stunPorts keeps STUN in-process; advertised host comes from exposedAddress. +# On Colima, extra_hosts maps host.docker.internal → 172.17.0.1 (docker0) so +# UDP reaches the published port (host-gateway is 192.168.5.2 and hairpins). +server: + listenAddress: ":80" + # customize and parametrize for local development + exposedAddress: "http://host.docker.internal:33073" + stunPorts: + - 3478 + metricsPort: 9090 + healthcheckAddress: ":9000" + logLevel: info + logFile: console + disableAnonymousMetrics: true + authSecret: "" + dataDir: /var/lib/netbird + auth: + issuer: http://localhost:33073/oauth2 + signKeyRefreshEnabled: true + dashboardRedirectURIs: + - http://localhost:8080/nb-auth + - http://localhost:8080/nb-silent-auth + cliRedirectURIs: + - http://localhost:53000/ + store: + engine: sqlite + encryptionKey: "" diff --git a/docs/examples/netbird-server/dashboard.env b/docs/examples/netbird-server/dashboard.env new file mode 100644 index 00000000..82212b8c --- /dev/null +++ b/docs/examples/netbird-server/dashboard.env @@ -0,0 +1,14 @@ +# Dashboard OIDC — API on netbird-server (host localhost:33073). +NETBIRD_MGMT_API_ENDPOINT=http://localhost:33073 +NETBIRD_MGMT_GRPC_API_ENDPOINT=http://localhost:33073 +AUTH_AUDIENCE=netbird-dashboard +AUTH_CLIENT_ID=netbird-dashboard +AUTH_CLIENT_SECRET= +AUTH_AUTHORITY=http://localhost:33073/oauth2 +USE_AUTH0=false +AUTH_SUPPORTED_SCOPES=openid profile email groups +AUTH_REDIRECT_URI=/nb-auth +AUTH_SILENT_REDIRECT_URI=/nb-silent-auth +NETBIRD_TOKEN_SOURCE=idToken +NGINX_SSL_PORT=443 +LETSENCRYPT_DOMAIN=none diff --git a/docs/examples/netbird-server/docker-compose.yml b/docs/examples/netbird-server/docker-compose.yml new file mode 100644 index 00000000..9d56a4a3 --- /dev/null +++ b/docs/examples/netbird-server/docker-compose.yml @@ -0,0 +1,39 @@ +# Local NetBird stack: dashboard + combined netbird-server (management, signal, +# relay, STUN). Matches NetBird getting-started exposed-ports mode. + +services: + dashboard: + image: netbirdio/dashboard:${NETBIRD_DASHBOARD_VERSION} + restart: unless-stopped + ports: + - ${NETBIRD_DASHBOARD_HTTP_PORT:-8080}:80 + env_file: + - dashboard.env + logging: + driver: json-file + options: + max-size: 500m + max-file: "2" + + netbird-server: + image: netbirdio/netbird-server:${NETBIRD_SERVER_VERSION} + restart: unless-stopped + ports: + - ${NETBIRD_MGMT_API_PORT}:80 + - ${NETBIRD_STUN_PORT}:3478/udp + volumes: + - netbird_data:/var/lib/netbird + # Produced by ./start.sh (secrets filled). Do not mount the tracked + # config.yaml placeholders — empty authSecret fails 0.72 local relay. + - ./config.local.yaml:/etc/netbird/config.yaml + command: ["--config", "/etc/netbird/config.yaml"] + environment: + - NB_SETUP_PAT_ENABLED=true + logging: + driver: json-file + options: + max-size: 500m + max-file: "2" + +volumes: + netbird_data: diff --git a/docs/examples/netbird-server/netbird-server.env b/docs/examples/netbird-server/netbird-server.env new file mode 100644 index 00000000..e3292f4e --- /dev/null +++ b/docs/examples/netbird-server/netbird-server.env @@ -0,0 +1,12 @@ +# Compose variable substitution for docs/examples/netbird-server/docker-compose.yml + +NETBIRD_DASHBOARD_VERSION=v2.39.0 +NETBIRD_SERVER_VERSION=0.72.4 + +NETBIRD_DASHBOARD_HTTP_PORT=8080 +NETBIRD_MGMT_API_PORT=33073 +NETBIRD_STUN_PORT=3478 + +# Generate with: openssl rand -base64 32 (keep stable across restarts) +NETBIRD_ENCRYPTION_KEY= +NETBIRD_RELAY_AUTH_SECRET= diff --git a/docs/examples/netbird-server/start.sh b/docs/examples/netbird-server/start.sh new file mode 100755 index 00000000..0f15777b --- /dev/null +++ b/docs/examples/netbird-server/start.sh @@ -0,0 +1,145 @@ +#!/usr/bin/env bash +# Prepare secrets (if missing) and start the example NetBird stack. +# +# Combined netbird-server 0.72 requires a non-empty server.authSecret for the +# embedded relay. store.encryptionKey should stay stable across recreates or +# encrypted setup keys / PATs in the volume become unreadable. +# +# Usage: +# ./start.sh +# Generate secrets into netbird-server.env if empty, write +# config.local.yaml, compose up -d. +# ./start.sh --secrets-from ~/.config/sandcat/netbird-server/config.yaml +# Copy authSecret + encryptionKey from that YAML into config.local.yaml +# only (does not write them to netbird-server.env). +# ./start.sh --force-recreate netbird-server +# ./start.sh --prepare-only +# Secrets + config.local.yaml only (no docker). +set -euo pipefail + +cd "$(dirname "$0")" + +PREPARE_ONLY=0 +SECRETS_FROM="${NETBIRD_SECRETS_CONFIG:-}" +COMPOSE_ARGS=() +while [[ $# -gt 0 ]]; do + case "$1" in + --prepare-only) + PREPARE_ONLY=1 + shift + ;; + --secrets-from) + SECRETS_FROM="${2:?--secrets-from requires a path}" + shift 2 + ;; + --secrets-from=*) + SECRETS_FROM="${1#*=}" + shift + ;; + *) + COMPOSE_ARGS+=("$1") + shift + ;; + esac +done +export NETBIRD_SECRETS_CONFIG="${SECRETS_FROM}" + +python3 - <<'PY' +import os, pathlib, re, secrets, base64, sys + +root = pathlib.Path(".") +env_path = root / "netbird-server.env" +cfg_path = root / "config.yaml" +local_path = root / "config.local.yaml" +secrets_from = os.environ.get("NETBIRD_SECRETS_CONFIG", "").strip() + +env_text = env_path.read_text() +cfg_text = cfg_path.read_text() + + +def env_value(text: str, key: str) -> str: + m = re.search(rf"^{re.escape(key)}=(.*)$", text, flags=re.M) + if not m: + return "" + return m.group(1).strip() + + +def yaml_scalar(text: str, key: str) -> str: + m = re.search(rf"^[ \t]*{re.escape(key)}:[ \t]*(.*)$", text, flags=re.M) + if not m: + return "" + raw = m.group(1).strip() + if len(raw) >= 2 and raw[0] == raw[-1] and raw[0] in "\"'": + return raw[1:-1].strip() + return raw + + +def new_secret() -> str: + return base64.b64encode(secrets.token_bytes(32)).decode() + + +mapping = ( + ("NETBIRD_RELAY_AUTH_SECRET", "authSecret"), + ("NETBIRD_ENCRYPTION_KEY", "encryptionKey"), +) + +values = {} +if secrets_from: + src = pathlib.Path(secrets_from).expanduser() + if not src.is_file(): + print(f"start.sh: --secrets-from file not found: {src}", file=sys.stderr) + sys.exit(1) + src_text = src.read_text() + for env_key, yaml_key in mapping: + value = yaml_scalar(src_text, yaml_key) + if not value: + print(f"start.sh: {yaml_key} missing in {src}", file=sys.stderr) + sys.exit(1) + values[env_key] = value + print(f"loaded secrets from {src} (not written to netbird-server.env)") +else: + generated = [] + local_existing = local_path.read_text() if local_path.is_file() else "" + for env_key, yaml_key in mapping: + value = ( + yaml_scalar(local_existing, yaml_key) + or env_value(env_text, env_key) + or yaml_scalar(cfg_text, yaml_key) + ) + if not value: + value = new_secret() + generated.append(env_key) + values[env_key] = value + if generated: + print("generated: " + ", ".join(generated) + " (kept in config.local.yaml)") + else: + print("reusing secrets from config.local.yaml or netbird-server.env") + +local = cfg_text +for env_key, yaml_key in mapping: + local = re.sub( + rf"^([ \t]*{re.escape(yaml_key)}:[ \t]*).*$", + rf'\1"{values[env_key]}"', + local, + count=1, + flags=re.M, + ) +os.umask(0o077) +local_path.write_text(local) +try: + os.chmod(local_path, 0o600) +except OSError: + pass +print(f"wrote {local_path}") +PY + +if [[ "$PREPARE_ONLY" -eq 1 ]]; then + exit 0 +fi + +if [[ ! -f config.local.yaml ]]; then + echo "start.sh: config.local.yaml was not written" >&2 + exit 1 +fi + +exec docker compose --env-file netbird-server.env up -d ${COMPOSE_ARGS[@]+"${COMPOSE_ARGS[@]}"} diff --git a/docs/examples/proxy-peer/.env.example b/docs/examples/proxy-peer/.env.example new file mode 100644 index 00000000..03e0412c --- /dev/null +++ b/docs/examples/proxy-peer/.env.example @@ -0,0 +1,5 @@ +# Copy to .env and fill. Do not commit .env. +NB_SETUP_KEY= +NB_MANAGEMENT_URL= +NB_API_TOKEN= +NB_PEER_NAME=proxy-peer diff --git a/docs/examples/proxy-peer/Dockerfile.proxy-peer b/docs/examples/proxy-peer/Dockerfile.proxy-peer new file mode 100644 index 00000000..e192910e --- /dev/null +++ b/docs/examples/proxy-peer/Dockerfile.proxy-peer @@ -0,0 +1,42 @@ +FROM debian:trixie-slim + +# iproute2 and iptables are required by proxy-peer-init.sh for host-bound +# NetBird management routing (self-hosted NB_MANAGEMENT_URL on the Docker host). +RUN apt-get update \ + && apt-get install -y --no-install-recommends \ + ca-certificates curl python3 iproute2 iptables jq \ + && rm -rf /var/lib/apt/lists/* + +# Install the NetBird client daemon. The daemon manages wt0 (the NetBird +# overlay mesh) inside this container, which already holds NET_ADMIN. +# +# Version + per-arch checksums are the single source of truth in netbird.env +# (sibling to this Dockerfile) and MUST be supplied as build args. No defaults +# here on purpose, so the pin lives in exactly one place. +ARG NETBIRD_VERSION +ARG NETBIRD_SHA256_AMD64 +ARG NETBIRD_SHA256_ARM64 +RUN test -n "$NETBIRD_VERSION" || { echo "NETBIRD_VERSION build arg is required (source netbird.env)" >&2; exit 1; } \ + && ARCH=$(dpkg --print-architecture) \ + && case "$ARCH" in \ + amd64) NETBIRD_SHA256="$NETBIRD_SHA256_AMD64" ;; \ + arm64) NETBIRD_SHA256="$NETBIRD_SHA256_ARM64" ;; \ + *) echo "Unsupported architecture: $ARCH" >&2; exit 1 ;; \ + esac \ + && test -n "$NETBIRD_SHA256" \ + && curl -sSLf -o /tmp/netbird.tar.gz \ + "https://github.com/netbirdio/netbird/releases/download/v${NETBIRD_VERSION}/netbird_${NETBIRD_VERSION}_linux_${ARCH}.tar.gz" \ + && echo "${NETBIRD_SHA256} /tmp/netbird.tar.gz" | sha256sum -c - \ + && tar xzf /tmp/netbird.tar.gz -C /usr/local/bin netbird \ + && chmod +x /usr/local/bin/netbird \ + && rm /tmp/netbird.tar.gz \ + && netbird version + +# Canonical enroll helpers live in the CLI mitmproxy template; this example +# does not keep a second copy. Compose named context `sandcat-scripts`. +COPY --from=sandcat-scripts netbird-peer-lifecycle.sh /usr/local/lib/netbird-peer-lifecycle.sh +COPY scripts/proxy-peer-init.sh /usr/local/bin/proxy-peer-init.sh +COPY scripts/proxy-peer-hello.py /usr/local/bin/proxy-peer-hello.py +RUN chmod +x /usr/local/bin/proxy-peer-init.sh + +ENTRYPOINT ["/usr/local/bin/proxy-peer-init.sh"] diff --git a/docs/examples/proxy-peer/README.md b/docs/examples/proxy-peer/README.md new file mode 100644 index 00000000..66142906 --- /dev/null +++ b/docs/examples/proxy-peer/README.md @@ -0,0 +1,107 @@ +# Proxy-peer gateway (manual) + +Sandcat does not create this container. It is a NetBird-enrolled HTTP service +(`GET /hello` on port 8080) you run with Docker Compose beside a project +initialized with `sandcat init --netbird`. + +## Prerequisites + +- A running NetBird management server ([../netbird-server](../netbird-server/)) +- `sandcat init --netbird` already done; mitmproxy enrolled +- Setup key and API token (literals in `.env` for this example) + +## Data path + +```mermaid +flowchart LR + agent --> wg0 + wg0 --> mitmproxy + mitmproxy --> wt0 + wt0 --> proxyPeer["proxy-peer :8080"] +``` + +Agent traffic is Layer 1–filtered in mitmproxy, then forwarded on `wt0` to the gateway. + +## Policy path + +```mermaid +sequenceDiagram + participant Op as Operator + participant Dash as NetBird dashboard + participant Mitm as mitmproxy peer + participant Gw as proxy-peer + Op->>Dash: Enroll both peers (setup key) + Op->>Dash: Groups sandcat-proxy and proxy-peer + Op->>Dash: ACL sandcat-proxy to proxy-peer TCP 8080 + Op->>Dash: Delete Default All-All policy + Op->>Mitm: Layer 1 allow FQDN (settings + restart-proxy) + Mitm->>Gw: GET /hello +``` + +## 1. Start the gateway + +```bash +cp .env.example .env +# Fill NB_SETUP_KEY, NB_MANAGEMENT_URL, NB_API_TOKEN +docker compose --env-file netbird.env --env-file .env \ + -f compose-proxy-peer.yml up -d --build +``` + +Build from a full repo checkout: the image copies +`cli/templates/devcontainer/sandcat/scripts/netbird-peer-lifecycle.sh` +via the compose `sandcat-scripts` additional context. A copy of only +`docs/examples/proxy-peer/` cannot build. + +`--env-file netbird.env` is required so the image gets client **0.72.4**. Up +without `--build` keeps a leftover 0.28.9 binary: Relays stay `stun:`-only and +peers stick on Connecting (no `rel://`, no WireGuard handshake). Confirm both +peers with `netbird status --json`: `daemonVersion` 0.72.4 and a `rel://` +relay. Mitmproxy being 0.72.4 is not enough. + +Default `NB_PEER_NAME` is `proxy-peer`. + +## 2. Confirm enrollment + +Open the NetBird dashboard **Peers** page. You should see `proxy-peer` and the +mitmproxy peer (`{project}-proxy`). + +## 3. Layer 1 allow rule + +Merge [settings-proxy-peer.json](settings-proxy-peer.json) into the project's +`.sandcat/settings.json`. Reload: `sandcat restart-proxy`. + +If the dashboard FQDN still has an IP suffix, put that FQDN in the Layer 1 +`host` field instead of `proxy-peer.netbird.selfhosted`. + +## 4. Access control + +NetBird accounts start with a Default policy (`All` ↔ `All`, any protocol). +New policies do nothing until you delete it. + +1. **Access Control → Groups:** create `sandcat-proxy` and `proxy-peer`. +2. **Peers:** assign the mitmproxy peer to `sandcat-proxy`, the gateway to `proxy-peer`. +3. **Access Control → Policies:** add unidirectional TCP **8080**, + source `sandcat-proxy`, destination `proxy-peer`. +4. Delete the Default `All` ↔ `All` policy. + +Do not create a Network or a legacy Route for this hello service. Those are for +CIDRs behind a routing peer, not for a process listening on the peer itself. + +## 5. Smoke + +```bash +sandcat run curl -sS http://proxy-peer.netbird.selfhosted:8080/hello +``` + +Expect `{"service": "proxy-peer", "ok": true}`. + +## Layout + +| File | Role | +|------|------| +| `Dockerfile.proxy-peer` | Debian image with pinned NetBird client | +| `compose-proxy-peer.yml` | Standalone compose service | +| `netbird.env` | Client version + checksums | +| `.env.example` | Literal secret placeholders | +| `scripts/proxy-peer-init.sh` | Enroll + hello server | +| `settings-proxy-peer.json` | Layer 1 allow-list example | diff --git a/docs/examples/proxy-peer/compose-proxy-peer.yml b/docs/examples/proxy-peer/compose-proxy-peer.yml new file mode 100644 index 00000000..025cae57 --- /dev/null +++ b/docs/examples/proxy-peer/compose-proxy-peer.yml @@ -0,0 +1,38 @@ +# Standalone proxy-peer gateway. Not created by sandcat. +# +# cp .env.example .env # fill literals +# docker compose --env-file netbird.env --env-file .env -f compose-proxy-peer.yml up -d --build +# +# Required env: NB_SETUP_KEY, NB_MANAGEMENT_URL, NB_API_TOKEN +# Optional: NB_PEER_NAME (default proxy-peer) +# extra_hosts maps host.docker.internal → 172.17.0.1 (docker0). Colima's +# host-gateway is 192.168.5.2; UDP STUN to that IP hairpins. + +services: + proxy-peer: + build: + context: . + dockerfile: Dockerfile.proxy-peer + additional_contexts: + sandcat-scripts: ../../../cli/templates/devcontainer/sandcat/scripts + args: + NETBIRD_VERSION: ${NETBIRD_VERSION} + NETBIRD_SHA256_AMD64: ${NETBIRD_SHA256_AMD64} + NETBIRD_SHA256_ARM64: ${NETBIRD_SHA256_ARM64} + hostname: ${NB_PEER_NAME:-proxy-peer} + extra_hosts: + - "host.docker.internal:172.17.0.1" + cap_add: + - NET_ADMIN + environment: + - NB_SETUP_KEY=${NB_SETUP_KEY} + - NB_MANAGEMENT_URL=${NB_MANAGEMENT_URL} + - NB_API_TOKEN=${NB_API_TOKEN} + - NB_PEER_NAME=${NB_PEER_NAME:-proxy-peer} + - PROXY_PEER_PORT=8080 + volumes: + - netbird-proxy-peer-state:/var/lib/netbird + restart: unless-stopped + +volumes: + netbird-proxy-peer-state: diff --git a/docs/examples/proxy-peer/netbird.env b/docs/examples/proxy-peer/netbird.env new file mode 100644 index 00000000..fe3ee3f1 --- /dev/null +++ b/docs/examples/proxy-peer/netbird.env @@ -0,0 +1,16 @@ +# Pinned NetBird client binary for this example (copy of the sandcat mitmproxy pin). +# +# Consumed by compose-proxy-peer.yml build args. When bumping, keep in sync with +# cli/templates/devcontainer/sandcat/netbird.env. +# +# When bumping the version: +# 1. Update NETBIRD_VERSION and BOTH checksums below from the release assets: +# https://github.com/netbirdio/netbird/releases/download/v/netbird__checksums.txt +# 2. Look for netbird__linux_amd64.tar.gz and netbird__linux_arm64.tar.gz +# +# Format note: simple KEY=value lines only (no quotes, no spaces around `=`) so +# this file is consumable by `source`, compose build-arg injection, and contract +# tests alike. +NETBIRD_VERSION=0.72.4 +NETBIRD_SHA256_AMD64=8ee7807d716ed088ab05976bc161838120730f9cf9fec794aacb5b51d904f1fc +NETBIRD_SHA256_ARM64=7d2be0ef0cbe82bc18071505f69ff7d9967b492664aafdc39ce531233d6e6405 diff --git a/docs/examples/proxy-peer/scripts/proxy-peer-hello.py b/docs/examples/proxy-peer/scripts/proxy-peer-hello.py new file mode 100755 index 00000000..6983d4bf --- /dev/null +++ b/docs/examples/proxy-peer/scripts/proxy-peer-hello.py @@ -0,0 +1,36 @@ +#!/usr/bin/env python3 +"""Minimal HTTP hello for proxy-peer mesh smoke tests.""" +from __future__ import annotations + +import argparse +import json +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer + + +class Handler(BaseHTTPRequestHandler): + def do_GET(self) -> None: # noqa: N802 + if self.path not in ("/hello", "/hello/"): + self.send_error(404) + return + body = json.dumps({"service": "proxy-peer", "ok": True}).encode() + self.send_response(200) + self.send_header("Content-Type", "application/json") + self.send_header("Content-Length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def log_message(self, format: str, *args: object) -> None: + return + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--host", default="0.0.0.0") + parser.add_argument("--port", type=int, default=8080) + args = parser.parse_args() + server = ThreadingHTTPServer((args.host, args.port), Handler) + server.serve_forever() + + +if __name__ == "__main__": + main() diff --git a/docs/examples/proxy-peer/scripts/proxy-peer-init.sh b/docs/examples/proxy-peer/scripts/proxy-peer-init.sh new file mode 100755 index 00000000..14e399ec --- /dev/null +++ b/docs/examples/proxy-peer/scripts/proxy-peer-init.sh @@ -0,0 +1,22 @@ +#!/usr/bin/env bash +# docs/examples/proxy-peer/scripts/proxy-peer-init.sh +set -euo pipefail + +NB_SETUP_KEY="${NB_SETUP_KEY:?NB_SETUP_KEY is required}" +NB_MANAGEMENT_URL="${NB_MANAGEMENT_URL:-}" +NETBIRD_IFACE="${NETBIRD_IFACE:-wt0}" +HELLO_PORT="${PROXY_PEER_PORT:-8080}" +# DNS label used for both `netbird up --hostname` and the post-enrollment +# PATCH /api/peers/{id}. +NB_PEER_NAME="${NB_PEER_NAME:?NB_PEER_NAME must be set by compose}" +NETBIRD_PEER_LOG_PREFIX="${NETBIRD_PEER_LOG_PREFIX:-proxy-peer}" +NETBIRD_PEER_LIFECYCLE_PATH="${NETBIRD_PEER_LIFECYCLE_PATH:-/usr/local/lib/netbird-peer-lifecycle.sh}" + +# shellcheck source=/usr/local/lib/netbird-peer-lifecycle.sh +source "$NETBIRD_PEER_LIFECYCLE_PATH" + +netbird_start "${NETBIRD_IFACE}" +netbird_set_dns_label +netbird_supervise_daemon "${NETBIRD_IFACE}" & + +exec python3 /usr/local/bin/proxy-peer-hello.py --port "$HELLO_PORT" diff --git a/docs/examples/proxy-peer/settings-proxy-peer.json b/docs/examples/proxy-peer/settings-proxy-peer.json new file mode 100644 index 00000000..4f9ae007 --- /dev/null +++ b/docs/examples/proxy-peer/settings-proxy-peer.json @@ -0,0 +1,15 @@ +{ + "network": [ + { + "action": "allow", + "host": "proxy-peer.netbird.selfhosted", + "port": 8080, + "comment": "Layer 1 allow: stable NetBird FQDN for proxy-peer. Mesh permission is a NetBird dashboard ACL (groups + TCP 8080)." + } + ], + "_layer1_mesh_notes": [ + "Layer 1 (mitmproxy allow/deny) decides whether mitmproxy forwards the request.", + "Mesh permission is a NetBird dashboard ACL (groups + TCP 8080), not a sandcat lease.", + "Delete the Default All-to-All policy or the custom ACL has no effect." + ] +}