Harden installer: minisign signatures, manifest schema validation, opt-in tracking

.github/workflows/ci.yml

@@ -0,0 +1,43 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+    paths: ['install.sh', 'scripts/**', 'tests/**']
+  pull_request:
+    paths: ['install.sh', 'scripts/**', 'tests/**']
+  workflow_dispatch:
+
+jobs:
+  shellcheck:
+    name: Shellcheck
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install shellcheck
+        run: sudo apt-get update && sudo apt-get install -y shellcheck
+      - name: Lint
+        run: |
+          shellcheck --shell=bash --severity=style install.sh
+          shellcheck scripts/sign-release.sh
+          shellcheck tests/*.sh
+
+  tests:
+    name: Installer tests (${{ matrix.os }})
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+    runs-on: ${{ matrix.os }}
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install minisign
+        if: runner.os == 'Linux'
+        run: sudo apt-get update && sudo apt-get install -y minisign
+      - name: Install minisign
+        if: runner.os == 'macOS'
+        run: brew install minisign
+      - name: Syntax check
+        run: bash -n install.sh && bash -n scripts/sign-release.sh
+      - name: Run test suite
+        run: bash tests/run.sh

README.md

@@ -82,6 +82,22 @@ First run opens the browser for authentication. See the [Quickstart](https://doc
 
 More on the [CLI product page](https://adalagent.ai/product/cli).
 
+## Security & releases
+
+The installer in this repo is the hardened reference implementation:
+
+- **Ed25519 signature verification** (minisign) of every downloaded artifact,
+  with a transition mode while signatures are being rolled out
+- **Manifest schema validation** — malformed manifests are refused, never
+  silently trusted
+- Strict semver + tarball-filename validation, checksum + size verification,
+  temp-dir cleanup on failure
+- **Install tracking is opt-in** (`--track` / `ADAL_TRACK=1`) and signed
+
+See [SECURITY.md](SECURITY.md) for the threat model and [RELEASE.md](RELEASE.md)
+for the release/signing runbook. The test suite in [`tests/`](tests/) runs in
+CI via `.github/workflows/ci.yml`.
+
 ## Documentation
 
 Everything is published at [docs.sylph.ai](https://docs.sylph.ai/).

RELEASE.md

@@ -0,0 +1,177 @@
+# Release & deploy runbook — AdaL CLI
+
+How to publish a signed release and keep the installers honest. This document
+is the companion to SECURITY.md; read that first.
+
+## Layout (observed from the live CDN)
+
+```
+s3://<bucket>/cli/
+├── latest                      # plain text: latest stable version, e.g. "1.5.7"
+├── beta                        # plain text: latest beta version
+├── manifests/
+│   └── manifest-<version>.json # one per version
+└── <version>/
+    ├── adal-<version>-darwin-arm64.tar.gz
+    ├── adal-<version>-darwin-arm64.tar.gz.minisig     ← NEW
+    ├── adal-<version>-darwin-x64.tar.gz
+    ├── adal-<version>-darwin-x64.tar.gz.minisig       ← NEW
+    ├── adal-<version>-linux-x64.tar.gz
+    ├── adal-<version>-linux-x64.tar.gz.minisig        ← NEW
+    ├── adal-<version>-linux-arm64.tar.gz
+    ├── adal-<version>-linux-arm64.tar.gz.minisig      ← NEW
+    └── adal-<version>-win32-x64.zip
+    └── adal-<version>-win32-x64.zip.minisig           ← NEW
+```
+
+## Manifest schema (enforced by install.sh)
+
+```json
+{
+  "version": "1.5.7",
+  "channel": "latest",
+  "platforms": {
+    "darwin-arm64": {
+      "filename": "adal-1.5.7-darwin-arm64.tar.gz",
+      "checksum": "<sha256 hex, 64 chars>",
+      "size": 74559913
+    }
+  }
+}
+```
+
+The installer **hard-fails** if `version` is missing, if the platform block is
+missing, if `checksum` is not 64 hex chars, if `size` is not a positive
+integer, or if `filename` does not match the URL being downloaded. Keep this
+schema stable — a rename silently disables verification for old installers.
+
+Platform keys currently accepted: `darwin-arm64`, `darwin-x64`, `linux-x64`,
+`linux-x64-musl`, `linux-arm64`, `win32-x64`. `linux-arm64-musl` is detected
+but **not published yet** — the installer prints a specific, actionable error
+for it instead of a generic one.
+
+## Release procedure (add to the existing release pipeline)
+
+```bash
+# 1. Build tarballs for every platform (existing pipeline step), then:
+
+# 2. Generate the manifest (existing step), then:
+
+# 3. Sign every artifact with the release key
+./scripts/sign-release.sh \
+  --version 1.5.7 \
+  --key     $ADAL_SIGNING_KEY \        # CI secret, NOT committed
+  --files   "dist/adal-1.5.7-*.tar.gz" "dist/adal-1.5.7-*.zip"
+
+# 4. Sanity-check the signatures
+./scripts/sign-release.sh --check --files "dist/adal-1.5.7-*"
+
+# 5. Upload to S3 (existing step): each <tarball> AND its <tarball>.minisig
+# 6. Update manifests/, latest, beta (existing steps)
+# 7. Deploy the hardened install.sh to adal.sylph.ai/install.sh
+```
+
+If the pipeline generates manifests itself, keep the schema above. If it
+already uploads `.minisig` files under a different convention, adjust
+`verify_signature` in install.sh accordingly.
+
+## First release with signing — key custody
+
+A keypair was generated for this change:
+
+- Private key: `adal-release.minisign` (this value is held out-of-band; it is
+  NOT in this repository).
+- Public key: already embedded in `install.sh` as `SIGNING_PUBLIC_KEY`.
+
+Actions:
+
+1. Copy the private key into the release pipeline's secret store (GitHub
+   Actions secret, AWS Secrets Manager, etc.).
+2. Confirm the pipeline can sign (step 3 above) before publishing.
+3. Delete the private key from the machine that generated it.
+4. If the pipeline must sign without human interaction, store the key
+   **encrypted** in the secrets manager and decrypt in the workflow. Do not
+   commit it, do not embed it in images.
+
+## CI matrix for linux-arm64-musl (close the gap)
+
+Add the missing build to the release matrix, then the existing
+`linux-x64-musl` logic in `detect_platform()` handles it automatically:
+
+```yaml
+# In the release workflow's build matrix:
+matrix:
+  include:
+    - os: ubuntu-latest
+      target: linux-arm64-musl
+      command: |
+        cargo build --release --target aarch64-unknown-linux-musl
+        # then bundle with the musl bun runtime (same as linux-x64-musl)
+```
+
+Verify the `apk add libstdc++ libgcc` runtime-deps path in
+`ensure_musl_runtime_deps()` against the produced artifact before shipping.
+
+## Tracking endpoint — server-side reference (HMAC + timestamp)
+
+Client behavior (`track_install` in install.sh, opt-in only):
+
+- Default OFF. Enabled by `--track` or `ADAL_TRACK=1`; `--no-track` /
+  `ADAL_NO_TRACK=1` always win.
+- `POST` with `content-type: application/json`:
+  `{"platform":"cli","channel":"stable|beta","event_type":"install|upgrade","version":"1.5.7","ts":<unix>,"nonce":"<32 hex>"}`
+- Headers: `X-Adal-Signature: <hex hmac-sha256(body, secret)>`,
+  `X-Adal-Ts: <unix>`, `X-Adal-Nonce: <32 hex>`.
+
+Minimal server-side validation (Python/FastAPI flavor):
+
+```python
+import hashlib, hmac, json, time
+
+SECRET = b"adal-cli-install-track-v1"  # keep in sync with install.sh
+MAX_AGE = 300  # seconds
+
+async def track(request):
+    body = await request.body()
+    sig = request.headers.get("X-Adal-Signature", "")
+    ts = int(request.headers.get("X-Adal-Ts", "0"))
+    if hmac.compare_digest(sig, hmac.new(SECRET, body, hashlib.sha256).hexdigest()) \
+       and 0 <= time.time() - ts <= MAX_AGE \
+       and request.headers.get("X-Adal-Nonce"):
+        record(json.loads(body))
+    # rate-limit by IP + User-Agent; treat signature as a filter, not identity
+    return 200
+```
+
+Reminder from SECURITY.md: the secret ships in a public installer, so the HMAC
+is an anti-naive-abuse filter, not authentication. Rate limiting is the real
+protection.
+
+## Installing the hardened installer
+
+```bash
+# Directly:
+curl -fsSL https://adal.sylph.ai/install.sh | bash
+
+# Specific version / channel:
+curl -fsSL https://adal.sylph.ai/install.sh | bash -s -- --version 1.5.7
+curl -fsSL https://adal.sylph.ai/install.sh | bash -s -- --version beta
+
+# Dockerfiles that install AdaL should pin + enforce signatures:
+RUN curl -fsSL https://adal.sylph.ai/install.sh | \
+    ADAL_REQUIRE_SIGNATURE=1 bash
+```
+
+## Testing
+
+```bash
+brew install shellcheck minisign   # or: apt install shellcheck minisign
+bash -n install.sh
+shellcheck --shell=bash install.sh scripts/*.sh tests/*.sh
+bash tests/run.sh
+```
+
+The E2E suite builds a fake signed release served over `file://` and verifies
+the full pipeline: happy path, tampered tarball, forged manifest, missing
+signature (transition + `ADAL_REQUIRE_SIGNATURE=1`), schema violations, channel
+resolution, and the HMAC tracking payload (captured by a local HTTP server).

SECURITY.md

@@ -0,0 +1,96 @@
+# Security — AdaL Installer
+
+This document describes the threat model of `install.sh`, what the hardening
+protects against, what remains intentionally unguarded, and what the SylphAI
+release pipeline must do to close the remaining gaps.
+
+## Threat model
+
+The installer runs as the invoking user (root in Dockerfiles). The relevant
+attackers are:
+
+| Attacker | Capability | Defended by |
+|---|---|---|
+| CDN/S3 bucket compromise | Rewrite any artifact + manifest served from `adal.sylph.ai` | **minisign Ed25519 signature** over each tarball; public key embedded in the installer |
+| Compromised manifest only | Modify checksums/sizes in `manifest-<ver>.json` | Signature: checksum is not a trust root, only a corruption check |
+| Content-delivery tampering | MITM or modify bytes in transit (non-TLS, or TLS breakage) | Signature covers the artifact bytes; checksum additionally verifies integrity |
+| Malicious version/filename injection | Craft `--version` or a tarball name that smuggles shell into URLs/paths | Strict semver regex + strict tarball filename regex |
+| Malformed/manipulated manifest | Omit or falsify checksum/size to weaken verification | Manifest schema validation; **hard fail** when a field is missing/invalid |
+
+## What the installer verifies, in order
+
+1. **Version string** — strict semver (`X.Y.Z` or `X.Y.Z-pre.n`) before it is
+   interpolated into any URL.
+2. **Manifest schema** — `version`, `platforms.<platform>.filename`,
+   `.checksum` (64 hex), `.size` (positive int) must all be present and valid;
+   otherwise the install refuses to proceed.
+3. **Downloaded size** — cheap first-line check against the manifest.
+4. **SHA-256 checksum** — against the manifest value (integrity check).
+5. **Ed25519 signature** — `minisign -Vm <tarball> -P <embedded pubkey>` using
+   the detached `<tarball>.minisig`. This is the tamper-protection layer.
+
+A `curl | bash` install can never be fully protected by the script itself —
+the user must review what they pipe to `bash` — but the combination above
+means an attacker who controls the CDN still cannot install signed payloads
+they did not author.
+
+## Transition mode (signature availability)
+
+The release pipeline does not yet publish `.minisig` files. To avoid breaking
+installs today, the installer runs in **transition mode**:
+
+- A `.minisig` file **is** published → it is *always* verified; a bad signature
+  is a hard failure (this was tested with a forged-manifest attack).
+- A `.minisig` file is **missing** → a loud warning is printed and install
+  continues with SHA-256 only.
+- `ADAL_REQUIRE_SIGNATURE=1` → missing signature or missing `minisign` binary
+  becomes a hard failure.
+
+**Rollout plan:** publish `.minisig` files for all platforms for ≥ 2 release
+cycles (so existing cached/old installers keep working), then flip the default
+in `install.sh` so a missing signature is a hard error, and finally advertise
+`ADAL_REQUIRE_SIGNATURE=1` in Dockerfiles/CI.
+
+## Key management
+
+- **Private key:** `adal-release.minisign` — must live ONLY in the release
+  pipeline (GitHub Actions secrets, a secrets manager, or a KMS-backed
+  signing service). Never commit it, never embed it, never send it to a
+  non-release machine.
+- **Public key:** embedded in `install.sh` as `SIGNING_PUBLIC_KEY`. Anyone can
+  see it — that is by design.
+- **Rotation:** generate a new keypair, embed the new public key in a new
+  `install.sh`, publish signatures with the new key for a full release cycle
+  (old installers still trust the old key), then move the old private key to
+  cold storage/delete it.
+- A keypair has already been generated and the public key embedded. **The
+  private key must be moved to the release pipeline and deleted from the
+  machine that generated it** (see RELEASE.md, "First release").
+
+## Known limitations (accepted)
+
+1. **HMAC tracking secret is extractable.** The tracking payload is signed
+   with a secret embedded in a public script. Anyone can read it, so the
+   signature is *obfuscation*, not authentication. The tracking endpoint must
+   not treat `X-Adal-Signature` as proof of identity; it should rate-limit by
+   IP/UA and validate the timestamp window. Reference server code is in
+   RELEASE.md.
+2. **No certificate pinning.** The connection to the release CDN uses standard
+   TLS. Pinning is not implemented because CDN cert rotation would break
+   installs; the signature layer is the tamper protection.
+3. **Local tarball installs** (`--local-tarball`) are not signed/checksummed —
+   by design, the file is already on the user's disk. The filename regex and
+   entry-point verification still apply.
+4. **The bootstrap is `curl | bash`.** The very first fetch of `install.sh`
+   itself is not authenticated. Consider publishing the script's SHA-256 on
+   the docs site as a secondary channel.
+
+## Org-side actions still required
+
+1. Wire the hardened `install.sh` into the private release pipeline and
+   deploy it to `adal.sylph.ai/install.sh`.
+2. Publish `.minisig` files for every artifact (scripts/sign-release.sh).
+3. Move the signing key into CI secrets; delete it from the generator machine.
+4. Add the `linux-arm64-musl` build to the release matrix (or stop advertising
+   it) — see RELEASE.md.
+5. Add HMAC + timestamp + rate-limit verification to the tracking endpoint.