diff --git a/docs/design/keyless-secrets/README.md b/docs/design/keyless-secrets/README.md index 3279d65e..fd921e89 100644 --- a/docs/design/keyless-secrets/README.md +++ b/docs/design/keyless-secrets/README.md @@ -4,6 +4,15 @@ **Area:** `pkg/api/secrets` (encryption envelope, recipients, CLI `secrets` commands) **Relates to:** [`docs/SECRETS-POLICY.md`](../../SECRETS-POLICY.md), [`docs/SECURITY.md`](../../SECURITY.md) +> **v1 implementation note (supersedes parts of this RFC):** the shipped "Minimal v1" is +> [`scoped-store-v1.md`](./scoped-store-v1.md). It deliberately diverges from the RFC's +> tooling decision: it reuses **sc's own ciphers** (RSA-OAEP + X25519 sealed box), NOT SOPS +> (adding SOPS would be a large new dependency for no capability sc lacks), and recipients are +> **SSH keys**, not native age. Scope selection is **key-driven** (a job decrypts exactly the +> scopes its key is a recipient of), which replaces the RFC's `secretScope:` config field with +> a cryptographic clamp. The KMS-wrapped / OIDC-federated recipient in this RFC remains the +> **v2** target and slots in as an additional recipient type on the same file format. + ## Summary Today, decrypting an SC secret store in CI requires materializing the store's @@ -268,9 +277,15 @@ itself; supply-chain compromise of the CLI or CI actions (mitigated by signed re ## Migration phases 0. **Design + review** (this document). -1. **v2 envelope + version-aware fail-closed client**, released and rolled out before - any v2 write. Modernize the asymmetric recipient scheme as part of v2. -2. **`KeyProvider` + first KMS provider + OIDC acquisition**, behind a feature flag. +1. **Envelope + version-aware fail-closed client**, released and rolled out before + any scoped write. Asymmetric recipient scheme modernized (X25519 for ed25519). + **DONE** — shipped; see [`scoped-store-v1.md`](./scoped-store-v1.md). +2. **`KeyProvider` + first KMS provider (AWS) + OIDC acquisition.** **DONE (code)** — the + `awskms://` recipient kind is implemented in the scoped store: the per-value data key is + wrapped with `kms:Encrypt` and opened with `kms:Decrypt` under the ambient (OIDC) AWS + credential chain, bound by a KMS EncryptionContext. It is additive (no flag needed — a + scope simply lists a KMS recipient); the remaining work is the per-stack consumer rollout + in phases 3–7 below. 3. **Canary** on one low-risk staging stack. Gate: `sc deploy` obtains cloud credentials from the federated environment, not the store; measured KMS latency; verified rollback runbook. diff --git a/docs/design/keyless-secrets/scoped-store-v1.md b/docs/design/keyless-secrets/scoped-store-v1.md new file mode 100644 index 00000000..bab61b59 --- /dev/null +++ b/docs/design/keyless-secrets/scoped-store-v1.md @@ -0,0 +1,365 @@ +# Scoped secret store v1 — implementation spec (`secrets..yaml`) + +Status: IMPLEMENTED (pending final review) — landed in this PR: the crypto core + scope +model (`pkg/api/secrets/scoped`: scopes.yaml, secrets..yaml sealing on sc's own +ciphers with scope/key AAD binding, fail-closed version guard, consistency checks); the +`sc secrets scope {set,get,list,delete,allow,disallow,lint,doctor}` CLI; key-driven +deploy-time resolution wired into the provisioner; **and the v2 KMS-recipient KeyProvider +(RFC phase 2): a scope may list `awskms://?region=` recipients whose data-key wrap is +performed with `kms:Encrypt` and opened with `kms:Decrypt` using the ambient (OIDC-federated) +AWS credential chain — no stored private key.** Both recipient kinds coexist in one scope +(the KMS path binds `(stack,scope,key)` via a KMS EncryptionContext, exactly parallel to the +SSH path's byte-AAD). It is purely additive: a scope with no KMS recipient never constructs an +AWS client. Multi-model review (Codex + Gemini) of the crypto/CLI surface passed with no P0s; +findings fixed. The remaining out-of-repo step is the consumer rollout (the Integrail +`secrets.pr.yaml` sweep, then adding the `awskms://` recipient + `ci-oidc-pr-scan` role and +deleting `SC_KEY_PR`). Implements "Minimal v1" **and** the v2 KeyProvider of the +keyless-secrets RFC in this directory. +Prerequisite already shipped: the fail-closed `schemaVersion` store guard is released +and baked fleet-wide, so old binaries hard-fail on formats they do not understand +instead of silently rewriting them. + +## Scope decision (settled — panel: Codex + Gemini + 2 Claude lenses) + +v1 is deliberately narrow. Three decisions were reviewed against the real consumer +workflows and `pkg/githubactions/actions/parent_repo.go`: + +- **D1 — v1 covers PR *scan/lint* jobs ONLY.** The `pull_request` jobs whose secrets move + into a scope are the pure scanners: `pr-security-scan`, `dast-zap`, `dast-nuclei-ddp`, + `defectdojo-cleanup`. Their whole secret need is a small low/med set + (`defectdojo-api-key`, `cf-access-client-id/secret`, `security-triage-slack-webhook-url`, + `pr-integrail-superadmin-password`). **Deploy-shaped PR jobs stay out of v1** + (preview deploy, provision-preview, destroy-service): per the parent/child model below, + dropping their master key forces *every* deploy secret into a scope file — that just + renames the master key. **Anti-goal made explicit:** the Pulumi crown jewels that + `pulumi-crossguard-scan` uses on PR (`pulumi-github-aws-*`, config passphrase, github + token) MUST move to a GitHub-OIDC role (`ci-oidc-pulumi-preview`), **never** into the + `pr` scope. A scope file that contains deploy-grade credentials is the failure state + this feature exists to prevent. +- **D2 — `SC_KEY_PR` (SSH ed25519 private key in a GitHub Actions secret) is the transition + interim; the v2 KMS-via-OIDC KeyProvider is now IMPLEMENTED in this PR.** The stored SSH key + is strictly smaller blast radius than today (`SC_CONFIG` opens the whole store) and, for a + same-org non-fork PR, an OIDC-fetched key would not stop a poisoned job from printing what it + decrypted — so it was never worth *blocking* the feature on the keyless path. But because + `sc` already encrypts Pulumi state with AWS KMS and CI already federates via OIDC (see + `secrets_providers.type=cloud` / `awskms://` in live state), the KMS-recipient-via-OIDC path + reuses existing infra and is a recipient-list swap on the *same* scope files — so it is now + shipped in-code as an additive recipient kind rather than deferred. Operationally: a scope + can carry both `SC_KEY_PR` (SSH) and an `awskms://` recipient during the transition; once the + `ci-oidc-pr-scan` role is wired in the consumer repo, `sc secrets scope disallow` the SSH key + and delete the `SC_KEY_PR` GitHub secret. No v1 work is thrown away. +- **D3 — scope files live in the devops PARENT repo (`integrail` stack store) for v1.** + Every PR consumer fetches `-s integrail`, and resolution merges parent+child (below), so + one CODEOWNERS-guarded `secrets.pr.yaml` in the parent serves all consumers. The resolver + still supports consumer-repo scope files (they merge on top of parent) — that path is + reserved for the later deploy sweep, not populated in v1. + +## Problem + +Today one master key (`SIMPLE_CONTAINER_CONFIG`) decrypts the entire whole-file store +(`.sc/secrets.yaml`). Every CI context that needs *any* secret gets *all* secrets — +including `pull_request`-triggered jobs (previews, provision-preview), which are the +most attacker-reachable contexts in the pipeline. Per-scope files with per-scope +recipients replace "one key opens everything" with "a key opens exactly the scope files +it is a recipient of". + +## Design summary + +- **Crypto/format layer: sc's existing `pkg/api/secrets/ciphers`, NOT SOPS.** The RFC + named SOPS, but sc depends on neither `getsops/sops` nor `filippo.io/age`, and adding + them would be a large new supply-chain surface for no capability sc lacks: the ciphers + package already does per-recipient sealing (RSA-OAEP for `ssh-rsa`, ephemeral-static + X25519 + ChaCha20-Poly1305 for `ssh-ed25519`) with authenticated encryption. The scoped + store reuses it, adding backward-compatible associated-data variants + (`EncryptLargeStringWithAAD` etc.) — a nil AAD reproduces the legacy wire format + byte-for-byte, so the whole-file store is untouched — plus symmetric envelope helpers + (`SealAEAD`/`OpenAEAD`, ChaCha20-Poly1305). +- **Envelope encryption per value.** Each value is encrypted ONCE under a fresh random + 32-byte data key (DEK) with the value's `(stack, scope, key)` bound as AEAD associated + data; the DEK is then wrapped per recipient. This was chosen over direct-per-recipient + value sealing after review, because it (a) makes the value ciphertext identical for every + recipient — so a tampered per-recipient slot yields a *decrypt failure, never a different + plaintext* (no targeted per-recipient divergence), (b) gives one whole-value MAC (no chunk + splicing), and (c) makes the DEK a single 32-byte block — no RSA chunking at all. +- **Recipients are SSH public keys OR AWS KMS keys.** SSH recipients (`ssh-ed25519` / + `ssh-rsa`) are the same key material the whole-file store and `sc secrets allow` already use + — not native age recipients; the transitional `SC_KEY_PR` is an unencrypted SSH ed25519 + private key. A KMS recipient is an `awskms://?region=` URL (sc's canonical KMS-key + form): its data-key wrap is a `kms:Encrypt`/`kms:Decrypt` pair run under the ambient AWS + credential chain, so in CI it is opened by an OIDC-federated role with no stored key. The two + kinds are interchangeable per-recipient and can coexist in one scope. A KMS recipient's + identity is its normalized URL, compared as a string (resolving an alias→key needs a KMS + call, so it cannot be done offline); recipients must therefore be spelled consistently — + **prefer the key ARN over an alias** (an alias can be repointed, and the same key written as + alias vs ARN counts as two different recipients for drift/dedup). `sc` pins the `KeyId` on + every decrypt, so a wrap made under a different key than the recipient names is rejected. +- KMS Decrypt failures are classified three ways: `InvalidCiphertext`/`IncorrectKey` → + integrity (tamper, hard-fail); `AccessDenied`/disabled/no-credentials → not-a-recipient + (least-privilege skip); anything else (throttle after SDK retries, `KMSInternal`, timeout) → + `ErrScopedUnavailable` (fatal retry, never mislabeled as tamper and never a silent skip — + position-independent regardless of which value hit it). +- One committed-encrypted file per scope: `.sc/stacks//secrets..yaml` — its + structure (schemaVersion, stack, scope, recipients, value KEYS) is readable/diffable; each + value is `{ciphertext: , wraps: {: }}`, values opaque. Confidentiality + integrity come from the + AEAD layer, not a separate MAC. +- Both static SSH recipients and KMS/OIDC recipients are implemented (the KMS `KeyProvider` + wraps the same per-value data key to a KMS-wrapped key decrypted via OIDC — a recipient kind, + no value-format change). The KMS wrap's `(stack,scope,key)` binding is a KMS EncryptionContext + (server-enforced), so a KMS-wrapped value is as transplant-resistant as an SSH-wrapped one. +- The legacy whole-file store keeps working unchanged (mode A). Scoped files are + additive (mode B); old binaries never open them. + +## Parent/child resolution — the load-bearing constraint (P0-1) + +A child (client) stack with `parent: /` does NOT decrypt in isolation. +`parent_repo.go` clones the parent repo, reveals its `.sc/stacks/*` plaintext, **copies it +into the child workspace**, then optionally reveals the child's own `.sc/stacks//`. +So `${secret:KEY}` resolves against a MERGE of (parent store) ⊕ (child store). Two +consequences the spec must honor: + +1. The parent is the real source of shared secrets and the real boundary. v1 scope files + therefore live in the parent (D3). +2. Any job that drops `SC_CONFIG` and holds only `SC_KEY_PR` can decrypt **only** the + parent's `secrets.pr.yaml` — it can no longer reveal the parent whole-file store. This + is exactly why deploy-shaped jobs are excluded from v1 (D1): they would need their whole + transitive secret set scoped, which recreates a broad key. + +## File layout + +``` +.sc/ # (devops PARENT repo, stack `integrail`) + scopes.yaml # scope -> recipients mapping (CODEOWNERS-gated) + secrets.yaml # legacy whole-file registry (mode A, unchanged) + stacks/integrail/ + secrets.yaml # legacy plaintext (gitignored), mode A + secrets.pr.yaml # sc-cipher encrypted, committed, scope "pr" <-- v1 +``` + +`secrets..yaml` files are **committed encrypted**: keys/structure (schemaVersion, +scope, recipients, value names) stay diffable, each value is an opaque `{ciphertext, wraps}` +envelope. They +are NOT listed in the legacy registry and NOT touched by `sc secrets hide/reveal` legacy +paths. Consumer-repo scope files +(`/.sc/stacks//secrets..yaml`) are a supported location the +resolver merges on top of the parent, reserved for the later deploy sweep — v1 ships only +the parent `secrets.pr.yaml`. + +## `scopes.yaml` (the governance surface) + +```yaml +schemaVersion: 1 +scopes: + pr: + description: values safe to expose to pull_request-triggered scan/lint CI + recipients: + - ssh-ed25519 AAAA...admin # break-glass admin key (keep ≥1 SSH for reseal) + - awskms://arn:aws:kms:us-east-1:123:key/abcd # keyless CI (prefer ARN): opened by ci-oidc-pr-scan role + # transition-only: the SSH ci-pr key below is a recipient until ci-oidc-pr-scan is wired, + # then `sc secrets scope disallow` it and delete the SC_KEY_PR GitHub secret. + - ssh-ed25519 AAAA...ci-pr # GitHub secret SC_KEY_PR [interim] +``` + +- **CODEOWNERS-gated** (`/.sc/scopes.yaml @/devops`): recipient changes cannot ride + an ordinary PR (RFC non-negotiable 3). +- **CODEOWNERS is necessary but not self-enforcing (P0-4):** `sc secrets lint` independently + verifies each scope file's recipient set == `scopes.yaml` recipients and FAILS on + drift, so a recipient added by editing a scope file directly (bypassing `scopes.yaml`) is + caught even if CODEOWNERS review is skipped. +- `sc` regenerates each scope file's recipient set from `scopes.yaml` and reseals its + values on `allow`/`disallow` (there is no separate `updatekeys` verb). +- Removing a recipient re-encrypts the file but does NOT protect history: + `sc secrets disallow --scope` prints a mandatory rotate-values warning + (RFC non-negotiable 5). + +## CLI UX + +The scoped verbs are namespaced under `sc secrets scope` so they never collide with the +whole-file store's existing `secrets add/allow/disallow/reveal/hide` (which have different +semantics): + +``` +sc secrets scope set --scope pr -s KEY [VALUE|-] # seal/update one value (VALUE arg or stdin) +sc secrets scope get --scope pr -s KEY # decrypt one value with the ambient/scope key +sc secrets scope list --scope pr -s # list value names (never prints values) +sc secrets scope delete --scope pr -s KEY # remove a value +sc secrets scope allow --scope pr # add recipient to scopes.yaml + reseal its files +sc secrets scope disallow --scope pr # remove recipient + reseal + rotate-values warning +sc secrets scope lint # plaintext-leak + recipient-drift + scope/stack binding + duplicate-key gate +sc secrets scope doctor # which scopes the ambient key can open +sc secrets reveal # legacy mode-A whole-file store, unchanged +``` + +Key discovery order for decrypt (`get`/`doctor`/deploy): an explicit `--key-file`, then the +per-scope CI env `SC_KEY_` (e.g. `SC_KEY_PR`) or the generic `SC_SCOPE_KEY`, then the +ambient `SIMPLE_CONTAINER_CONFIG`/`SC_CONFIG` SSH key, and finally — for any value that carries +an `awskms://` wrap — a KMS `Decrypt` using the ambient AWS credential chain (an OIDC role in +CI). A `pull_request` scan job can therefore hold ONLY its scope key, or NO key at all when the +scope's recipient is a KMS key and the job federates to AWS via OIDC. Being a scope recipient is +opt-in per value; a value with no KMS recipient never triggers an AWS call. There is no `edit` +verb (use `get`/`set`); reseal is folded into `allow`/`disallow` (no separate `updatekeys`). +`allow`/`disallow` reseal by decrypting with an SSH recipient key, so a scope should keep at +least one SSH break-glass recipient even after adding a KMS one. + +## Scope integrity — binding beyond confidentiality (P0-3) + +Per-recipient AEAD/OAEP gives confidentiality + tamper-evidence of each value, but not, by +itself, binding to *where* the value lives. These attacks are closed by explicit binding +(implemented in `pkg/api/secrets/scoped`): + +- **File rename / move across scope or stack:** a PR renames `secrets.prod.yaml` → + `secrets.pr.yaml`, or copies one stack's `secrets.prod.yaml` into another stack's dir. + Mitigation: the scope name AND the stack name are first-class fields inside the file, and + `LoadScopeFile` + deploy-time resolution FAIL if the in-file scope doesn't match the + filename's `` or the in-file stack doesn't match the parent directory + (`sc secrets scope lint` enforces both). +- **Ciphertext transplant / PR write-poisoning:** a PR copies a `prod`-scoped encrypted + value blob into `secrets.pr.yaml` (or another stack's file) to get it decrypted by a key + it holds. Mitigation: each encrypted value's AEAD/OAEP associated data is the + domain-separated `stack\0scope\0key`, and for multi-chunk RSA values the OAEP label also + binds each chunk's index + count — so a value blob only decrypts under the exact + `(stack, scope, key)` it was written for, in its original chunk order, and a + transplanted/reordered/truncated blob fails its AAD/OAEP check. +- **Legacy-blob downgrade:** the pre-X25519 ed25519 scheme derived its key from public data + and ignored associated data. A decrypt under a scoped AAD refuses any non-X25519 + (legacy-shaped) blob, so a forged legacy blob cannot bypass the binding. +- **Offline shape gate:** `lint` decodes every value chunk and checks it has the exact + ciphertext shape for its recipient's key type (RSA modulus size, or an X25519 sealed box + with the magic prefix), rejecting a plaintext value smuggled under a recipient fingerprint + without needing a private key. + +## Deploy-time resolution (`${secret:...}`) + +Resolution is **key-driven, not config-driven** — this is stronger than the RFC's +`secretScope:` field and drops P0-6 entirely. Implemented in +`scoped.ResolveScopedValues`, called from the provisioner's secrets-read +(`readSecretsDescriptorFromFile`) so both `${secret:}` and `sc stack secret-get` see the +result transparently: + +1. For the stack's `.sc/stacks//` directory, every `secrets..yaml` the + **ambient key is a recipient of** contributes its values; scope files the key cannot + open are skipped. There is no scope config field to set or subvert. +2. **The `pull_request` clamp is therefore cryptographic (supersedes P0-6):** a job holding + only `SC_KEY_PR` is a recipient of the `pr` scope alone, so it *cannot* decrypt + `secrets.prod.yaml` — not because a config says so, but because it is not a recipient. + Nothing in a PR's `client.yaml` can widen this. +3. Merge order: the legacy whole-file store (mode A) wins on conflict; scoped values only + **add** keys not already present, so a scoped file can never change an existing + `${secret:}` resolution. Repos with no scope files are entirely unaffected (the resolver + returns empty without even parsing the key). +4. **Hard-fail — a real error, not a swallowed warn (P0-2):** a value the key IS a recipient + of but cannot decrypt (tampered ciphertext / broken binding), a corrupt or renamed scope + file, or the same key present in two openable scopes (ambiguous) all abort the read with + a non-nil error naming the scope. A `${secret:KEY}` that resolves to nothing still + hard-fails in the placeholder resolver (existing behavior). Not being a recipient of a + scope is NOT an error — you simply don't see it (least privilege). +5. A KEY must live in exactly one mode; `sc secrets scope lint` rejects duplicates + (mode A vs mode B, and cross-scope) to keep resolution deterministic. + +## CI wiring (consumer side, Integrail) — v1 = scan/lint only (D1) + +- New GitHub secret `SC_KEY_PR` (an SSH ed25519 private key, recipient of the `pr` scope + only). The four scan/lint workflows triggered by `pull_request` get `SC_KEY_PR` and **stop + receiving `SC_CONFIG`**: `pr-security-scan`, `dast-zap`, `dast-nuclei-ddp`, + `defectdojo-cleanup`. They read values via `sc secrets scope get --scope pr -s integrail + ` (which picks up `SC_KEY_PR` from the env), replacing `sc stack secret-get`. +- One-time `sc secrets set --scope pr` sweep in the parent for the exact keys those jobs + read: `defectdojo-api-key`, `cf-access-client-id`, `cf-access-client-secret`, + `security-triage-slack-webhook-url`, `pr-integrail-superadmin-password` — values rotated + as they move (history hygiene). +- **Out of scope for v1, stated so no one relabels the master key:** + - deploy-shaped PR jobs (`provision-preview`, `build-and-deploy` PR-preview, + `destroy-service`) keep `SC_CONFIG` until v2; + - `pulumi-crossguard-scan`'s Pulumi credentials move to `ci-oidc-pulumi-preview` + (GitHub-OIDC), **not** the `pr` scope. (A crossguard PR job currently still fetching + static `pulumi-github-aws-*` is a separate OIDC-migration fix, tracked outside this + spec.) +- Trusted contexts (push to main / RC/*) keep `SC_CONFIG` until v2; their AWS creds are + already OIDC-federated (`ci-oidc-sc-deploy`). + +## Plaintext-leak lint (ships with the feature, RFC non-negotiable 4) + +`sc secrets lint` (and a CI gate in the central security scan): +- every `secrets..yaml` parses; every value's `ciphertext` is a plausibly-shaped + envelope AEAD blob and its `wraps` are non-empty per-recipient DEK wraps of the exact + shape for each recipient's key type — no plaintext value leaf — `ScopeFile.VerifyConsistency`; +- each value's DEK is wrapped to exactly the declared `recipients` set (no missing/extra + recipient fingerprint), and those recipients == `scopes.yaml` recipients for the scope + (drift = fail) (P0-4); +- in-file scope name == filename `` and in-file stack == parent dir (P0-3); value + binding (`stack\0scope\0key` AAD) is + enforced structurally at decrypt, not lintable offline; +- no `secrets..yaml` is gitignored (must be committed encrypted); +- legacy plaintext files (`stacks/*/secrets.yaml`) remain gitignored (unchanged rule). + +## Compatibility & versioning + +- Mode A is untouched: format, registry, `hide`/`reveal`, recipients — zero change for + every current user; nothing is deprecated in v1. +- Old binaries: never open `secrets..yaml`; if a value is moved to a scope file + and an old binary deploys that stack, resolution fails **closed** (missing secret + hard-fail), never silently empty. `scopes.yaml` carries its own `schemaVersion`, + covered by the shipped fail-closed guard pattern. +- No new module dependency: the SSH path reuses `pkg/api/secrets/ciphers` + (`golang.org/x/crypto`, already present). The KMS `KeyProvider` uses + `aws-sdk-go-v2/service/kms` + `config` + `smithy-go`, all of which were already in the + dependency graph (transitively via the existing AWS integrations) — this PR only promotes + `service/kms` and `smithy-go` from indirect to direct in `go.mod`. The KMS recipient is an + additive recipient kind, not a change to the value format or the file layout. + +## Threat-model deltas + +| Threat | Before | After v1 | +|---|---|---| +| Compromised PR scan job reads fleet secrets | full store via `SC_CONFIG` | `pr` scope only (low/med scan creds) | +| Deploy-grade creds reachable from a PR scope | n/a | explicitly excluded (D1); crossguard creds → OIDC | +| Malicious PR adds itself as recipient | n/a (single key) | blocked: CODEOWNERS on `scopes.yaml` + `sc` recipient-verify lint (P0-4) | +| PR renames/transplants a scope file | undetected by MAC | scope-name + `path:scope:key` AAD binding (P0-3) | +| PR reaches a wider scope's secrets | n/a | impossible: resolution is key-driven, a `pr` key is not a recipient of `prod` (P0-6 dropped — no config to subvert) | +| Missing/undecryptable secret silently empty | possible | real error, deploy aborts (P0-2) | +| Old binary corrupts new format | guarded (schemaVersion) | scoped files never opened by old binaries | +| Recipient removed ≠ revoked | same | explicit rotate-values warning; runbook | +| Stored `SC_KEY_PR` is a smaller master key | — | true, but scoped to low/med scan creds only; eliminated by using a KMS recipient (no stored key — CI decrypts via an OIDC role), now implemented (D2) | +| CI private key stolen from a GitHub secret | whole store via `SC_CONFIG` | none for a KMS-recipient scope: there is no stored key — decryption requires assuming the OIDC role, gated by branch/environment protections + IAM | + +## Testing + +- Unit: scope resolution order, hard-fail matrix (missing key / undecryptable scope / + duplicate KEY), scopes.yaml↔scope-file recipient drift, scope-name/AAD binding rejection, + key-driven scope resolution (recipient sees only its scopes), cross-scope duplicate + rejection, allow/disallow re-encrypt. +- KMS unit (with a fake KMS client — no AWS call): `awskms://` parse/normalize (ARN region + derivation, missing-region rejection), envelope roundtrip via KMS, EncryptionContext (AAD) + mismatch → integrity error, AccessDenied → least-privilege skip (not integrity), a mixed + SSH+KMS scope opened by EITHER an SSH key or the KMS role to the SAME value, recipient-drift + with KMS URLs, KMS short-blob lint rejection, resolver via a KMS role, and CLI allow/disallow + governance of a KMS recipient. The live-KMS + real-OIDC path is validated at consumer-rollout + canary (RFC phase 3), not in unit tests. +- e2e (preview build, real binary): PR-key can `get --scope pr` but not `--scope prod`; + a `pull_request` run resolves the four scan jobs' keys from `secrets.pr.yaml` with + `SC_KEY_PR` and NO `SC_CONFIG`; a PR that renames/transplants a scope file fails lint; + a key that is a recipient of only `pr` resolves pr values but not prod's; old released binary against a repo with + scoped files deploys mode-A-only stacks untouched and hard-fails on a scoped-value stack. +- Panel review (Codex + Gemini + Claude lenses) on the crypto-adjacent surface before + merge, same as P1. + +## Delivery plan (single consolidated PR, after design sign-off) + +1. `pkg/api/secrets/scoped/`: scopes.yaml model + scope-file sealing (sc ciphers) + resolution + + scope-name/AAD binding + key-driven resolver (ResolveScopedValues) + real hard-fail. +2. CLI verbs (`set/edit/get/allow/disallow/lint/doctor` scope forms), with `lint` + enforcing recipient-verify, path/scope binding, and plaintext-leak gates. +3. Placeholder resolution hook (`${secret:}` order above) wired through parent/child merge. +4. Docs (`secrets-management.md` section) + this spec updated to Status: IMPLEMENTED. +5. Consumer rollout PR (Integrail parent): `secrets.pr.yaml` sweep of the four scan jobs' + keys (rotated on move) + `SC_KEY_PR` + drop `SC_CONFIG` from the four `pull_request` + scan workflows. Deploy-shaped jobs and crossguard are NOT touched here. +6. **v2 KeyProvider — DONE in this PR:** `awskms://` KMS recipient whose data-key wrap is + `kms:Encrypt`/`kms:Decrypt` under the ambient (OIDC) AWS credentials, bound via a KMS + EncryptionContext; `sc secrets scope allow --scope pr awskms://…` and `get`/`doctor`/deploy + open it keylessly. Consumer-side follow-up (out-of-repo): add the `awskms://` recipient + + `ci-oidc-pr-scan` role, then `disallow` the SSH key and delete `SC_KEY_PR`. + +Open question for review: scope-name↔environment conventions (free-form names vs +enforcing env names), and whether `doctor` should print recipient fingerprints for +audit evidence (ISO/SOC). diff --git a/docs/docs/guides/secrets-management.md b/docs/docs/guides/secrets-management.md index 59462e44..5057fa74 100644 --- a/docs/docs/guides/secrets-management.md +++ b/docs/docs/guides/secrets-management.md @@ -707,6 +707,223 @@ sc secrets allowed-keys --verbose sc secrets allowed-keys --profile github --verbose ``` +## Per-scope secrets (`secrets..yaml`) + +The whole-file store (`.sc/secrets.yaml`) is decrypted by a single master key +(`SIMPLE_CONTAINER_CONFIG`) — every CI context that needs *any* secret can read *all* of +them. **Per-scope secrets** let you carve out a named subset (a "scope") sealed to its own +recipient set, so a narrow, attacker-reachable context — typically a `pull_request`-triggered +scan job — can hold a key that decrypts only that scope. + +Scoped values live in committed, encrypted files alongside a stack's config: + +``` +.sc/ + scopes.yaml # scope -> recipient keys (governance; CODEOWNERS-gate this) + stacks// + secrets.yaml # legacy whole-file store (unchanged) + secrets.pr.yaml # scope "pr": committed encrypted, structure diffable, values opaque +``` + +A recipient is either an **SSH public key** (`ssh-ed25519` / `ssh-rsa`) or an **AWS KMS key** +(`awskms://?region=`). Each value is encrypted once under a random data key bound +to its `(stack, scope, key)`; that data key is then wrapped for every recipient. A ciphertext +therefore cannot be moved to another stack, scope, or key, and every recipient decrypts the +same value. Old `sc` binaries never read these files (they fail closed on the newer schema). + +- **SSH recipient** — the private key must be provided to decrypt (a file, a CI secret, or the + ambient config). Good for humans and for the interim CI key. +- **KMS recipient** — the data key is wrapped with `kms:Encrypt` and unwrapped with + `kms:Decrypt`, using the caller's **ambient AWS credentials**. In CI those come from an + **OIDC-federated role**, so *no private key is stored anywhere* — this is the recommended + end state for CI (see "KMS recipients" below). + +### Commands + +```bash +# Declare a scope and its recipients (edit .sc/scopes.yaml behind a CODEOWNERS gate). +# A recipient is an SSH public key OR an awskms:// URL. +sc secrets scope allow --scope pr # add an SSH recipient + reseal +sc secrets scope allow --scope pr 'awskms://alias/sc-pr?region=us-east-1' # add a KMS recipient + reseal +sc secrets scope disallow --scope pr # remove a recipient + reseal (then rotate values!) + +# Manage values in a scope. +sc secrets scope set --scope pr -s KEY # or omit value / pass '-' to read stdin +sc secrets scope get --scope pr -s KEY +sc secrets scope list --scope pr -s +sc secrets scope delete --scope pr -s KEY + +# Guardrails. +sc secrets scope lint # CI gate: encryption shape, recipient drift, scope/stack binding, duplicate keys +sc secrets scope doctor # which scopes the current key can open +``` + +Commit only the encrypted `secrets..yaml` and `scopes.yaml`; never the legacy +plaintext `stacks/*/secrets.yaml`. + +### Using a scope key in CI + +Give the job the scope's private key and it decrypts only that scope — no +`SIMPLE_CONTAINER_CONFIG` required. `get` (and deploy-time `${secret:}` resolution) look up +the decryption key in order: `--key-file`, then env `SC_KEY_` (e.g. `SC_KEY_PR`) or the +generic `SC_SCOPE_KEY`, then the ambient `SIMPLE_CONTAINER_CONFIG`. + +```yaml +# a pull_request scan job — holds ONLY the pr-scope key +env: + SC_KEY_PR: ${{ secrets.SC_KEY_PR }} # ssh-ed25519 private key, recipient of the "pr" scope only +steps: + - run: DD_KEY=$(sc secrets scope get --scope pr -s integrail defectdojo-api-key) +``` + +At deploy time, `${secret:KEY}` and `sc stack secret-get` transparently include every scope +the job's key can open, merged over the whole-file store (the legacy store wins on conflict; +`sc secrets scope lint` rejects a key that appears in two scopes or in both a scope and the +legacy store). A key that is **not** a recipient of a scope cannot decrypt it — the +`pull_request` clamp is cryptographic, not a config flag. + +> **Rotation:** `disallow` re-encrypts current files but does NOT rewrite git history — that +> recipient can still read previously committed versions. Always rotate the scope's values +> after removing a recipient. + +### Runbook: a leaked scope key + +`disallow` stops a removed recipient from reading *future* commits, but the values it could +already read must be treated as compromised — git history is immutable. To revoke a leaked +scope key (SSH or KMS): + +```bash +# 1. Remove the leaked recipient from the scope (reseals current files to the rest). +sc secrets scope disallow --scope pr + +# 2. ROTATE every value in the scope at its source, then re-set it — the old ciphertext in +# git history is still readable by the leaked key, so the values themselves must change. +sc secrets scope list --scope pr -s # enumerate what to rotate +# for each KEY: rotate upstream (new API token, etc.), then: +sc secrets scope set --scope pr -s KEY + +# 3. Commit the resealed scope file(s) + scopes.yaml. +git add .sc/scopes.yaml .sc/stacks//secrets.*.yaml && git commit -S -s -m "rotate pr scope after key revocation" +``` + +Then, depending on the recipient kind: + +- **SSH scope key (`SC_KEY_PR`)** — delete/replace the GitHub secret; the old private key can + no longer decrypt the rotated values (and is no longer a recipient going forward). +- **KMS recipient** — the "key" is the IAM permission to call `kms:Decrypt`, not a stored + secret. Revoke it by removing `kms:Decrypt` from the compromised role/principal (or, if the + KMS key itself is suspect, schedule key deletion / rotate to a new key and re-`allow` it). + Because decryption requires assuming the OIDC role, tightening the role's trust policy or + IAM immediately cuts off access — no secret to invalidate. + +Prefer at least one **SSH break-glass recipient** on every scope so you can always reseal in +step 1 even if the KMS path is unavailable, and keep `sc secrets scope lint` a **required** CI +check so the collision/duplicate guarantees are enforced, not just advisory. + +### KMS recipients (keyless CI, no stored key) + +A stored `SC_KEY_PR` is a smaller master key, but it is still a long-lived private key sitting +in a CI secret. A **KMS recipient** removes it entirely: the scope's data keys are wrapped for +an AWS KMS key, and a CI job unwraps them with an **OIDC-federated IAM role** — so the job +proves its identity to AWS per-run and holds no secret key at all. + +The recipient string is `sc`'s canonical KMS URL — the same form the Pulumi state secrets +provider uses: + +``` +awskms:// | arn:aws:kms:...>?region= +``` + +**Add the KMS recipient** (needs `kms:Encrypt` on the key for the reseal). Keep at least one +SSH break-glass recipient so you can always reseal/recover if the KMS path is unavailable: + +```bash +# scopes.yaml already has an SSH break-glass recipient; add the KMS key beside it. +# Prefer the key ARN over an alias (see "Secure-usage rules" — an alias can be repointed). +sc secrets scope allow --scope pr 'awskms://arn:aws:kms:us-east-1:123456789012:key/abcd-1234?region=us-east-1' +sc secrets scope set --scope pr -s integrail defectdojo-api-key 'dd-…' # sealed to BOTH recipients +``` + +**The resulting scope file** carries one opaque KMS wrap per value keyed by the KMS URL, +alongside the SSH wrap — no plaintext, fully diffable structure: + +```yaml +schemaVersion: 1 +stack: integrail +scope: pr +recipients: + - awskms://arn:aws:kms:us-east-1:123456789012:key/abcd-1234?region=us-east-1 + - ssh-ed25519 AAAA…break-glass +values: + defectdojo-api-key: + ciphertext: + wraps: + "awskms://arn:aws:kms:us-east-1:123456789012:key/abcd-1234?region=us-east-1": [] + "SHA256:…break-glass": [] +``` + +**Consume it in CI with OIDC — no `SC_KEY_PR`, no `SIMPLE_CONTAINER_CONFIG`:** + +```yaml +# a pull_request scan job — federates to AWS, then decrypts only the "pr" scope +permissions: + id-token: write # required for OIDC + contents: read +steps: + - uses: aws-actions/configure-aws-credentials@v5 + with: + role-to-assume: arn:aws:iam:::role/ci-oidc-pr-scan # kms:Decrypt on the pr key only + aws-region: us-east-1 + - run: DD_KEY=$(sc secrets scope get --scope pr -s integrail defectdojo-api-key) +``` + +`get`, `doctor`, and deploy-time `${secret:}` resolution all try, in order: an explicit +`--key-file`, the `SC_KEY_` / `SC_SCOPE_KEY` env keys, the ambient +`SIMPLE_CONTAINER_CONFIG`, and finally a KMS Decrypt for any `awskms://` recipient using the +ambient AWS credentials. A value with no KMS recipient never triggers an AWS call. + +**Secure-usage rules for KMS recipients** + +- **Prefer a key ARN as the recipient, not an alias.** `awskms://alias/…` is convenient, but an + alias only pins the key *by name* — an account admin who repoints the alias changes which key + the recipient resolves to. The **key ARN** (`awskms://arn:aws:kms:::key/`) + is the durable security boundary; use it for anything sensitive and treat aliases as a + convenience for low-risk scopes. `sc` pins the `KeyId` on every decrypt, so a value wrapped + under a *different* key than the recipient names is rejected as a hard integrity error. +- **Scope the IAM role tightly — pin stack AND scope.** Allow `kms:Decrypt` on *only* the + scope's key, and constrain the encryption context. `sc` sets it to + `{sc:domain, sc:stack, sc:scope, sc:key}`, so condition on both `sc:stack` and `sc:scope`: + + ```json + "Condition": { "StringEquals": { + "kms:EncryptionContext:sc:scope": "pr", + "kms:EncryptionContext:sc:stack": "integrail" + }} + ``` + + Conditioning on `sc:scope` alone lets one role decrypt *every* stack's `pr` values wrapped + under that key — fine if one key serves one trust level and stacks are mutually trusted, but + add `sc:stack` when stacks must not read each other. KMS enforces the context server-side, so + a value transplanted to another stack/scope/key fails to decrypt. +- **Never give the PR-scan role `kms:Decrypt` on a deploy/prod key.** The whole point is that + a compromised PR job cannot reach deploy-grade secrets. Use a separate key (and role) per + trust level; a deploy scope's key stays out of the PR role's policy. +- **The clamp is cryptographic.** A role that is not a recipient of `secrets.prod.yaml` (no + `kms:Decrypt` on its key) cannot open it, regardless of anything in a PR's config. +- **The stack/scope/key are NOT secret.** They ride as the KMS EncryptionContext and are logged + in CloudTrail (desirable for audit) — do not put a secret-ish string in a stack, scope, or + key *name* expecting it to stay private. +- **How failures are classified.** A KMS `InvalidCiphertext` (mangled blob or broken + stack/scope/key binding) or `IncorrectKey` (the wrap was made under a different key than the + recipient names — a transplant) is a hard **integrity** error that fails the deploy as tamper. + `AccessDenied`, a disabled key, or no ambient credentials is a least-privilege **skip** (the + scope is simply not yours; a *missing* required `${secret:}` then fails closed downstream). A + transient KMS fault (throttle after SDK retries, `KMSInternal`, timeout) fails the deploy as + **unavailable** (retry) — never silently and never mislabeled as tamper. +- **Rotation still applies.** Removing a KMS recipient re-encrypts current files but does not + rewrite history — rotate the values, and additionally rotate/disable the KMS key material if + the key itself is the concern. + ## Summary Simple Container's secrets management provides a secure, Git-native way to handle sensitive data in your projects. Key benefits: @@ -715,5 +932,6 @@ Simple Container's secrets management provides a secure, Git-native way to handl - **Simple**: Easy-to-use commands for all secret operations - **Collaborative**: Git-based workflow for team secret sharing - **Integrated**: Seamless integration with Simple Container deployments +- **Scoped**: Per-scope recipients (SSH keys or KMS-via-OIDC) so a narrow CI context decrypts only what it needs — with the KMS path storing no key at all Use the commands outlined in this guide to implement robust secrets management in your Simple Container projects. diff --git a/go.mod b/go.mod index 997f0ccd..9d0a2ebe 100644 --- a/go.mod +++ b/go.mod @@ -15,7 +15,9 @@ require ( github.com/aws/aws-sdk-go-v2/credentials v1.19.24 github.com/aws/aws-sdk-go-v2/service/cloudtrail v1.56.4 github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs v1.75.2 + github.com/aws/aws-sdk-go-v2/service/kms v1.50.3 github.com/aws/aws-secretsmanager-caching-go/v2 v2.2.0 + github.com/aws/smithy-go v1.27.1 github.com/cloudflare/cloudflare-go v0.117.0 github.com/compose-spec/compose-go v1.20.2 github.com/containerd/platforms v0.2.1 @@ -129,14 +131,12 @@ require ( github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.18 // indirect github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.29 // indirect github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.25 // indirect - github.com/aws/aws-sdk-go-v2/service/kms v1.50.3 // indirect github.com/aws/aws-sdk-go-v2/service/s3 v1.102.2 // indirect github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.41.4 // indirect github.com/aws/aws-sdk-go-v2/service/signin v1.2.0 // indirect github.com/aws/aws-sdk-go-v2/service/sso v1.31.3 // indirect github.com/aws/aws-sdk-go-v2/service/ssooidc v1.36.6 // indirect github.com/aws/aws-sdk-go-v2/service/sts v1.43.3 // indirect - github.com/aws/smithy-go v1.27.1 // indirect github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect github.com/bazelbuild/buildtools v0.0.0-20260211083412-859bfffeef82 // indirect github.com/beorn7/perks v1.0.1 // indirect diff --git a/pkg/api/secrets/ciphers/aad_test.go b/pkg/api/secrets/ciphers/aad_test.go new file mode 100644 index 00000000..c33d2bbe --- /dev/null +++ b/pkg/api/secrets/ciphers/aad_test.go @@ -0,0 +1,109 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) Simple Container + +package ciphers + +import ( + "encoding/base64" + "strings" + "testing" + + . "github.com/onsi/gomega" +) + +// TestAAD_NilRoundTrip_LegacyCompatible proves a nil-AAD seal (the whole-file +// store's path) still round-trips, for both RSA and ed25519 recipients. +func TestAAD_NilRoundTrip_LegacyCompatible(t *testing.T) { + RegisterTestingT(t) + rpriv, rpub, err := GenerateKeyPair(2048) + Expect(err).NotTo(HaveOccurred()) + chunks, err := EncryptLargeStringWithAAD(rpub, "legacy-value", nil) + Expect(err).NotTo(HaveOccurred()) + got, err := DecryptLargeStringWithAAD(rpriv, chunks, nil) + Expect(err).NotTo(HaveOccurred()) + Expect(string(got)).To(Equal("legacy-value")) + + epriv, epub, err := GenerateEd25519KeyPair() + Expect(err).NotTo(HaveOccurred()) + echunks, err := EncryptLargeStringWithAAD(epub, "legacy-ed", nil) + Expect(err).NotTo(HaveOccurred()) + egot, err := DecryptLargeStringWithEd25519AAD(epriv, echunks, nil) + Expect(err).NotTo(HaveOccurred()) + Expect(string(egot)).To(Equal("legacy-ed")) +} + +// TestAAD_MismatchFails is the core security invariant: a value sealed under a +// specific AAD must NOT decrypt under a different (or nil) AAD, for both schemes. +func TestAAD_MismatchFails(t *testing.T) { + RegisterTestingT(t) + ctx := []byte("sc-scope-v1\x00pr\x00k") + other := []byte("sc-scope-v1\x00prod\x00k") + + // RSA (OAEP label) + rpriv, rpub, err := GenerateKeyPair(2048) + Expect(err).NotTo(HaveOccurred()) + rchunks, err := EncryptLargeStringWithAAD(rpub, "v", ctx) + Expect(err).NotTo(HaveOccurred()) + _, err = DecryptLargeStringWithAAD(rpriv, rchunks, nil) + Expect(err).To(HaveOccurred()) + _, err = DecryptLargeStringWithAAD(rpriv, rchunks, other) + Expect(err).To(HaveOccurred()) + ok, err := DecryptLargeStringWithAAD(rpriv, rchunks, ctx) + Expect(err).NotTo(HaveOccurred()) + Expect(string(ok)).To(Equal("v")) + + // ed25519 (AEAD associated data) + epriv, epub, err := GenerateEd25519KeyPair() + Expect(err).NotTo(HaveOccurred()) + echunks, err := EncryptLargeStringWithAAD(epub, "v", ctx) + Expect(err).NotTo(HaveOccurred()) + _, err = DecryptLargeStringWithEd25519AAD(epriv, echunks, nil) + Expect(err).To(HaveOccurred()) + _, err = DecryptLargeStringWithEd25519AAD(epriv, echunks, other) + Expect(err).To(HaveOccurred()) + eok, err := DecryptLargeStringWithEd25519AAD(epriv, echunks, ctx) + Expect(err).NotTo(HaveOccurred()) + Expect(string(eok)).To(Equal("v")) +} + +// TestRSAChunkFraming_ReorderTruncateSpliceFails covers P0-B: a multi-chunk RSA +// value bound with a non-nil AAD must reject reordered/dropped chunks. +func TestRSAChunkFraming_ReorderTruncateSpliceFails(t *testing.T) { + RegisterTestingT(t) + priv, pub, err := GenerateKeyPair(2048) + Expect(err).NotTo(HaveOccurred()) + aad := []byte("sc-scope-v1\x00prod\x00conn") + long := strings.Repeat("A", 200) + strings.Repeat("B", 200) // >128B ⇒ multiple chunks + chunks, err := EncryptLargeStringWithAAD(pub, long, aad) + Expect(err).NotTo(HaveOccurred()) + Expect(len(chunks)).To(BeNumerically(">=", 2), "value must span multiple RSA chunks") + + // baseline round-trip works + got, err := DecryptLargeStringWithAAD(priv, chunks, aad) + Expect(err).NotTo(HaveOccurred()) + Expect(string(got)).To(Equal(long)) + + // reorder → fail (index binding) + reordered := append([]string{}, chunks...) + reordered[0], reordered[1] = reordered[1], reordered[0] + _, err = DecryptLargeStringWithAAD(priv, reordered, aad) + Expect(err).To(HaveOccurred()) + + // truncate (drop last) → fail (count binding) + _, err = DecryptLargeStringWithAAD(priv, chunks[:len(chunks)-1], aad) + Expect(err).To(HaveOccurred()) +} + +// TestEd25519Downgrade_RejectedUnderAAD covers P0-A: a non-X25519 (legacy-shaped) +// blob must be refused when an AAD binding is expected, so a forged legacy blob +// cannot bypass the scope/key binding. +func TestEd25519Downgrade_RejectedUnderAAD(t *testing.T) { + RegisterTestingT(t) + priv, _, err := GenerateEd25519KeyPair() + Expect(err).NotTo(HaveOccurred()) + // A blob without the scx25519 magic looks like the legacy format. + fakeLegacy := base64.StdEncoding.EncodeToString(make([]byte, 60)) + _, err = DecryptLargeStringWithEd25519AAD(priv, []string{fakeLegacy}, []byte("sc-scope-v1\x00pr\x00k")) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("legacy")) +} diff --git a/pkg/api/secrets/ciphers/dek.go b/pkg/api/secrets/ciphers/dek.go new file mode 100644 index 00000000..44daebd3 --- /dev/null +++ b/pkg/api/secrets/ciphers/dek.go @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) Simple Container + +package ciphers + +import ( + "crypto/rand" + + "github.com/pkg/errors" + "golang.org/x/crypto/chacha20poly1305" +) + +// DEKSize is the length of a data-encryption key for the envelope AEAD. +const DEKSize = chacha20poly1305.KeySize + +// GenerateDEK returns a fresh random 32-byte data-encryption key. +func GenerateDEK() ([]byte, error) { + k := make([]byte, DEKSize) + if _, err := rand.Read(k); err != nil { + return nil, errors.Wrap(err, "failed to generate data key") + } + return k, nil +} + +// SealAEAD encrypts plaintext under a 32-byte key with ChaCha20-Poly1305, binding +// aad, and returns nonce(12) || ciphertext+tag. This is the envelope value blob: +// the value is encrypted ONCE with a random DEK and the DEK is wrapped per +// recipient, so all recipients see the same value (no per-recipient divergence) +// and a single whole-value MAC prevents chunk splicing. +func SealAEAD(key, plaintext, aad []byte) ([]byte, error) { + aead, err := chacha20poly1305.New(key) + if err != nil { + return nil, errors.Wrap(err, "failed to create AEAD") + } + nonce := make([]byte, chacha20poly1305.NonceSize) + if _, err := rand.Read(nonce); err != nil { + return nil, errors.Wrap(err, "failed to generate nonce") + } + ct := aead.Seal(nil, nonce, plaintext, aad) + out := make([]byte, 0, len(nonce)+len(ct)) + out = append(out, nonce...) + out = append(out, ct...) + return out, nil +} + +// OpenAEAD reverses SealAEAD; aad must match. +func OpenAEAD(key, blob, aad []byte) ([]byte, error) { + if len(blob) < chacha20poly1305.NonceSize+chacha20poly1305.Overhead { + return nil, errors.New("envelope ciphertext too short") + } + aead, err := chacha20poly1305.New(key) + if err != nil { + return nil, errors.Wrap(err, "failed to create AEAD") + } + nonce := blob[:chacha20poly1305.NonceSize] + ct := blob[chacha20poly1305.NonceSize:] + pt, err := aead.Open(nil, nonce, ct, aad) + if err != nil { + return nil, errors.Wrap(err, "failed to decrypt envelope value") + } + return pt, nil +} + +// ValidEnvelopeBlob reports whether raw is a plausibly-shaped envelope value blob +// (nonce + at least the AEAD tag), for the offline lint gate. +func ValidEnvelopeBlob(raw []byte) error { + if len(raw) < chacha20poly1305.NonceSize+chacha20poly1305.Overhead { + return errors.Errorf("envelope value blob is %d bytes, below the %d-byte minimum", len(raw), chacha20poly1305.NonceSize+chacha20poly1305.Overhead) + } + return nil +} diff --git a/pkg/api/secrets/ciphers/encryption.go b/pkg/api/secrets/ciphers/encryption.go index b9f0b19d..da30d3c0 100644 --- a/pkg/api/secrets/ciphers/encryption.go +++ b/pkg/api/secrets/ciphers/encryption.go @@ -11,6 +11,7 @@ import ( "crypto/sha512" "crypto/x509" "encoding/base64" + "encoding/binary" "encoding/pem" "strings" @@ -147,13 +148,50 @@ func ParsePublicKey(s string) (crypto.PublicKey, error) { } } +// ValidateCiphertextShape checks that a single decoded ciphertext chunk has the +// shape expected for a recipient's key type, WITHOUT a private key. It is the +// offline plaintext-leak / tamper gate used by scoped-secret lint: an RSA chunk +// must be exactly the modulus size (OAEP output is fixed-width); an ed25519 +// recipient's chunk must be an X25519 sealed box (magic prefix + minimum length). +// A plaintext value smuggled under a recipient fingerprint fails this. +func ValidateCiphertextShape(pub crypto.PublicKey, raw []byte) error { + switch k := pub.(type) { + case *rsa.PublicKey: + if len(raw) != k.Size() { + return errors.Errorf("RSA ciphertext chunk is %d bytes, expected the modulus size %d", len(raw), k.Size()) + } + case ed25519.PublicKey: + minLen := len(x25519Magic) + 1 + 32 + chacha20poly1305.NonceSize + chacha20poly1305.Overhead + if !isX25519Blob(raw) { + return errors.New("ed25519 recipient ciphertext is not an X25519 sealed box (plaintext or legacy blob?)") + } + if len(raw) < minLen { + return errors.Errorf("X25519 sealed box is %d bytes, below the %d-byte minimum", len(raw), minLen) + } + default: + return errors.Errorf("unsupported recipient key type %T", pub) + } + return nil +} + func EncryptLargeString(key crypto.PublicKey, s string) ([]string, error) { + return EncryptLargeStringWithAAD(key, s, nil) +} + +// EncryptLargeStringWithAAD is EncryptLargeString with an associated-data binding. +// aad (may be nil) is bound into the ciphertext so a blob only decrypts under the +// exact same aad: for RSA it is the OAEP label, for ed25519/X25519 it is folded +// into the AEAD associated data. Scoped secrets pass a domain-separated +// "scope\x00key" context so a value cannot be transplanted to another scope/key. +// A nil aad reproduces the exact wire format of the legacy whole-file store. +func EncryptLargeStringWithAAD(key crypto.PublicKey, s string, aad []byte) ([]string, error) { var res []string if rsaKey, ok := key.(*rsa.PublicKey); ok { chunks := lo.ChunkString(s, rsaKey.Size()/2) res = make([]string, len(chunks)) + n := len(chunks) for idx, chunk := range chunks { - encryptedData, err := rsa.EncryptOAEP(sha256.New(), rand.Reader, rsaKey, []byte(chunk), nil) + encryptedData, err := rsa.EncryptOAEP(sha256.New(), rand.Reader, rsaKey, []byte(chunk), chunkLabel(aad, idx, n)) if err != nil { return nil, errors.Wrapf(err, "failed to encrypt secret") } @@ -164,7 +202,7 @@ func EncryptLargeString(key crypto.PublicKey, s string) ([]string, error) { // (see x25519.go): the AEAD key is derived from the ECDH shared secret, // so only the holder of the private key can decrypt. The previous scheme // derived the key from the public key alone and is no longer produced. - encryptedData, err := encryptWithX25519(ed25519Key, []byte(s)) + encryptedData, err := encryptWithX25519AAD(ed25519Key, []byte(s), aad) if err != nil { return nil, errors.Wrapf(err, "failed to encrypt secret for ed25519 recipient") } @@ -176,13 +214,29 @@ func EncryptLargeString(key crypto.PublicKey, s string) ([]string, error) { } func DecryptLargeString(key *rsa.PrivateKey, chunks []string) ([]byte, error) { + return DecryptLargeStringWithAAD(key, chunks, nil) +} + +// DecryptLargeStringWithAAD is DecryptLargeString with an OAEP-label binding that +// must match what EncryptLargeStringWithAAD used (nil for legacy blobs). When aad +// is non-nil the label also binds each chunk's index and the total chunk count, so +// a reordered, dropped, or spliced chunk fails OAEP instead of silently yielding a +// permuted/truncated plaintext. +func DecryptLargeStringWithAAD(key *rsa.PrivateKey, chunks []string, aad []byte) ([]byte, error) { + // An empty chunk list under an AAD binding is a tamper signal (an attacker + // blanking a value), not a legitimately-empty secret; refuse it. The legacy + // path (nil aad) keeps the historical behavior of returning "" for no chunks. + if len(aad) > 0 && len(chunks) == 0 { + return nil, errors.New("scoped RSA value has no ciphertext chunks (blanked?)") + } + n := len(chunks) decrChunks := make([][]byte, len(chunks)) for idx, chunk := range chunks { chunkBytes, err := base64.StdEncoding.DecodeString(chunk) if err != nil { return nil, errors.Wrapf(err, "failed to decode base64 string") } - decrypted, err := rsa.DecryptOAEP(sha256.New(), rand.Reader, key, chunkBytes, nil) + decrypted, err := rsa.DecryptOAEP(sha256.New(), rand.Reader, key, chunkBytes, chunkLabel(aad, idx, n)) if err != nil { return nil, errors.Wrapf(err, "failed to decrypt secret") } @@ -193,8 +247,35 @@ func DecryptLargeString(key *rsa.PrivateKey, chunks []string) ([]byte, error) { }), "")), nil } +// chunkLabel binds an OAEP label to a chunk's position and the total chunk count. +// A nil/empty aad returns the aad unchanged so the legacy whole-file store's wire +// format (label = nil) is byte-for-byte identical; a non-empty aad yields +// aad || 0x00 || uint32(idx) || uint32(count), so single-chunk scoped values still +// carry (idx=0,count=1) and multi-chunk values become order/count-bound. +func chunkLabel(aad []byte, idx, count int) []byte { + if len(aad) == 0 { + return aad + } + out := make([]byte, 0, len(aad)+1+8) + out = append(out, aad...) + out = append(out, 0x00) + var framing [8]byte + binary.BigEndian.PutUint32(framing[0:4], uint32(idx)) + binary.BigEndian.PutUint32(framing[4:8], uint32(count)) + out = append(out, framing[:]...) + return out +} + // DecryptLargeStringWithEd25519 decrypts data encrypted with ed25519 hybrid encryption func DecryptLargeStringWithEd25519(key ed25519.PrivateKey, chunks []string) ([]byte, error) { + return DecryptLargeStringWithEd25519AAD(key, chunks, nil) +} + +// DecryptLargeStringWithEd25519AAD is DecryptLargeStringWithEd25519 with an +// associated-data binding that must match encrypt time (nil for legacy blobs). +// Legacy (pre-X25519) blobs do not carry AAD and are decrypted unbound; they are +// deprecated and non-confidential regardless (see decryptWithEd25519). +func DecryptLargeStringWithEd25519AAD(key ed25519.PrivateKey, chunks []string, aad []byte) ([]byte, error) { if len(chunks) != 1 { return nil, errors.New("ed25519 decryption expects exactly one chunk") } @@ -207,7 +288,16 @@ func DecryptLargeStringWithEd25519(key ed25519.PrivateKey, chunks []string) ([]b // are NOT confidential (the key was derived from public data); rotate any // secret ever stored in one. if isX25519Blob(chunkBytes) { - return decryptWithX25519(key, chunkBytes) + return decryptWithX25519AAD(key, chunkBytes, aad) + } + // Downgrade guard: the legacy scheme derives its key from public data and + // ignores associated data, so a legacy blob is forgeable and cannot satisfy an + // AAD binding. Any caller that supplies an aad (e.g. the scoped store binding a + // value to scope+key) must therefore refuse legacy blobs — otherwise an + // attacker could inject a forged legacy blob to bypass the binding. The + // aad-less legacy/whole-file path still reads legacy blobs for migration. + if len(aad) > 0 { + return nil, errors.New("refusing to decrypt a legacy (non-X25519) ed25519 blob under an AAD binding: legacy blobs predate authenticated binding and are forgeable — re-seal the value") } return decryptWithEd25519(key, chunkBytes) } diff --git a/pkg/api/secrets/ciphers/x25519.go b/pkg/api/secrets/ciphers/x25519.go index 3c2d07bb..1a180655 100644 --- a/pkg/api/secrets/ciphers/x25519.go +++ b/pkg/api/secrets/ciphers/x25519.go @@ -57,8 +57,19 @@ func ed25519PrivateKeyToX25519(priv ed25519.PrivateKey) []byte { // decrypt — unlike the removed legacy scheme, whose key was derived from the // public key alone (and was therefore decryptable by anyone with read access). // +// extraAAD is folded into the AEAD associated data on top of the format binding +// (magic|version|ephPub). Callers pass a domain-separated context (e.g. a scoped +// secret's "scope\x00key") so a sealed value cannot be transplanted to a different +// scope or key without failing Open. Pass nil for the legacy/whole-file store — the +// resulting blob is byte-identical to the pre-AAD format. +// // Blob layout: magic(8) | version(1) | ephPub(32) | nonce(12) | ciphertext. func encryptWithX25519(recipientEd25519Pub ed25519.PublicKey, plaintext []byte) ([]byte, error) { + return encryptWithX25519AAD(recipientEd25519Pub, plaintext, nil) +} + +// encryptWithX25519AAD is encryptWithX25519 with caller-supplied context binding. +func encryptWithX25519AAD(recipientEd25519Pub ed25519.PublicKey, plaintext, extraAAD []byte) ([]byte, error) { xPub, err := ed25519PublicKeyToX25519(recipientEd25519Pub) if err != nil { return nil, err @@ -89,7 +100,7 @@ func encryptWithX25519(recipientEd25519Pub ed25519.PublicKey, plaintext []byte) if _, err := rand.Read(nonce); err != nil { return nil, errors.Wrap(err, "failed to generate nonce") } - ct := aead.Seal(nil, nonce, plaintext, x25519AAD(ephPub)) + ct := aead.Seal(nil, nonce, plaintext, x25519AAD(ephPub, extraAAD)) blob := make([]byte, 0, len(x25519Magic)+1+len(ephPub)+len(nonce)+len(ct)) blob = append(blob, x25519Magic...) @@ -101,8 +112,15 @@ func encryptWithX25519(recipientEd25519Pub ed25519.PublicKey, plaintext []byte) } // decryptWithX25519 reverses encryptWithX25519 using the recipient's Ed25519 -// private key (converted to X25519). +// private key (converted to X25519). extraAAD must match what was passed at +// encrypt time (nil for legacy/whole-file blobs). func decryptWithX25519(recipientEd25519Priv ed25519.PrivateKey, blob []byte) ([]byte, error) { + return decryptWithX25519AAD(recipientEd25519Priv, blob, nil) +} + +// decryptWithX25519AAD is decryptWithX25519 with caller-supplied context binding +// that must match encrypt time. +func decryptWithX25519AAD(recipientEd25519Priv ed25519.PrivateKey, blob, extraAAD []byte) ([]byte, error) { // ed25519.PrivateKey.Seed() panics on a wrong-sized key; fail safely instead. if len(recipientEd25519Priv) != ed25519.PrivateKeySize { return nil, errors.Errorf("invalid ed25519 private key size: %d", len(recipientEd25519Priv)) @@ -149,7 +167,7 @@ func decryptWithX25519(recipientEd25519Priv ed25519.PrivateKey, blob []byte) ([] if err != nil { return nil, errors.Wrap(err, "failed to create AEAD") } - pt, err := aead.Open(nil, nonce, ct, x25519AAD(ephPub)) + pt, err := aead.Open(nil, nonce, ct, x25519AAD(ephPub, extraAAD)) if err != nil { return nil, errors.Wrap(err, "failed to decrypt data") } @@ -174,11 +192,15 @@ func deriveX25519Key(shared, ephPub, recipientXPub []byte) ([]byte, error) { // x25519AAD binds the format magic, version, and ephemeral public key into the // AEAD's associated data so none of them can be altered without failing Open. -func x25519AAD(ephPub []byte) []byte { - aad := make([]byte, 0, len(x25519Magic)+1+len(ephPub)) +// extraAAD (may be nil) is appended for caller-supplied context binding; a nil +// extraAAD yields exactly the pre-AAD associated data, keeping legacy blobs +// bit-for-bit compatible. +func x25519AAD(ephPub, extraAAD []byte) []byte { + aad := make([]byte, 0, len(x25519Magic)+1+len(ephPub)+len(extraAAD)) aad = append(aad, x25519Magic...) aad = append(aad, x25519Version) aad = append(aad, ephPub...) + aad = append(aad, extraAAD...) return aad } diff --git a/pkg/api/secrets/scoped/crypto.go b/pkg/api/secrets/scoped/crypto.go new file mode 100644 index 00000000..3d7de42b --- /dev/null +++ b/pkg/api/secrets/scoped/crypto.go @@ -0,0 +1,176 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) Simple Container + +package scoped + +import ( + "crypto/ed25519" + "crypto/rsa" + "encoding/base64" + + "github.com/pkg/errors" + "golang.org/x/crypto/ssh" + + "github.com/simple-container-com/api/pkg/api/secrets" + "github.com/simple-container-com/api/pkg/api/secrets/ciphers" +) + +// aadDomain domain-separates scoped-secret associated data from any other use of +// the underlying cipher. Bumping it is a breaking format change. +const aadDomain = "sc-scope-v1" + +// valueAAD binds a sealed value to its (stack, scope, key) so a ciphertext blob +// cannot be transplanted into another scope file, another stack's file, or moved +// onto another key without failing AEAD/OAEP verification on decrypt. The NUL +// separators cannot appear in a stack/scope name or key (all validated to a +// restricted charset), so the concatenation is unambiguous. The KMS recipient path +// binds the same fields via a KMS EncryptionContext (see kmsEncryptionContext). +func valueAAD(stack, scope, key string) []byte { + return []byte(aadDomain + "\x00" + stack + "\x00" + scope + "\x00" + key) +} + +// recipientFingerprint returns the stable SHA256 SSH fingerprint of an authorized +// public key (e.g. "SHA256:abc…"). It is used as the per-recipient map key inside a +// scope file for SSH recipients: readable, order-independent, and derivable from a +// private key so a decryptor can find its own slot. KMS recipients use their +// normalized awskms:// URL instead — see recipientID. +// +// This funnels through the same ssh.ParseAuthorizedKey that ciphers.ParsePublicKey +// uses, so the fingerprint (identity) and the encryptability check +// (validateEncryptableRecipient, via ParsePublicKey) never disagree on which keys +// parse; the latter is the sole authority on whether a parsed key can actually +// receive a secret (rsa/ed25519 only), enforced at `allow` time. +func recipientFingerprint(authorizedKey string) (string, error) { + pub, err := parseAuthorizedKey(authorizedKey) + if err != nil { + return "", err + } + return ssh.FingerprintSHA256(pub), nil +} + +// recipientID returns the stable wrap-slot identity for any recipient: the SHA256 +// SSH fingerprint for an ssh-* key, or the normalized awskms:// URL for a KMS key. +// It is offline-derivable for both kinds, so recipient-drift lint and dedup work +// without a private key or a KMS call. +func recipientID(recipient string) (string, error) { + if isKMSRecipient(recipient) { + r, err := parseKMSRecipient(recipient) + if err != nil { + return "", err + } + return r.raw, nil + } + return recipientFingerprint(recipient) +} + +// validateEncryptableRecipient rejects recipients that fingerprint fine but cannot +// actually receive a scoped secret. SSH recipients must be ssh-rsa or ssh-ed25519 +// (the cipher layer supports no others); KMS recipients must be a well-formed +// awskms:// URL. Caught at governance time (`allow`) rather than failing later on +// the first `set`. +func validateEncryptableRecipient(recipient string) error { + if isKMSRecipient(recipient) { + return errors.Wrap(validateKMSRecipient(recipient), "unusable KMS recipient") + } + pub, err := ciphers.ParsePublicKey(secrets.TrimPubKey(recipient)) + if err != nil { + return errors.Wrap(err, "unusable recipient key") + } + switch pub.(type) { + case *rsa.PublicKey, ed25519.PublicKey: + return nil + default: + return errors.Errorf("unsupported recipient key type %T (only ssh-rsa, ssh-ed25519, and awskms:// can receive scoped secrets)", pub) + } +} + +// parseAuthorizedKey parses one " [comment]" authorized-key line into +// an ssh.PublicKey. +func parseAuthorizedKey(authorizedKey string) (ssh.PublicKey, error) { + pub, _, _, _, err := ssh.ParseAuthorizedKey([]byte(authorizedKey)) + if err != nil { + return nil, errors.Wrapf(err, "failed to parse recipient public key") + } + return pub, nil +} + +// encryptForRecipients builds the envelope for value: it is AEAD-encrypted ONCE +// under a fresh random data key (bound to stack/scope/key), and that data key is +// wrapped per recipient (keyed by recipient ID — SSH fingerprint or awskms:// URL). +// Because the value ciphertext is shared, every recipient decrypts the SAME +// plaintext — a tampered per-recipient slot yields a decrypt failure, never a +// different value — and the whole value is one AEAD blob (no chunk splicing). Every +// recipient must wrap successfully — a partial result is never returned. A KMS +// recipient's wrap calls kms:Encrypt, so the operator needs that permission. +func encryptForRecipients(recipients []string, stack, scope, key, value string) (EncryptedValue, error) { + aad := valueAAD(stack, scope, key) + dek, err := ciphers.GenerateDEK() + if err != nil { + return EncryptedValue{}, err + } + blob, err := ciphers.SealAEAD(dek, []byte(value), aad) + if err != nil { + return EncryptedValue{}, errors.Wrapf(err, "failed to seal value %q in scope %q", key, scope) + } + wraps := make(map[string][]string, len(recipients)) + for _, rk := range recipients { + id, err := recipientID(rk) + if err != nil { + return EncryptedValue{}, err + } + if _, dup := wraps[id]; dup { + return EncryptedValue{}, errors.Errorf("duplicate recipient %s in scope %q", id, scope) + } + var wrapped []string + if isKMSRecipient(rk) { + // The 32-byte DEK is wrapped by KMS; the (stack,scope,key) binding rides + // as a KMS EncryptionContext, so the wrap cannot be moved to another value. + wrapped, err = wrapDEKKMS(rk, dek, stack, scope, key) + } else { + cryptoPub, perr := ciphers.ParsePublicKey(secrets.TrimPubKey(rk)) + if perr != nil { + return EncryptedValue{}, errors.Wrapf(perr, "failed to parse recipient %s", id) + } + // The DEK is 32 bytes → always a single RSA-OAEP block / X25519 box, so the + // wrap is never chunked. The wrap is AAD-bound too, so it cannot be moved to + // another (stack,scope,key). + wrapped, err = ciphers.EncryptLargeStringWithAAD(cryptoPub, string(dek), aad) + } + if err != nil { + return EncryptedValue{}, errors.Wrapf(err, "failed to wrap data key for recipient %s", id) + } + wraps[id] = wrapped + } + if len(wraps) == 0 { + return EncryptedValue{}, errors.Errorf("scope %q has no recipients to encrypt %q for", scope, key) + } + return EncryptedValue{Ciphertext: base64.StdEncoding.EncodeToString(blob), Wraps: wraps}, nil +} + +// privateKeyFingerprint parses an unencrypted PEM private key and returns its +// public SSH fingerprint plus the parsed key. Passphrase-protected keys are +// rejected with a clear error (CI scope keys are provisioned unencrypted). +func privateKeyFingerprint(privateKey string) (string, any, error) { + raw, err := ssh.ParseRawPrivateKey([]byte(privateKey)) + if err != nil { + if _, isMissing := err.(*ssh.PassphraseMissingError); isMissing { + return "", nil, errors.New("scope private key is passphrase-protected; scoped CI keys must be unencrypted") + } + return "", nil, errors.Wrap(err, "failed to parse scope private key") + } + var sshPub ssh.PublicKey + switch k := raw.(type) { + case *rsa.PrivateKey: + sshPub, err = ssh.NewPublicKey(&k.PublicKey) + case ed25519.PrivateKey: + sshPub, err = ssh.NewPublicKey(k.Public()) + case *ed25519.PrivateKey: + sshPub, err = ssh.NewPublicKey(k.Public()) + default: + return "", nil, errors.Errorf("unsupported private key type %T", raw) + } + if err != nil { + return "", nil, errors.Wrap(err, "failed to derive public key from scope private key") + } + return ssh.FingerprintSHA256(sshPub), raw, nil +} diff --git a/pkg/api/secrets/scoped/kms.go b/pkg/api/secrets/scoped/kms.go new file mode 100644 index 00000000..99216dcf --- /dev/null +++ b/pkg/api/secrets/scoped/kms.go @@ -0,0 +1,246 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) Simple Container + +package scoped + +import ( + "context" + "encoding/base64" + "fmt" + "net/url" + "strings" + "time" + + "github.com/aws/aws-sdk-go-v2/aws" + "github.com/aws/aws-sdk-go-v2/config" + "github.com/aws/aws-sdk-go-v2/service/kms" + kmstypes "github.com/aws/aws-sdk-go-v2/service/kms/types" + smithy "github.com/aws/smithy-go" + "github.com/pkg/errors" +) + +// kmsRecipientScheme prefixes an AWS KMS recipient in scopes.yaml. The rest is +// sc's canonical KMS key URL — the SAME form the Pulumi state secrets provider +// already uses (see aws.SecretsProviderConfig.KeyName): +// +// awskms://?region= +// +// A KMS recipient is the v2 "KeyProvider": instead of sealing the data key to a +// static SSH key, the data key is wrapped with kms:Encrypt and unwrapped with +// kms:Decrypt using the ambient AWS credential chain — which in CI is an +// OIDC-federated role, so no private key is stored in a GitHub secret. It is +// purely additive: a scope with no awskms:// recipient never constructs a KMS +// client and never touches AWS. +const kmsRecipientScheme = "awskms://" + +// kmsCallTimeout bounds a single KMS Encrypt/Decrypt (and the ambient-credential +// resolution that precedes it) so a stalled credential probe cannot hang a deploy. +const kmsCallTimeout = 30 * time.Second + +// kmsMinCiphertextLen is a conservative lower bound on a real KMS ciphertext blob +// for a 32-byte data key (KMS wraps carry key metadata and are well over 100 bytes). +// It is the offline lint gate: it rejects a raw/plaintext data key (32 bytes) or a +// short string smuggled into a KMS wrap slot without a private key or a KMS call. +const kmsMinCiphertextLen = 64 + +// kmsAPI is the subset of the AWS KMS client the scoped store uses. It is an +// interface purely so unit tests inject a fake — no unit test ever calls AWS. +type kmsAPI interface { + Encrypt(ctx context.Context, in *kms.EncryptInput, optFns ...func(*kms.Options)) (*kms.EncryptOutput, error) + Decrypt(ctx context.Context, in *kms.DecryptInput, optFns ...func(*kms.Options)) (*kms.DecryptOutput, error) +} + +// newKMSClient builds a KMS client for region using the ambient AWS credential +// chain (env vars, OIDC web-identity token, shared config, or instance profile). +// It is a package var so tests can override it; it is only ever invoked when a +// scope actually has an awskms:// recipient. +var newKMSClient = func(ctx context.Context, region string) (kmsAPI, error) { + cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion(region)) + if err != nil { + return nil, errors.Wrap(err, "failed to load AWS config for KMS") + } + return kms.NewFromConfig(cfg), nil +} + +// kmsRecipient is a parsed awskms:// recipient. +type kmsRecipient struct { + raw string // normalized "awskms://?region=" — the wrap-slot map key + keyID string // key id, "alias/", or full ARN + region string +} + +// isKMSRecipient reports whether a scopes.yaml recipient entry is an awskms:// URL. +func isKMSRecipient(recipient string) bool { + return strings.HasPrefix(strings.TrimSpace(recipient), kmsRecipientScheme) +} + +// isKMSRecipientID reports whether a wrap-slot map key is a KMS recipient (vs an +// SSH "SHA256:…" fingerprint). +func isKMSRecipientID(id string) bool { + return strings.HasPrefix(id, kmsRecipientScheme) +} + +// parseKMSRecipient parses and normalizes an awskms:// recipient. The region is +// taken from the ?region= query, or derived from a full key ARN when omitted. The +// normalized raw form is stable and offline-derivable, so it is used as the wrap +// map key and for recipient-drift lint without any KMS call. +func parseKMSRecipient(recipient string) (kmsRecipient, error) { + raw := strings.TrimSpace(recipient) + if !strings.HasPrefix(raw, kmsRecipientScheme) { + return kmsRecipient{}, errors.Errorf("not a KMS recipient: %q", recipient) + } + rest := strings.TrimPrefix(raw, kmsRecipientScheme) + keyID := rest + region := "" + if i := strings.IndexByte(rest, '?'); i >= 0 { + keyID = rest[:i] + q, err := url.ParseQuery(rest[i+1:]) + if err != nil { + return kmsRecipient{}, errors.Wrapf(err, "invalid query in KMS recipient %q", recipient) + } + region = q.Get("region") + } + keyID = strings.TrimSpace(keyID) + if keyID == "" { + return kmsRecipient{}, errors.Errorf("KMS recipient %q has no key id", recipient) + } + if region == "" && strings.HasPrefix(keyID, "arn:") { + // arn::kms:::key/ — match structurally so the + // aws-us-gov and aws-cn partitions derive their region too, not just aws. + if parts := strings.Split(keyID, ":"); len(parts) >= 6 && parts[0] == "arn" && parts[2] == "kms" { + region = parts[3] + } + } + if region == "" { + return kmsRecipient{}, errors.Errorf("KMS recipient %q has no region (use awskms://?region=)", recipient) + } + return kmsRecipient{raw: normalizeKMSRecipient(keyID, region), keyID: keyID, region: region}, nil +} + +// normalizeKMSRecipient renders the canonical wrap-slot identity for a KMS key. +func normalizeKMSRecipient(keyID, region string) string { + return fmt.Sprintf("%s%s?region=%s", kmsRecipientScheme, keyID, region) +} + +// kmsEncryptionContext binds a KMS-wrapped data key to its (stack, scope, key) — +// the same domain-separated fields the SSH path binds via byte-AAD, expressed as a +// KMS EncryptionContext so the binding is enforced by KMS server-side (a Decrypt +// with a different context fails as InvalidCiphertext). These are non-secret +// identifiers; they are logged in CloudTrail, which is desirable for audit. +func kmsEncryptionContext(stack, scope, key string) map[string]string { + return map[string]string{ + "sc:domain": aadDomain, + "sc:stack": stack, + "sc:scope": scope, + "sc:key": key, + } +} + +// wrapDEKKMS wraps a data key for a KMS recipient. The operator running set/allow +// needs kms:Encrypt on the key. Returns a single-element slice (the base64 KMS +// ciphertext blob) to match the SSH wrap's []string shape. +func wrapDEKKMS(recipient string, dek []byte, stack, scope, key string) ([]string, error) { + r, err := parseKMSRecipient(recipient) + if err != nil { + return nil, err + } + ctx, cancel := context.WithTimeout(context.Background(), kmsCallTimeout) + defer cancel() + cli, err := newKMSClient(ctx, r.region) + if err != nil { + return nil, err + } + out, err := cli.Encrypt(ctx, &kms.EncryptInput{ + KeyId: aws.String(r.keyID), + Plaintext: dek, + EncryptionContext: kmsEncryptionContext(stack, scope, key), + }) + if err != nil { + return nil, errors.Wrapf(err, "KMS encrypt for recipient %s (need kms:Encrypt on the key)", r.raw) + } + return []string{base64.StdEncoding.EncodeToString(out.CiphertextBlob)}, nil +} + +// decryptKMSWrap unwraps a KMS-wrapped data key with a caller-provided client (the +// Opener caches the client + its credential probe). Its three outcomes map onto the +// resolver's semantics — and, critically, keep them position-independent so a +// per-call KMS hiccup on a later value is never mistaken for a stripped wrap: +// +// - (dek, true, nil) — decrypted; the caller is a recipient (has kms:Decrypt). +// - (nil, false, nil) — NOT a recipient here: AccessDenied / NotFound / disabled +// key / invalid key state. Least-privilege skip. +// - (nil, true, err) — INTEGRITY failure (tamper): InvalidCiphertext (broken +// EncryptionContext binding / mangled blob) or IncorrectKey (the pinned key is +// not the one that produced the ciphertext — a transplant onto a scope whose +// awskms:// recipient names a different key). The resolver hard-fails as tamper. +// - (nil, false, err) — TRANSIENT/unknown backend error (throttle after SDK +// retries, KMSInternal, timeout, network). The resolver hard-fails as +// "unavailable" (retry), distinct from tamper and never a silent skip. +func decryptKMSWrap(ctx context.Context, cli kmsAPI, r kmsRecipient, blob []byte, stack, scope, key string) ([]byte, bool, error) { + out, err := cli.Decrypt(ctx, &kms.DecryptInput{ + CiphertextBlob: blob, + KeyId: aws.String(r.keyID), // pin the key: defense against confused-deputy + EncryptionContext: kmsEncryptionContext(stack, scope, key), + }) + if err != nil { + if isKMSIntegrityError(err) { + return nil, true, errors.Wrapf(err, "KMS rejected the wrap for %s (tampered blob, wrong key, or wrong stack/scope/key binding)", r.raw) + } + if isKMSNotAuthorized(err) { + return nil, false, nil // not a recipient of this key → skip + } + return nil, false, errors.Wrapf(err, "KMS decrypt failed for %s (transient — retry)", r.raw) + } + return out.Plaintext, true, nil +} + +// isKMSIntegrityError reports whether a KMS Decrypt error means the ciphertext or its +// binding is WRONG — tamper, never a least-privilege skip. InvalidCiphertext = +// mangled blob or EncryptionContext (our AAD) mismatch; IncorrectKey = the pinned +// KeyId is not the key that produced the blob (a transplant that the KeyId pin +// caught). AWS returns IncorrectKeyException — a DISTINCT type from +// InvalidCiphertextException — so it must be matched explicitly or a transplant would +// be misread as "not a recipient". +func isKMSIntegrityError(err error) bool { + var ice *kmstypes.InvalidCiphertextException + var ike *kmstypes.IncorrectKeyException + if errors.As(err, &ice) || errors.As(err, &ike) { + return true + } + var apiErr smithy.APIError + if errors.As(err, &apiErr) { + switch apiErr.ErrorCode() { + case "InvalidCiphertextException", "IncorrectKeyException": + return true + } + } + return false +} + +// isKMSNotAuthorized reports whether a KMS Decrypt error means the caller is simply +// not a recipient of this key (or the key is not usable for them) — a least-privilege +// skip, distinct from tamper (isKMSIntegrityError) and from a transient backend fault +// (everything not matched by either, which is surfaced as a hard "unavailable"). +func isKMSNotAuthorized(err error) bool { + var nfe *kmstypes.NotFoundException + var dis *kmstypes.DisabledException + var invState *kmstypes.KMSInvalidStateException + if errors.As(err, &nfe) || errors.As(err, &dis) || errors.As(err, &invState) { + return true + } + var apiErr smithy.APIError + if errors.As(err, &apiErr) { + switch apiErr.ErrorCode() { + case "AccessDeniedException", "NotFoundException", "DisabledException", "KMSInvalidStateException": + return true + } + } + return false +} + +// validateKMSRecipient checks an awskms:// recipient parses (used at governance +// time so a malformed KMS URL is rejected on `allow`, not on first `set`). +func validateKMSRecipient(recipient string) error { + _, err := parseKMSRecipient(recipient) + return err +} diff --git a/pkg/api/secrets/scoped/kms_test.go b/pkg/api/secrets/scoped/kms_test.go new file mode 100644 index 00000000..468c000e --- /dev/null +++ b/pkg/api/secrets/scoped/kms_test.go @@ -0,0 +1,587 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) Simple Container + +package scoped + +import ( + "context" + "encoding/base64" + "encoding/json" + "errors" + "os" + "reflect" + "strings" + "testing" + + "github.com/aws/aws-sdk-go-v2/aws" + "github.com/aws/aws-sdk-go-v2/service/kms" + kmstypes "github.com/aws/aws-sdk-go-v2/service/kms/types" + smithy "github.com/aws/smithy-go" +) + +// fakeKMSBlob is the in-memory stand-in for a KMS ciphertext: it carries the key +// id, the EncryptionContext, and the plaintext so the fake Decrypt can enforce the +// same invariants a real KMS would (key match + context match). +type fakeKMSBlob struct { + KeyID string `json:"k"` + EncCtx map[string]string `json:"c"` + PT []byte `json:"p"` + Pad string `json:"pad"` // keep the blob comfortably over kmsMinCiphertextLen +} + +// fakeKMS implements kmsAPI. allow is the set of key ids the current "role" may +// Decrypt; an unlisted key returns AccessDenied (the "not a recipient" case). +// Encrypt always succeeds (any operator can wrap for a key they name). If decryptErr +// is set, Decrypt returns it unconditionally (used to model transient/error classes). +type fakeKMS struct { + allow map[string]bool + decryptErr error +} + +func (f *fakeKMS) Encrypt(_ context.Context, in *kms.EncryptInput, _ ...func(*kms.Options)) (*kms.EncryptOutput, error) { + raw, err := json.Marshal(fakeKMSBlob{ + KeyID: aws.ToString(in.KeyId), + EncCtx: in.EncryptionContext, + PT: in.Plaintext, + Pad: strings.Repeat("x", 96), + }) + if err != nil { + return nil, err + } + return &kms.EncryptOutput{CiphertextBlob: raw, KeyId: in.KeyId}, nil +} + +func (f *fakeKMS) Decrypt(_ context.Context, in *kms.DecryptInput, _ ...func(*kms.Options)) (*kms.DecryptOutput, error) { + if f.decryptErr != nil { + return nil, f.decryptErr + } + var blob fakeKMSBlob + if err := json.Unmarshal(in.CiphertextBlob, &blob); err != nil { + return nil, &kmstypes.InvalidCiphertextException{} // mangled ciphertext + } + if kid := aws.ToString(in.KeyId); kid != "" && kid != blob.KeyID { + // KeyId pin names a different key than produced the blob: real AWS KMS returns + // IncorrectKeyException here (a DISTINCT type from InvalidCiphertext). + return nil, &kmstypes.IncorrectKeyException{} + } + if f.allow != nil && !f.allow[blob.KeyID] { + return nil, &smithy.GenericAPIError{Code: "AccessDeniedException", Message: "not authorized to use " + blob.KeyID} + } + if !reflect.DeepEqual(blob.EncCtx, in.EncryptionContext) { + return nil, &kmstypes.InvalidCiphertextException{} // EncryptionContext (our AAD) mismatch + } + return &kms.DecryptOutput{Plaintext: blob.PT, KeyId: aws.String(blob.KeyID)}, nil +} + +// withFakeKMS installs a fake KMS client for the duration of a test, restoring the +// real factory on cleanup. No unit test ever reaches AWS. +func withFakeKMS(t *testing.T, allow ...string) { + t.Helper() + set := map[string]bool{} + for _, k := range allow { + set[k] = true + } + prev := newKMSClient + newKMSClient = func(_ context.Context, _ string) (kmsAPI, error) { + return &fakeKMS{allow: set}, nil + } + t.Cleanup(func() { newKMSClient = prev }) +} + +// withFakeKMSDecryptErr installs a fake whose Decrypt always returns err (models a +// specific KMS error class); Encrypt still succeeds so a value can be sealed first. +func withFakeKMSDecryptErr(t *testing.T, err error) { + t.Helper() + prev := newKMSClient + newKMSClient = func(_ context.Context, _ string) (kmsAPI, error) { + return &fakeKMS{decryptErr: err}, nil + } + t.Cleanup(func() { newKMSClient = prev }) +} + +const testKMSKeyID = "alias/sc-test-pr" + +func testKMSRecipient() string { + return "awskms://" + testKMSKeyID + "?region=us-east-1" +} + +func TestKMSRecipient_ParseNormalize(t *testing.T) { + cases := []struct { + in string + wantID string + wantErr bool + wantNorm string + }{ + {in: "awskms://alias/foo?region=eu-central-1", wantID: "alias/foo", wantNorm: "awskms://alias/foo?region=eu-central-1"}, + {in: "awskms://arn:aws:kms:us-west-2:123456789012:key/abcd-ef", wantID: "arn:aws:kms:us-west-2:123456789012:key/abcd-ef", wantNorm: "awskms://arn:aws:kms:us-west-2:123456789012:key/abcd-ef?region=us-west-2"}, + {in: "awskms://alias/foo", wantErr: true}, // no region and not an ARN + {in: "awskms://?region=us-east-1", wantErr: true}, // no key id + {in: "ssh-ed25519 AAAA...", wantErr: true}, // not a KMS recipient + } + for _, c := range cases { + r, err := parseKMSRecipient(c.in) + if c.wantErr { + if err == nil { + t.Errorf("parseKMSRecipient(%q): expected error, got %+v", c.in, r) + } + continue + } + if err != nil { + t.Errorf("parseKMSRecipient(%q): unexpected error %v", c.in, err) + continue + } + if r.keyID != c.wantID { + t.Errorf("parseKMSRecipient(%q): keyID=%q want %q", c.in, r.keyID, c.wantID) + } + if r.raw != c.wantNorm { + t.Errorf("parseKMSRecipient(%q): raw=%q want %q", c.in, r.raw, c.wantNorm) + } + // recipientID must equal the normalized form (used as the wrap-slot key). + id, err := recipientID(c.in) + if err != nil || id != c.wantNorm { + t.Errorf("recipientID(%q)=%q,%v want %q", c.in, id, err, c.wantNorm) + } + } +} + +func TestScopeFile_RoundTrip_KMS(t *testing.T) { + withFakeKMS(t, testKMSKeyID) + rec := testKMSRecipient() + f, err := NewScopeFile("mystack", "pr", []string{rec}) + if err != nil { + t.Fatal(err) + } + if err := f.Set("API_KEY", "s3cr3t"); err != nil { + t.Fatalf("Set: %v", err) + } + if err := f.VerifyConsistency(); err != nil { + t.Fatalf("VerifyConsistency: %v", err) + } + // The wrap slot is keyed by the normalized KMS URL, not an SSH fingerprint. + if _, ok := f.Values["API_KEY"].Wraps[rec]; !ok { + t.Fatalf("expected a KMS wrap slot keyed by %q; got %v", rec, f.Values["API_KEY"].Wraps) + } + // Opened via ambient KMS (no SSH key at all). + val, owned, err := f.Open("API_KEY", NewOpener(nil, true)) + if err != nil || !owned || val != "s3cr3t" { + t.Fatalf("Open via KMS = %q,%v,%v; want s3cr3t,true,nil", val, owned, err) + } + // KMS disabled → not openable (not owned, no error). + if _, owned, err := f.Open("API_KEY", NewOpener(nil, false)); owned || err != nil { + t.Fatalf("Open with KMS disabled = owned=%v err=%v; want not-owned, nil", owned, err) + } +} + +func TestKMS_AccessDeniedIsNotOwned(t *testing.T) { + withFakeKMS(t /* allow nothing → every Decrypt is AccessDenied */) + f, _ := NewScopeFile("mystack", "pr", []string{testKMSRecipient()}) + // Encrypt is allowed even when Decrypt is not, so Set succeeds. + if err := f.Set("K", "v"); err != nil { + t.Fatalf("Set: %v", err) + } + val, owned, err := f.Open("K", NewOpener(nil, true)) + if owned || err != nil { + t.Fatalf("AccessDenied should be a least-privilege skip: got val=%q owned=%v err=%v", val, owned, err) + } +} + +func TestKMS_EncryptionContextBindingIsIntegrityError(t *testing.T) { + withFakeKMS(t, testKMSKeyID) + rec := testKMSRecipient() + // Seal a value in stackA, then transplant it into a file bound to stackB. At + // decrypt the EncryptionContext is (stackB,pr,K) but the wrap was made for + // (stackA,pr,K), so KMS rejects it as InvalidCiphertext → integrity (owned+err). + fA, _ := NewScopeFile("stackA", "pr", []string{rec}) + if err := fA.Set("K", "v"); err != nil { + t.Fatal(err) + } + fB, _ := NewScopeFile("stackB", "pr", []string{rec}) + fB.Values["K"] = fA.Values["K"] + _, owned, err := fB.Open("K", NewOpener(nil, true)) + if err == nil || !owned { + t.Fatalf("cross-stack transplant should be owned+integrity error; got owned=%v err=%v", owned, err) + } +} + +func TestKMSErrorClassifiers(t *testing.T) { + integrity := []error{ + &kmstypes.InvalidCiphertextException{}, + &kmstypes.IncorrectKeyException{}, + &smithy.GenericAPIError{Code: "InvalidCiphertextException"}, + &smithy.GenericAPIError{Code: "IncorrectKeyException"}, + } + for _, e := range integrity { + if !isKMSIntegrityError(e) { + t.Errorf("expected integrity for %T/%v", e, e) + } + if isKMSNotAuthorized(e) { + t.Errorf("integrity error %T must not be classified not-authorized", e) + } + } + notAuthorized := []error{ + &kmstypes.NotFoundException{}, + &kmstypes.DisabledException{}, + &kmstypes.KMSInvalidStateException{}, + &smithy.GenericAPIError{Code: "AccessDeniedException"}, + } + for _, e := range notAuthorized { + if !isKMSNotAuthorized(e) { + t.Errorf("expected not-authorized for %T/%v", e, e) + } + if isKMSIntegrityError(e) { + t.Errorf("not-authorized error %T must not be classified integrity", e) + } + } + transient := []error{ + &smithy.GenericAPIError{Code: "KMSInternalException"}, + &smithy.GenericAPIError{Code: "ThrottlingException"}, + errors.New("dial tcp 10.0.0.1:443: i/o timeout"), + } + for _, e := range transient { + if isKMSIntegrityError(e) || isKMSNotAuthorized(e) { + t.Errorf("transient error %T must be neither integrity nor not-authorized (surfaced as unavailable)", e) + } + } +} + +func TestDecryptKMSWrap_Classification(t *testing.T) { + r, err := parseKMSRecipient(testKMSRecipient()) + if err != nil { + t.Fatal(err) + } + cases := []struct { + name string + err error + wantOwned bool + wantErr bool + }{ + {"incorrect-key integrity", &kmstypes.IncorrectKeyException{}, true, true}, + {"invalid-ciphertext integrity", &kmstypes.InvalidCiphertextException{}, true, true}, + {"access-denied skip", &smithy.GenericAPIError{Code: "AccessDeniedException"}, false, false}, + {"transient surfaced", &smithy.GenericAPIError{Code: "KMSInternalException"}, false, true}, + } + for _, c := range cases { + cli := &fakeKMS{decryptErr: c.err} + _, owned, derr := decryptKMSWrap(context.Background(), cli, r, []byte("blob"), "s", "pr", "K") + if owned != c.wantOwned || (derr != nil) != c.wantErr { + t.Errorf("%s: got owned=%v err=%v; want owned=%v err=%v", c.name, owned, derr, c.wantOwned, c.wantErr) + } + } +} + +func TestResolveScopedValues_KMSTransientIsUnavailableNotIntegrity(t *testing.T) { + withFakeKMSDecryptErr(t, &smithy.GenericAPIError{Code: "KMSInternalException"}) + dir := t.TempDir() + // Seal needs Encrypt (the fake's Encrypt succeeds); only Decrypt errors. + f, _ := NewScopeFile(baseName(dir), "pr", []string{testKMSRecipient()}) + if err := f.Set("K", "v"); err != nil { + t.Fatal(err) + } + if err := f.Save(ScopeFileNamePath(dir, "pr")); err != nil { + t.Fatal(err) + } + _, err := ResolveScopedValues(dir, nil) + if err == nil || !errors.Is(err, ErrScopedUnavailable) { + t.Fatalf("transient KMS error must be ErrScopedUnavailable, got %v", err) + } + if errors.Is(err, ErrScopedIntegrity) { + t.Fatalf("a transient error must NOT be reported as integrity/tamper: %v", err) + } +} + +func TestResolveScopedValues_MalformedKMSSlotSkipped(t *testing.T) { + withFakeKMS(t, testKMSKeyID) + dir := t.TempDir() + f, _ := NewScopeFile(baseName(dir), "pr", []string{testKMSRecipient()}) + if err := f.Set("K", "v"); err != nil { + t.Fatal(err) + } + // Relabel the wrap slot to an unparseable awskms:// id (foreign/corrupt file). + ev := f.Values["K"] + ev.Wraps = map[string][]string{"awskms://no-region-here": ev.Wraps[testKMSRecipient()]} + f.Values["K"] = ev + if err := f.Save(ScopeFileNamePath(dir, "pr")); err != nil { + t.Fatal(err) + } + // A least-privilege reader must SKIP a slot it cannot even parse, never hard-fail. + got, err := ResolveScopedValues(dir, nil) + if err != nil { + t.Fatalf("malformed foreign KMS slot must be skipped, not error: %v", err) + } + if len(got) != 0 { + t.Fatalf("expected nothing resolved, got %v", got) + } +} + +func TestResolveScopedValues_KMSNoCredentialsSkips(t *testing.T) { + withFakeKMS(t, testKMSKeyID) // seal with a working client + dir := t.TempDir() + f, _ := NewScopeFile(baseName(dir), "pr", []string{testKMSRecipient()}) + if err := f.Set("K", "v"); err != nil { + t.Fatal(err) + } + if err := f.Save(ScopeFileNamePath(dir, "pr")); err != nil { + t.Fatal(err) + } + // Now a caller with no ambient AWS credentials: client build fails → skip, no error. + prev := newKMSClient + newKMSClient = func(_ context.Context, _ string) (kmsAPI, error) { + return nil, errors.New("no ambient AWS credentials") + } + defer func() { newKMSClient = prev }() + got, err := ResolveScopedValues(dir, nil) + if err != nil { + t.Fatalf("no-credentials must be a skip, not error: %v", err) + } + if len(got) != 0 { + t.Fatalf("expected nothing resolved without KMS identity, got %v", got) + } +} + +func TestResolveScopedValues_StrippedWrapIsIntegrity(t *testing.T) { + authA, privA := genEd25519Recipient(t) + dir := t.TempDir() + f, _ := NewScopeFile(baseName(dir), "pr", []string{authA}) + if err := f.Set("A", "va"); err != nil { + t.Fatal(err) + } + if err := f.Set("B", "vb"); err != nil { + t.Fatal(err) + } + // Strip the recipient's wrap from the LATER value (Keys() is sorted: A then B), so + // value A establishes ownership and value B's missing slot reads as tamper. + fp, _ := recipientFingerprint(authA) + ev := f.Values["B"] + delete(ev.Wraps, fp) + f.Values["B"] = ev + if err := f.Save(ScopeFileNamePath(dir, "pr")); err != nil { + t.Fatal(err) + } + _, err := ResolveScopedValues(dir, []string{privA}) + if err == nil || !errors.Is(err, ErrScopedIntegrity) { + t.Fatalf("a stripped wrap on an owned file must be ErrScopedIntegrity, got %v", err) + } +} + +func TestResolveScopedValues_FirstValueStrippedIsIntegrity(t *testing.T) { + // A declared SSH recipient whose wrap is stripped from the FIRST (sorted) value + // must be a hard integrity error, not a silent "not my scope" skip — f.Recipients + // proves recipiency offline even before any value opens. + authA, privA := genEd25519Recipient(t) + dir := t.TempDir() + f, _ := NewScopeFile(baseName(dir), "pr", []string{authA}) + if err := f.Set("A", "va"); err != nil { + t.Fatal(err) + } + if err := f.Set("B", "vb"); err != nil { + t.Fatal(err) + } + fp, _ := recipientFingerprint(authA) + ev := f.Values["A"] // "A" sorts first + delete(ev.Wraps, fp) + f.Values["A"] = ev + if err := f.Save(ScopeFileNamePath(dir, "pr")); err != nil { + t.Fatal(err) + } + _, err := ResolveScopedValues(dir, []string{privA}) + if err == nil || !errors.Is(err, ErrScopedIntegrity) { + t.Fatalf("first-value stripped wrap for a declared recipient must be ErrScopedIntegrity, got %v", err) + } +} + +func TestResolveScopedValues_MixedScope_KMSRoleAndSSH(t *testing.T) { + withFakeKMS(t, testKMSKeyID) + sshAuth, sshPriv := genEd25519Recipient(t) + dir := t.TempDir() + f, _ := NewScopeFile(baseName(dir), "pr", []string{sshAuth, testKMSRecipient()}) + if err := f.Set("K", "v"); err != nil { + t.Fatal(err) + } + if err := f.Save(ScopeFileNamePath(dir, "pr")); err != nil { + t.Fatal(err) + } + // KMS role (no SSH key) resolves at deploy time. + got, err := ResolveScopedValues(dir, nil) + if err != nil || got["K"] != "v" { + t.Fatalf("KMS role deploy resolve = %v, %v; want K=v", got, err) + } + // SSH holder resolves the same value (SSH opens first — no KMS call needed). + got2, err := ResolveScopedValues(dir, []string{sshPriv}) + if err != nil || got2["K"] != "v" { + t.Fatalf("SSH holder deploy resolve = %v, %v; want K=v", got2, err) + } +} + +func TestKMSRecipient_GovCloudChinaARN(t *testing.T) { + cases := map[string]string{ + "awskms://arn:aws-us-gov:kms:us-gov-west-1:123456789012:key/abc": "us-gov-west-1", + "awskms://arn:aws-cn:kms:cn-north-1:123456789012:key/abc": "cn-north-1", + } + for in, wantRegion := range cases { + r, err := parseKMSRecipient(in) + if err != nil { + t.Errorf("parseKMSRecipient(%q): %v", in, err) + continue + } + if r.region != wantRegion { + t.Errorf("parseKMSRecipient(%q): region=%q want %q", in, r.region, wantRegion) + } + } +} + +func TestReencrypt_KMSRecipient(t *testing.T) { + withFakeKMS(t, testKMSKeyID) + sshAuth, sshPriv := genEd25519Recipient(t) + f, err := NewScopeFile("s", "pr", []string{sshAuth, testKMSRecipient()}) + if err != nil { + t.Fatal(err) + } + if err := f.Set("K", "v"); err != nil { + t.Fatal(err) + } + // Reseal to KMS-only, decrypting via a KMS-capable opener that holds NO SSH key. + if err := f.Reencrypt([]string{testKMSRecipient()}, NewOpener(nil, true)); err != nil { + t.Fatalf("KMS-only reseal (no SSH key) must succeed: %v", err) + } + if len(f.Values["K"].Wraps) != 1 { + t.Fatalf("expected 1 wrap after KMS-only reseal, got %d", len(f.Values["K"].Wraps)) + } + if v, owned, err := f.Open("K", NewOpener(nil, true)); !owned || err != nil || v != "v" { + t.Fatalf("KMS-only value should still open: %q,%v,%v", v, owned, err) + } + // Reseal back to include SSH, decrypting via KMS this time; SSH holder can then open. + if err := f.Reencrypt([]string{sshAuth, testKMSRecipient()}, NewOpener(nil, true)); err != nil { + t.Fatalf("reseal back to mixed via KMS must succeed: %v", err) + } + if v, owned, err := f.Open("K", NewOpener([]string{sshPriv}, false)); !owned || err != nil || v != "v" { + t.Fatalf("SSH holder should open after reseal: %q,%v,%v", v, owned, err) + } +} + +func TestWriteFileAtomic_PermAndNoTempLeak(t *testing.T) { + dir := t.TempDir() + p := dir + "/f.yaml" + if err := writeFileAtomic(p, []byte("data"), 0o644); err != nil { + t.Fatal(err) + } + st, err := os.Stat(p) + if err != nil { + t.Fatal(err) + } + if st.Mode().Perm() != 0o644 { + t.Fatalf("perm = %o, want 0644", st.Mode().Perm()) + } + entries, _ := os.ReadDir(dir) + for _, e := range entries { + if strings.Contains(e.Name(), ".tmp-") { + t.Fatalf("temp file leaked: %s", e.Name()) + } + } +} + +func TestScopeFile_MixedSSHAndKMS(t *testing.T) { + withFakeKMS(t, testKMSKeyID) + sshAuth, sshPriv := genEd25519Recipient(t) + recipients := []string{sshAuth, testKMSRecipient()} + f, err := NewScopeFile("mystack", "pr", recipients) + if err != nil { + t.Fatal(err) + } + if err := f.Set("SHARED", "one-value"); err != nil { + t.Fatal(err) + } + if err := f.VerifyConsistency(); err != nil { + t.Fatalf("VerifyConsistency mixed scope: %v", err) + } + // SSH holder opens (KMS disabled — proves SSH path never needs AWS). + if val, owned, err := f.Open("SHARED", NewOpener([]string{sshPriv}, false)); !owned || err != nil || val != "one-value" { + t.Fatalf("SSH open = %q,%v,%v; want one-value,true,nil", val, owned, err) + } + // KMS role opens the SAME value (no SSH key). + if val, owned, err := f.Open("SHARED", NewOpener(nil, true)); !owned || err != nil || val != "one-value" { + t.Fatalf("KMS open = %q,%v,%v; want one-value,true,nil", val, owned, err) + } +} + +func TestSameRecipients_KMS(t *testing.T) { + // ARN with explicit region vs derived region → same recipient. + a := []string{"awskms://arn:aws:kms:us-east-1:1:key/x"} + b := []string{"awskms://arn:aws:kms:us-east-1:1:key/x?region=us-east-1"} + if err := SameRecipients(a, b); err != nil { + t.Errorf("ARN region-derivation should be equal: %v", err) + } + // Different region → drift. + c := []string{"awskms://alias/x?region=us-east-1"} + d := []string{"awskms://alias/x?region=eu-west-1"} + if err := SameRecipients(c, d); err == nil { + t.Error("different regions must be detected as drift") + } +} + +func TestVerifyConsistency_KMS_RejectsShortBlob(t *testing.T) { + withFakeKMS(t, testKMSKeyID) + rec := testKMSRecipient() + f, _ := NewScopeFile("mystack", "pr", []string{rec}) + if err := f.Set("K", "v"); err != nil { + t.Fatal(err) + } + // Replace the KMS wrap with a too-short blob (a plaintext data key would be ~32B). + ev := f.Values["K"] + ev.Wraps[rec] = []string{base64.StdEncoding.EncodeToString([]byte("tooshort"))} + f.Values["K"] = ev + if err := f.VerifyConsistency(); err == nil || !strings.Contains(err.Error(), "minimum") { + t.Fatalf("expected KMS short-blob rejection, got %v", err) + } +} + +func TestResolveScopedValues_KMSRole(t *testing.T) { + withFakeKMS(t, testKMSKeyID) + dir := t.TempDir() + f, _ := NewScopeFile(baseName(dir), "pr", []string{testKMSRecipient()}) + if err := f.Set("PR_TOKEN", "abc123"); err != nil { + t.Fatal(err) + } + if err := f.Save(ScopeFileNamePath(dir, "pr")); err != nil { + t.Fatal(err) + } + // A role with KMS access resolves the value (no SSH keys passed). + got, err := ResolveScopedValues(dir, nil) + if err != nil { + t.Fatalf("ResolveScopedValues: %v", err) + } + if got["PR_TOKEN"] != "abc123" { + t.Fatalf("KMS-resolved value = %q; want abc123 (all=%v)", got["PR_TOKEN"], got) + } +} + +func TestResolveScopedValues_KMSAccessDeniedSkips(t *testing.T) { + withFakeKMS(t /* deny all */) + dir := t.TempDir() + f, _ := NewScopeFile(baseName(dir), "pr", []string{testKMSRecipient()}) + if err := f.Set("PR_TOKEN", "abc123"); err != nil { + t.Fatal(err) + } + if err := f.Save(ScopeFileNamePath(dir, "pr")); err != nil { + t.Fatal(err) + } + // No KMS access and no SSH key → the scope is simply not ours; empty, no error. + got, err := ResolveScopedValues(dir, nil) + if err != nil { + t.Fatalf("ResolveScopedValues should skip, not error: %v", err) + } + if len(got) != 0 { + t.Fatalf("expected no resolved values, got %v", got) + } +} + +// baseName returns the last path element (the stack name for a temp stack dir). +func baseName(dir string) string { + parts := strings.Split(strings.TrimRight(dir, "/"), "/") + return parts[len(parts)-1] +} + +// ScopeFileNamePath is a test helper: the scope file path directly inside dir +// (dir already IS the stack directory in these tests). +func ScopeFileNamePath(dir, scope string) string { + return dir + "/" + ScopeFileName(scope) +} diff --git a/pkg/api/secrets/scoped/opener.go b/pkg/api/secrets/scoped/opener.go new file mode 100644 index 00000000..44862ef0 --- /dev/null +++ b/pkg/api/secrets/scoped/opener.go @@ -0,0 +1,208 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) Simple Container + +package scoped + +import ( + "context" + "crypto/ed25519" + "crypto/rsa" + "encoding/base64" + "strings" + + "github.com/pkg/errors" + + "github.com/simple-container-com/api/pkg/api/secrets/ciphers" +) + +// Opener holds the decryption material a caller has and knows how to open an +// envelope value with any of it: a set of candidate SSH private keys (indexed by +// fingerprint) and, optionally, permission to attempt KMS Decrypt via the ambient +// AWS credential chain. It is the single decrypt path for both the CLI (`get`, +// `doctor`) and deploy-time resolution, so SSH and KMS recipients behave +// identically everywhere. An Opener is single-use per resolve/CLI run and is NOT +// safe for concurrent use (it lazily caches KMS clients). +type Opener struct { + sshByFP map[string]any // fingerprint -> *rsa.PrivateKey | ed25519.PrivateKey + kmsAllowed bool + + // kmsClients caches one KMS client per region so a deploy with many scoped + // values does not rebuild the client (and re-probe credentials) per value. + kmsClients map[string]kmsAPI + // kmsNoIdentity is a one-shot breaker: once building a KMS client fails (no + // ambient AWS credentials), further KMS slots are skipped without re-probing — + // so an SSH-only / local deploy in a repo that happens to contain KMS scopes it + // is not a recipient of does not pay a credential-probe stall per value. + kmsNoIdentity bool +} + +// NewOpener parses each candidate PEM private key (skipping empty/unparseable +// ones, so a caller may pass everything it has) and, when kmsAllowed is set, +// permits KMS Decrypt attempts for values that carry a KMS wrap slot. KMS is only +// ever attempted for a value that actually has an awskms:// wrap AND that no held +// SSH key already opened — so an SSH-only store never triggers an AWS call. +func NewOpener(privateKeys []string, kmsAllowed bool) *Opener { + o := &Opener{sshByFP: map[string]any{}, kmsAllowed: kmsAllowed, kmsClients: map[string]kmsAPI{}} + for _, pk := range privateKeys { + if strings.TrimSpace(pk) == "" { + continue + } + fp, signer, err := privateKeyFingerprint(pk) + if err != nil { + continue + } + o.sshByFP[fp] = signer + } + return o +} + +// hasMaterial reports whether the opener has any way to decrypt at all (used to +// short-circuit resolution when a caller supplied nothing usable). +func (o *Opener) hasMaterial() bool { + return len(o.sshByFP) > 0 || o.kmsAllowed +} + +// IsDeclaredSSHRecipient reports whether any held SSH key is listed in recipients. +// This is an OFFLINE proof of "I am a recipient of this file" for SSH keys — so if a +// value fails to open for a declared SSH recipient, that is tampering (a stripped +// wrap), never "not my scope". (KMS recipiency cannot be proven offline — it requires +// a Decrypt attempt — so KMS ownership is established by the first value that opens.) +func (o *Opener) IsDeclaredSSHRecipient(recipients []string) bool { + for _, r := range recipients { + if isKMSRecipient(r) { + continue + } + fp, err := recipientID(r) + if err != nil { + continue + } + if _, ok := o.sshByFP[fp]; ok { + return true + } + } + return false +} + +// kmsClientFor returns a cached KMS client for region, or (nil,false) if no ambient +// KMS identity is available. The "no identity" verdict is cached so credentials are +// probed at most once per Opener. +func (o *Opener) kmsClientFor(ctx context.Context, region string) (kmsAPI, bool) { + if o.kmsNoIdentity { + return nil, false + } + if o.kmsClients == nil { + o.kmsClients = map[string]kmsAPI{} + } + if c, ok := o.kmsClients[region]; ok { + return c, true + } + c, err := newKMSClient(ctx, region) + if err != nil { + o.kmsNoIdentity = true + return nil, false + } + o.kmsClients[region] = c + return c, true +} + +// OpenValue attempts to decrypt one envelope value bound to (stack, scope, key). +// It returns: +// +// - (value, true, nil) — decrypted with material the caller holds. +// - ("", false, nil) — the caller is not a usable recipient of this value +// (no held SSH key owns a wrap slot, and no KMS slot was decryptable). This is +// a least-privilege skip, NOT an error. +// - ("", true, err) — the caller IS a recipient (owns a slot) but decryption +// failed: tampered ciphertext, a broken (stack,scope,key) binding, or a KMS +// InvalidCiphertext. Callers treat this as a hard integrity error. +func (o *Opener) OpenValue(stack, scope, key string, ev EncryptedValue) (string, bool, error) { + aad := valueAAD(stack, scope, key) + // 1. SSH: a held private key whose fingerprint owns a wrap slot. Deterministic + // and offline — an SSH-openable value never triggers a KMS call. + for fp, signer := range o.sshByFP { + wrapped, ok := ev.Wraps[fp] + if !ok { + continue + } + dek, err := unwrapDEKWithSSH(signer, aad, wrapped) + if err != nil { + return "", true, errors.Wrapf(err, "failed to unwrap data key for %q in scope %q", key, scope) + } + return openWithDEK(ev, dek, aad, scope, key) + } + // 2. KMS: a wrap slot decryptable with the ambient credentials. Only attempted + // for values that actually carry a KMS slot. A malformed slot (bad URL / not + // base64 / wrong shape) is skipped, not hard-failed — a least-privilege reader + // must not abort on a foreign/corrupt file it is not a recipient of (lint catches + // such corruption offline). A transient backend error is remembered and only + // surfaced if NO slot opens, so another recipient slot still gets a chance. + if o.kmsAllowed { + var transient error + for slotID, wrapped := range ev.Wraps { + if !isKMSRecipientID(slotID) { + continue + } + r, err := parseKMSRecipient(slotID) + if err != nil || len(wrapped) != 1 { + continue // malformed slot → skip + } + blob, err := base64.StdEncoding.DecodeString(wrapped[0]) + if err != nil { + continue // not base64 → skip + } + ctx, cancel := context.WithTimeout(context.Background(), kmsCallTimeout) + cli, ok := o.kmsClientFor(ctx, r.region) + if !ok { + cancel() + continue // no ambient KMS identity → skip (breaker set, no re-probe) + } + dek, owned, err := decryptKMSWrap(ctx, cli, r, blob, stack, scope, key) + cancel() + if err != nil { + if owned { + return "", true, err // integrity (tamper) — definitive, stop + } + transient = err // remember; another slot may still open this value + continue + } + if !owned { + continue + } + return openWithDEK(ev, dek, aad, scope, key) + } + if transient != nil { + return "", false, transient // no slot opened; surface the transient fault + } + } + return "", false, nil +} + +// openWithDEK decodes the shared value ciphertext and opens it under the recovered +// data key, verifying the (stack,scope,key) AAD. A failure here (owned==true) is an +// integrity error, never a "not a recipient". +func openWithDEK(ev EncryptedValue, dek, aad []byte, scope, key string) (string, bool, error) { + blob, err := base64.StdEncoding.DecodeString(ev.Ciphertext) + if err != nil { + return "", true, errors.Wrapf(err, "failed to decode value ciphertext for %q in scope %q", key, scope) + } + plain, err := ciphers.OpenAEAD(dek, blob, aad) + if err != nil { + return "", true, errors.Wrapf(err, "failed to decrypt %q in scope %q", key, scope) + } + return string(plain), true, nil +} + +// unwrapDEKWithSSH recovers the data key from a per-recipient SSH wrap, verifying +// the (stack,scope,key) binding carried in aad. +func unwrapDEKWithSSH(signer any, aad []byte, wrapped []string) ([]byte, error) { + switch k := signer.(type) { + case *rsa.PrivateKey: + return ciphers.DecryptLargeStringWithAAD(k, wrapped, aad) + case ed25519.PrivateKey: + return ciphers.DecryptLargeStringWithEd25519AAD(k, wrapped, aad) + case *ed25519.PrivateKey: + return ciphers.DecryptLargeStringWithEd25519AAD(*k, wrapped, aad) + default: + return nil, errors.Errorf("unsupported private key type %T", signer) + } +} diff --git a/pkg/api/secrets/scoped/paths.go b/pkg/api/secrets/scoped/paths.go new file mode 100644 index 00000000..99d2aa97 --- /dev/null +++ b/pkg/api/secrets/scoped/paths.go @@ -0,0 +1,105 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) Simple Container + +package scoped + +import ( + "os" + "path/filepath" + "sort" + "strings" + + "github.com/pkg/errors" +) + +// stacksDirName is the conventional per-stack subdirectory under the .sc config +// root. Matches the whole-file store's layout (.sc/stacks//secrets.yaml). +const stacksDirName = "stacks" + +// ScopesPath returns the governance file path: /scopes.yaml. +func ScopesPath(scDir string) string { + return filepath.Join(scDir, ScopesFileName) +} + +// writeFileAtomic writes data to a UNIQUELY-named sibling temp file then renames it +// over path, so a crash mid-write can never leave a truncated/corrupt scope or scopes +// file (which would then hard-fail deploys). Rename is atomic on the same filesystem. +// A random temp suffix (os.CreateTemp) means two concurrent writers to the same file +// never collide on the temp path. +func writeFileAtomic(path string, data []byte, perm os.FileMode) error { + dir := filepath.Dir(path) + if err := os.MkdirAll(dir, 0o755); err != nil { + return errors.Wrapf(err, "failed to create directory for %s", path) + } + tmp, err := os.CreateTemp(dir, "."+filepath.Base(path)+".tmp-*") + if err != nil { + return errors.Wrapf(err, "failed to create temp file for %s", path) + } + tmpName := tmp.Name() + // Clean up the temp file on any early return; the successful path renames it away + // first, so the Remove then harmlessly no-ops. + defer func() { _ = os.Remove(tmpName) }() + if _, err := tmp.Write(data); err != nil { + _ = tmp.Close() + return errors.Wrapf(err, "failed to write %s", tmpName) + } + if err := tmp.Chmod(perm); err != nil { + _ = tmp.Close() + return errors.Wrapf(err, "failed to chmod %s", tmpName) + } + if err := tmp.Close(); err != nil { + return errors.Wrapf(err, "failed to close %s", tmpName) + } + if err := os.Rename(tmpName, path); err != nil { + return errors.Wrapf(err, "failed to rename %s -> %s", tmpName, path) + } + return nil +} + +// StackDir returns /stacks/. +func StackDir(scDir, stack string) string { + return filepath.Join(scDir, stacksDirName, stack) +} + +// ScopeFilePath returns /stacks//secrets..yaml. +func ScopeFilePath(scDir, stack, scope string) string { + return filepath.Join(StackDir(scDir, stack), ScopeFileName(scope)) +} + +// ListScopeFiles returns every committed scope file under /stacks/*/, +// sorted. It deliberately does NOT match the legacy plaintext secrets.yaml +// (that has no scope segment). Used by `lint` and by allow/disallow resealing. +func ListScopeFiles(scDir string) ([]string, error) { + stacksRoot := filepath.Join(scDir, stacksDirName) + entries, err := os.ReadDir(stacksRoot) + if os.IsNotExist(err) { + return nil, nil + } + if err != nil { + return nil, errors.Wrapf(err, "failed to list %s", stacksRoot) + } + var out []string + for _, e := range entries { + if !e.IsDir() { + continue + } + stackDir := filepath.Join(stacksRoot, e.Name()) + files, err := os.ReadDir(stackDir) + if err != nil { + return nil, errors.Wrapf(err, "failed to list %s", stackDir) + } + for _, f := range files { + name := f.Name() + // secrets..yaml but NOT the legacy secrets.yaml + if !strings.HasPrefix(name, scopeFilePrefix) || !strings.HasSuffix(name, scopeFileSuffix) { + continue + } + if ScopeNameFromFile(name) == "" { + continue + } + out = append(out, filepath.Join(stackDir, name)) + } + } + sort.Strings(out) + return out, nil +} diff --git a/pkg/api/secrets/scoped/resolve.go b/pkg/api/secrets/scoped/resolve.go new file mode 100644 index 00000000..c12e6746 --- /dev/null +++ b/pkg/api/secrets/scoped/resolve.go @@ -0,0 +1,142 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) Simple Container + +package scoped + +import ( + "os" + "path/filepath" + + "github.com/pkg/errors" +) + +// ErrScopedIntegrity tags a scoped-resolution failure that must NEVER be treated +// as "secrets simply absent": a value the key is a recipient of but cannot decrypt +// (tamper), a corrupt or renamed scope file, or the same key present in two +// openable scopes (ambiguous). Callers that otherwise tolerate a missing legacy +// secrets.yaml (IgnoreSecretsMissing) MUST still fail on errors.Is(err, this). +var ErrScopedIntegrity = errors.New("scoped secret integrity error") + +// ErrScopedUnavailable tags a scoped-resolution failure caused by a transient +// backend fault the caller IS entitled to but could not complete right now — e.g. a +// KMS throttle/outage after SDK retries. It is fatal (a deploy must not proceed with +// a secret it could not resolve) but distinct from ErrScopedIntegrity: it means +// "retry", not "tamper". Like ErrScopedIntegrity it must survive IgnoreSecretsMissing +// (see ReadStacks) — a transient outage is never "secrets simply absent". +var ErrScopedUnavailable = errors.New("scoped secret temporarily unavailable") + +// ResolveScopedValues returns every scoped secret in stackDir (a +// .sc/stacks/ directory) that ANY of privateKeys is a recipient of. This is +// the deploy-time read hook: the merge is keyed off the AMBIENT/CI KEYS, not a +// config field, so the pull_request clamp is cryptographic — a job holding only +// the "pr" scope key literally cannot decrypt secrets.prod.yaml (it is not a +// recipient), with no config surface to subvert. +// +// privateKeys are candidate PEM keys (the ambient config key plus any CI scope +// keys such as SC_KEY_PR / SC_SCOPE_KEY); an empty or unparseable candidate is +// skipped so callers can pass everything they have. +// +// Semantics: +// - No usable key, or no secrets..yaml in stackDir → empty map, no error +// (repos that have not adopted scopes are entirely unaffected). +// - A scope file no candidate key is a recipient of is skipped (least privilege). +// - A value a key IS a recipient of but cannot decrypt is a HARD error tagged +// ErrScopedIntegrity (tampered ciphertext / broken binding) — never a silent +// skip (RFC hard-fail). +// - A corrupt or renamed scope file, or the same key in two openable scopes, is a +// HARD error tagged ErrScopedIntegrity. +// +// The caller merges the result into the whole-file store's values WITHOUT +// overwriting existing keys, so a scoped value can never change the meaning of a +// secret already resolved from the legacy store. +func ResolveScopedValues(stackDir string, privateKeys []string) (map[string]string, error) { + out := map[string]string{} + + entries, err := os.ReadDir(stackDir) + if os.IsNotExist(err) { + return out, nil + } + if err != nil { + return nil, errors.Wrapf(err, "failed to list %s for scoped secrets", stackDir) + } + var scopeFiles []string + for _, e := range entries { + if e.IsDir() { + continue + } + if ScopeNameFromFile(e.Name()) == "" { + continue // not a secrets..yaml (legacy secrets.yaml is excluded) + } + scopeFiles = append(scopeFiles, filepath.Join(stackDir, e.Name())) + } + if len(scopeFiles) == 0 { + return out, nil // no scopes → do not even parse keys + } + + // The opener holds every usable candidate SSH key and permits KMS Decrypt via + // the ambient AWS credentials. KMS is only ever attempted for a value that has a + // KMS wrap slot AND that no held SSH key opened, so an SSH-only store never calls + // AWS. Unparseable/empty candidate keys are skipped (a caller may pass several). + op := NewOpener(privateKeys, true) + if !op.hasMaterial() { + return out, nil // nothing usable to open with + } + + origin := map[string]string{} // key -> scope, to detect cross-scope duplicates + for _, path := range scopeFiles { + f, err := LoadScopeFile(path) + if err != nil { + return nil, errors.Wrapf(ErrScopedIntegrity, "%v", err) + } + keys := f.Keys() + // All values in a scope file share the same recipient set, so ownership is + // file-uniform. We are a recipient of this file if EITHER a held SSH key is a + // declared recipient (offline proof) OR some value opens (establishes KMS + // recipiency, which cannot be proven offline). Once we know we are a recipient, + // ANY value that fails to open is tampering (a stripped wrap) — including the + // FIRST value, so first-value tamper is never swallowed as "not my scope". + sshDeclared := op.IsDeclaredSSHRecipient(f.Recipients) + fileOwned := false + vals := make(map[string]string, len(keys)) + for _, key := range keys { + val, owned, oerr := f.Open(key, op) + if oerr != nil { + if owned { + // We ARE a recipient of this value but it failed to open: tampered + // ciphertext, a broken (stack,scope,key) binding, or a wrong-key wrap. + return nil, errors.Wrapf(ErrScopedIntegrity, "scope %q: failed to open %q that this key/role is a recipient of (tampered ciphertext or broken binding?): %v", f.Scope, key, oerr) + } + // A transient/unavailable backend fault (e.g. a KMS throttle/outage) — + // fatal, but a retry, not tamper. Position-independent: it never masquerades + // as a stripped wrap regardless of which value hit it. + return nil, errors.Wrapf(ErrScopedUnavailable, "scope %q: could not open %q: %v", f.Scope, key, oerr) + } + if !owned { + if fileOwned || sshDeclared { + // We are provably a recipient (an earlier value opened, or a held SSH key + // is in the declared recipient set) yet this value has no wrap for us — + // its slot was stripped. Tamper, not least-privilege. + return nil, errors.Wrapf(ErrScopedIntegrity, "scope %q: value %q is missing this recipient's wrap (tampered?)", f.Scope, key) + } + break // not our scope + } + fileOwned = true + vals[key] = val + } + if !fileOwned { + continue + } + for _, key := range keys { + val, ok := vals[key] + if !ok { + continue + } + if prev, dup := origin[key]; dup { + return nil, errors.Wrapf(ErrScopedIntegrity, "secret %q is present in two openable scopes (%q and %q); resolution is ambiguous — run `sc secrets scope lint`", key, prev, f.Scope) + } + out[key] = val + origin[key] = f.Scope + } + } + return out, nil +} diff --git a/pkg/api/secrets/scoped/scoped_test.go b/pkg/api/secrets/scoped/scoped_test.go new file mode 100644 index 00000000..0a3e10ad --- /dev/null +++ b/pkg/api/secrets/scoped/scoped_test.go @@ -0,0 +1,576 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) Simple Container + +package scoped + +import ( + "crypto/ecdsa" + "crypto/elliptic" + "crypto/rand" + "encoding/pem" + "os" + "path/filepath" + "testing" + + . "github.com/onsi/gomega" + "github.com/pkg/errors" + "golang.org/x/crypto/ssh" + + "github.com/simple-container-com/api/pkg/api/secrets/ciphers" +) + +// genEd25519Recipient returns an SSH authorized-key line and the matching +// unencrypted PEM private key. +func genEd25519Recipient(t *testing.T) (authorized, privatePEM string) { + t.Helper() + priv, pub, err := ciphers.GenerateEd25519KeyPair() + NewWithT(t).Expect(err).NotTo(HaveOccurred()) + sshPub, err := ssh.NewPublicKey(pub) + NewWithT(t).Expect(err).NotTo(HaveOccurred()) + pem, err := ciphers.MarshalEd25519PrivateKey(priv) + NewWithT(t).Expect(err).NotTo(HaveOccurred()) + return string(ssh.MarshalAuthorizedKey(sshPub)), pem +} + +// genRSARecipient returns an SSH authorized-key line and the matching PEM key. +func genRSARecipient(t *testing.T) (authorized, privatePEM string) { + t.Helper() + priv, pub, err := ciphers.GenerateKeyPair(2048) + NewWithT(t).Expect(err).NotTo(HaveOccurred()) + sshPub, err := ssh.NewPublicKey(pub) + NewWithT(t).Expect(err).NotTo(HaveOccurred()) + return string(ssh.MarshalAuthorizedKey(sshPub)), ciphers.MarshalRSAPrivateKey(priv) +} + +func TestScopeFile_RoundTrip_Ed25519(t *testing.T) { + RegisterTestingT(t) + authorized, priv := genEd25519Recipient(t) + f, err := NewScopeFile("teststack", "pr", []string{authorized}) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Set("defectdojo-api-key", "s3cr3t-value")).To(Succeed()) + + got, err := f.Get("defectdojo-api-key", priv) + Expect(err).NotTo(HaveOccurred()) + Expect(got).To(Equal("s3cr3t-value")) +} + +func TestScopeFile_RoundTrip_RSA(t *testing.T) { + RegisterTestingT(t) + authorized, priv := genRSARecipient(t) + f, err := NewScopeFile("teststack", "pr", []string{authorized}) + Expect(err).NotTo(HaveOccurred()) + // value longer than one RSA-OAEP chunk to exercise chunking + AAD + long := "" + for i := 0; i < 500; i++ { + long += "x" + } + Expect(f.Set("big", long)).To(Succeed()) + got, err := f.Get("big", priv) + Expect(err).NotTo(HaveOccurred()) + Expect(got).To(Equal(long)) +} + +func TestScopeFile_MultiRecipient_And_WrongKey(t *testing.T) { + RegisterTestingT(t) + authA, privA := genEd25519Recipient(t) + authB, privB := genRSARecipient(t) + _, privC := genEd25519Recipient(t) // not a recipient + + f, err := NewScopeFile("teststack", "pr", []string{authA, authB}) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Set("k", "v")).To(Succeed()) + + // both recipients decrypt + gotA, err := f.Get("k", privA) + Expect(err).NotTo(HaveOccurred()) + Expect(gotA).To(Equal("v")) + gotB, err := f.Get("k", privB) + Expect(err).NotTo(HaveOccurred()) + Expect(gotB).To(Equal("v")) + + // value sealed exactly twice (once per recipient) + Expect(f.Values["k"].Wraps).To(HaveLen(2)) + + // a non-recipient key is rejected distinctly + _, err = f.Get("k", privC) + Expect(err).To(HaveOccurred()) + Expect(errors.Is(err, ErrRecipientNotAllowed)).To(BeTrue()) +} + +func TestScopeFile_TransplantResistance_ScopeAndKey(t *testing.T) { + RegisterTestingT(t) + authorized, priv := genEd25519Recipient(t) + + // Seal a value in scope "prod". + prod, err := NewScopeFile("teststack", "prod", []string{authorized}) + Expect(err).NotTo(HaveOccurred()) + Expect(prod.Set("api-key", "prod-secret")).To(Succeed()) + blob := prod.Values["api-key"] + + // Attacker copies the ciphertext into a "pr"-scoped file (same recipient). + // AAD binds to scope, so decrypt under scope "pr" must fail. + pr, err := NewScopeFile("teststack", "pr", []string{authorized}) + Expect(err).NotTo(HaveOccurred()) + pr.Values["api-key"] = blob + _, err = pr.Get("api-key", priv) + Expect(err).To(HaveOccurred()) + Expect(errors.Is(err, ErrRecipientNotAllowed)).To(BeFalse()) // right recipient, wrong binding + + // Attacker moves the ciphertext onto a different key in the SAME scope. + // AAD binds to key, so this must fail too. + prod.Values["other-key"] = blob + _, err = prod.Get("other-key", priv) + Expect(err).To(HaveOccurred()) +} + +func TestLoadScopeFile_RejectsRenamedFile(t *testing.T) { + RegisterTestingT(t) + authorized, _ := genEd25519Recipient(t) + f, err := NewScopeFile("teststack", "prod", []string{authorized}) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Set("k", "v")).To(Succeed()) + + dir := t.TempDir() + // Write the prod-scoped content under a pr filename (the rename attack). + renamed := filepath.Join(dir, ScopeFileName("pr")) + Expect(f.Save(renamed)).To(Succeed()) + + _, err = LoadScopeFile(renamed) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("filename")) +} + +func TestLoadScopeFile_SaveLoadRoundTrip(t *testing.T) { + RegisterTestingT(t) + authorized, priv := genEd25519Recipient(t) + f, err := NewScopeFile("teststack", "pr", []string{authorized}) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Set("k", "v")).To(Succeed()) + + path := ScopeFilePath(t.TempDir(), "teststack", "pr") + Expect(f.Save(path)).To(Succeed()) + + loaded, err := LoadScopeFile(path) + Expect(err).NotTo(HaveOccurred()) + Expect(loaded.Scope).To(Equal("pr")) + Expect(loaded.Stack).To(Equal("teststack")) + got, err := loaded.Get("k", priv) + Expect(err).NotTo(HaveOccurred()) + Expect(got).To(Equal("v")) +} + +func TestLoadScopeFile_VersionGuardFailsClosed(t *testing.T) { + RegisterTestingT(t) + authorized, _ := genEd25519Recipient(t) + f, err := NewScopeFile("teststack", "pr", []string{authorized}) + Expect(err).NotTo(HaveOccurred()) + f.SchemaVersion = CurrentScopesSchemaVersion + 1 + + dir := t.TempDir() + path := filepath.Join(dir, ScopeFileName("pr")) + Expect(f.Save(path)).To(Succeed()) + + _, err = LoadScopeFile(path) + Expect(err).To(HaveOccurred()) + Expect(errors.Is(err, ErrScopesVersionUnsupported)).To(BeTrue()) +} + +func TestVerifyConsistency_DetectsUnknownRecipient(t *testing.T) { + RegisterTestingT(t) + authA, _ := genEd25519Recipient(t) + f, err := NewScopeFile("teststack", "pr", []string{authA}) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Set("k", "v")).To(Succeed()) + Expect(f.VerifyConsistency()).To(Succeed()) + + // Inject a value sealed to a recipient not in the declared list. + f.Values["k"].Wraps["SHA256:bogusfingerprint"] = []string{"deadbeef"} + Expect(f.VerifyConsistency()).NotTo(Succeed()) +} + +func TestVerifyConsistency_RejectsPlaintextChunk(t *testing.T) { + RegisterTestingT(t) + authA, _ := genEd25519Recipient(t) + f, err := NewScopeFile("teststack", "pr", []string{authA}) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Set("k", "v")).To(Succeed()) + fp, err := Fingerprint(authA) + Expect(err).NotTo(HaveOccurred()) + + // Hand-edit: replace the sealed chunk with a non-base64 plaintext value. + f.Values["k"].Wraps[fp] = []string{"plaintext-secret"} + Expect(f.VerifyConsistency()).NotTo(Succeed()) + + // A valid-base64-but-implausibly-short chunk is also rejected. + f.Values["k"].Wraps[fp] = []string{"YWJj"} // "abc" + Expect(f.VerifyConsistency()).NotTo(Succeed()) +} + +func TestAllow_RejectsUnsupportedKeyType(t *testing.T) { + RegisterTestingT(t) + // ECDSA key: parses + fingerprints as SSH, but the cipher layer can't seal to it. + priv, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader) + Expect(err).NotTo(HaveOccurred()) + sshPub, err := ssh.NewPublicKey(&priv.PublicKey) + Expect(err).NotTo(HaveOccurred()) + ecdsaAuthorized := string(ssh.MarshalAuthorizedKey(sshPub)) + + s := &Scopes{SchemaVersion: CurrentScopesSchemaVersion, Scopes: map[string]Scope{}} + _, err = s.Allow("pr", ecdsaAuthorized) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("unsupported recipient key type")) +} + +func TestScopes_AllowDisallowRecipients(t *testing.T) { + RegisterTestingT(t) + authA, _ := genEd25519Recipient(t) + authB, _ := genRSARecipient(t) + s := &Scopes{SchemaVersion: CurrentScopesSchemaVersion, Scopes: map[string]Scope{}} + + added, err := s.Allow("pr", authA) + Expect(err).NotTo(HaveOccurred()) + Expect(added).To(BeTrue()) + // idempotent (same key with a different comment) + added, err = s.Allow("pr", authA+" my-comment") + Expect(err).NotTo(HaveOccurred()) + Expect(added).To(BeFalse()) + + _, err = s.Allow("pr", authB) + Expect(err).NotTo(HaveOccurred()) + recips, err := s.Recipients("pr") + Expect(err).NotTo(HaveOccurred()) + Expect(recips).To(HaveLen(2)) + + removed, err := s.Disallow("pr", authA) + Expect(err).NotTo(HaveOccurred()) + Expect(removed).To(BeTrue()) + recips, _ = s.Recipients("pr") + Expect(recips).To(HaveLen(1)) + + _, err = s.Recipients("nonexistent") + Expect(err).To(HaveOccurred()) +} + +func TestScopes_SaveLoadRoundTrip_And_VersionGuard(t *testing.T) { + RegisterTestingT(t) + authA, _ := genEd25519Recipient(t) + s := &Scopes{Scopes: map[string]Scope{"pr": {Description: "pr scope", Recipients: []string{authA}}}} + dir := t.TempDir() + path := filepath.Join(dir, ScopesFileName) + Expect(s.Save(path)).To(Succeed()) + + loaded, err := LoadScopes(path) + Expect(err).NotTo(HaveOccurred()) + Expect(loaded.Scopes).To(HaveKey("pr")) + Expect(loaded.SchemaVersion).To(Equal(CurrentScopesSchemaVersion)) + + // too-new version fails closed + loaded.SchemaVersion = CurrentScopesSchemaVersion + 1 + Expect(loaded.Save(path)).To(Succeed()) + _, err = LoadScopes(path) + Expect(errors.Is(err, ErrScopesVersionUnsupported)).To(BeTrue()) +} + +func TestLoadScopes_MissingFileIsEmpty(t *testing.T) { + RegisterTestingT(t) + s, err := LoadScopes(filepath.Join(t.TempDir(), "nope.yaml")) + Expect(err).NotTo(HaveOccurred()) + Expect(s.Scopes).To(BeEmpty()) + Expect(s.SchemaVersion).To(Equal(CurrentScopesSchemaVersion)) +} + +func TestValidation_RejectsUnsafeNames(t *testing.T) { + RegisterTestingT(t) + Expect(ValidateScopeName("pr")).To(Succeed()) + Expect(ValidateScopeName("PR")).NotTo(Succeed()) // uppercase + Expect(ValidateScopeName("pr/prod")).NotTo(Succeed()) // slash + Expect(ValidateScopeName("pr\x00x")).NotTo(Succeed()) // NUL + Expect(ValidateSecretKey("defectdojo-api-key")).To(Succeed()) + Expect(ValidateSecretKey("CLOUDFLARE_API_TOKEN")).To(Succeed()) + Expect(ValidateSecretKey("bad key")).NotTo(Succeed()) // space + Expect(ValidateSecretKey("bad\x00key")).NotTo(Succeed()) // NUL +} + +func TestReencrypt_AddAndRemoveRecipient(t *testing.T) { + RegisterTestingT(t) + authA, privA := genEd25519Recipient(t) + authB, privB := genRSARecipient(t) + + f, err := NewScopeFile("teststack", "pr", []string{authA}) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Set("k", "v")).To(Succeed()) + + // Add B by resealing with A's key. + Expect(f.Reencrypt([]string{authA, authB}, NewOpener([]string{privA}, false))).To(Succeed()) + gotA, err := f.Get("k", privA) + Expect(err).NotTo(HaveOccurred()) + Expect(gotA).To(Equal("v")) + gotB, err := f.Get("k", privB) + Expect(err).NotTo(HaveOccurred()) + Expect(gotB).To(Equal("v")) + Expect(f.Values["k"].Wraps).To(HaveLen(2)) + + // Remove B by resealing to A only. + Expect(f.Reencrypt([]string{authA}, NewOpener([]string{privA}, false))).To(Succeed()) + Expect(f.Values["k"].Wraps).To(HaveLen(1)) + _, err = f.Get("k", privB) + Expect(errors.Is(err, ErrRecipientNotAllowed)).To(BeTrue()) +} + +func TestReencrypt_NonRecipientKeyFailsAndLeavesFileIntact(t *testing.T) { + RegisterTestingT(t) + authA, _ := genEd25519Recipient(t) + authB, _ := genRSARecipient(t) + _, privC := genEd25519Recipient(t) // not a recipient + + f, err := NewScopeFile("teststack", "pr", []string{authA}) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Set("k", "v")).To(Succeed()) + before := f.Values["k"] + + err = f.Reencrypt([]string{authA, authB}, NewOpener([]string{privC}, false)) + Expect(err).To(HaveOccurred()) + // file untouched + Expect(f.Recipients).To(Equal([]string{authA})) + Expect(f.Values["k"]).To(Equal(before)) +} + +func TestListScopeFiles_ExcludesLegacyPlaintext(t *testing.T) { + RegisterTestingT(t) + authA, _ := genEd25519Recipient(t) + scDir := t.TempDir() + + mk := func(stack, base, scope string) { + dir := StackDir(scDir, stack) + Expect(os.MkdirAll(dir, 0o755)).To(Succeed()) + if scope == "" { + Expect(os.WriteFile(filepath.Join(dir, base), []byte("values:\n k: v\n"), 0o644)).To(Succeed()) + return + } + f, err := NewScopeFile(stack, scope, []string{authA}) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Save(filepath.Join(dir, base))).To(Succeed()) + } + mk("s1", ScopeFileName("pr"), "pr") + mk("s1", "secrets.yaml", "") // legacy plaintext — must be excluded + mk("s2", ScopeFileName("prod"), "prod") + + files, err := ListScopeFiles(scDir) + Expect(err).NotTo(HaveOccurred()) + Expect(files).To(HaveLen(2)) + for _, p := range files { + Expect(ScopeNameFromFile(p)).NotTo(Equal("")) + } +} + +func TestResolveScopedValues_KeyDeterminesScope(t *testing.T) { + RegisterTestingT(t) + authA, privA := genEd25519Recipient(t) + authB, privB := genRSARecipient(t) + _, privC := genEd25519Recipient(t) // recipient of nothing + scDir := t.TempDir() + stackDir := StackDir(scDir, "teststack") + + pr, err := NewScopeFile("teststack", "pr", []string{authA}) + Expect(err).NotTo(HaveOccurred()) + Expect(pr.Set("defectdojo-api-key", "dd")).To(Succeed()) + Expect(pr.Save(ScopeFilePath(scDir, "teststack", "pr"))).To(Succeed()) + + prod, err := NewScopeFile("teststack", "prod", []string{authB}) + Expect(err).NotTo(HaveOccurred()) + Expect(prod.Set("prod-mongo-uri", "mongo")).To(Succeed()) + Expect(prod.Save(ScopeFilePath(scDir, "teststack", "prod"))).To(Succeed()) + + // A's key opens pr only — the prod value is invisible (cryptographic clamp). + got, err := ResolveScopedValues(stackDir, []string{privA}) + Expect(err).NotTo(HaveOccurred()) + Expect(got).To(Equal(map[string]string{"defectdojo-api-key": "dd"})) + + // B's key opens prod only. + got, err = ResolveScopedValues(stackDir, []string{privB}) + Expect(err).NotTo(HaveOccurred()) + Expect(got).To(Equal(map[string]string{"prod-mongo-uri": "mongo"})) + + // A non-recipient key sees nothing. + got, err = ResolveScopedValues(stackDir, []string{privC}) + Expect(err).NotTo(HaveOccurred()) + Expect(got).To(BeEmpty()) + + // Empty key → nothing (no error). + got, err = ResolveScopedValues(stackDir, nil) + Expect(err).NotTo(HaveOccurred()) + Expect(got).To(BeEmpty()) + + // A stack dir with no scope files → nothing (no error) — backward compat. + got, err = ResolveScopedValues(StackDir(scDir, "nostack"), []string{privA}) + Expect(err).NotTo(HaveOccurred()) + Expect(got).To(BeEmpty()) +} + +func TestResolveScopedValues_CrossScopeDuplicateFails(t *testing.T) { + RegisterTestingT(t) + authA, privA := genEd25519Recipient(t) + scDir := t.TempDir() + stackDir := StackDir(scDir, "teststack") + + pr, err := NewScopeFile("teststack", "pr", []string{authA}) + Expect(err).NotTo(HaveOccurred()) + Expect(pr.Set("shared", "1")).To(Succeed()) + Expect(pr.Save(ScopeFilePath(scDir, "teststack", "pr"))).To(Succeed()) + + staging, err := NewScopeFile("teststack", "staging", []string{authA}) + Expect(err).NotTo(HaveOccurred()) + Expect(staging.Set("shared", "2")).To(Succeed()) + Expect(staging.Save(ScopeFilePath(scDir, "teststack", "staging"))).To(Succeed()) + + _, err = ResolveScopedValues(stackDir, []string{privA}) + Expect(err).To(HaveOccurred()) + Expect(errors.Is(err, ErrScopedIntegrity)).To(BeTrue()) + Expect(err.Error()).To(ContainSubstring("two openable scopes")) +} + +func TestResolveScopedValues_MultipleCandidateKeys(t *testing.T) { + RegisterTestingT(t) + // Simulates a caller passing several candidate keys (ambient + CI scope keys): + // each opens a different scope, and the union is resolved. + authA, privA := genEd25519Recipient(t) + authB, privB := genRSARecipient(t) + scDir := t.TempDir() + stackDir := StackDir(scDir, "teststack") + + pr, err := NewScopeFile("teststack", "pr", []string{authA}) + Expect(err).NotTo(HaveOccurred()) + Expect(pr.Set("defectdojo-api-key", "dd")).To(Succeed()) + Expect(pr.Save(ScopeFilePath(scDir, "teststack", "pr"))).To(Succeed()) + + stg, err := NewScopeFile("teststack", "staging", []string{authB}) + Expect(err).NotTo(HaveOccurred()) + Expect(stg.Set("staging-token", "stg")).To(Succeed()) + Expect(stg.Save(ScopeFilePath(scDir, "teststack", "staging"))).To(Succeed()) + + got, err := ResolveScopedValues(stackDir, []string{privA, privB, "", "not-a-key"}) + Expect(err).NotTo(HaveOccurred()) + Expect(got).To(Equal(map[string]string{"defectdojo-api-key": "dd", "staging-token": "stg"})) +} + +func TestEnvelope_NoPerRecipientDivergence(t *testing.T) { + RegisterTestingT(t) + // The envelope's shared value ciphertext makes it impossible to show two + // recipients DIFFERENT valid plaintexts for the same key: tampering one + // recipient's wrap yields a decrypt failure, never an attacker-chosen value. + authA, privA := genEd25519Recipient(t) + authB, privB := genEd25519Recipient(t) + f, err := NewScopeFile("teststack", "pr", []string{authA, authB}) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Set("k", "real")).To(Succeed()) + fpB, err := Fingerprint(authB) + Expect(err).NotTo(HaveOccurred()) + + // Attacker (holds B's public key, can edit the file) crafts a separate envelope + // of "evil" and splices B's wrap in, leaving the shared value ciphertext intact. + evil, err := encryptForRecipients([]string{authB}, "teststack", "pr", "k", "evil") + Expect(err).NotTo(HaveOccurred()) + f.Values["k"].Wraps[fpB] = evil.Wraps[fpB] + + // B now unwraps a data key that does not open the shared ciphertext → error, + // NOT the value "evil". + _, err = f.Get("k", privB) + Expect(err).To(HaveOccurred()) + // A is unaffected and still reads the real value. + got, err := f.Get("k", privA) + Expect(err).NotTo(HaveOccurred()) + Expect(got).To(Equal("real")) +} + +func TestScopeFile_TransplantResistance_RSA(t *testing.T) { + RegisterTestingT(t) + authorized, priv := genRSARecipient(t) + + prod, err := NewScopeFile("teststack", "prod", []string{authorized}) + Expect(err).NotTo(HaveOccurred()) + Expect(prod.Set("api-key", "prod-secret")).To(Succeed()) + blob := prod.Values["api-key"] + + // scope transplant (OAEP label binds scope) + pr, err := NewScopeFile("teststack", "pr", []string{authorized}) + Expect(err).NotTo(HaveOccurred()) + pr.Values["api-key"] = blob + _, err = pr.Get("api-key", priv) + Expect(err).To(HaveOccurred()) + + // key transplant (OAEP label binds key) + prod.Values["other-key"] = blob + _, err = prod.Get("other-key", priv) + Expect(err).To(HaveOccurred()) +} + +func TestScopeFile_TransplantResistance_CrossStack(t *testing.T) { + RegisterTestingT(t) + // ed25519 + RSA both: a value sealed for (stackA, prod, k) must not decrypt as + // (stackB, prod, k) — the AAD binds the stack. + for _, gen := range []func(*testing.T) (string, string){genEd25519Recipient, genRSARecipient} { + authorized, priv := gen(t) + a, err := NewScopeFile("stackA", "prod", []string{authorized}) + Expect(err).NotTo(HaveOccurred()) + Expect(a.Set("k", "v")).To(Succeed()) + + b, err := NewScopeFile("stackB", "prod", []string{authorized}) + Expect(err).NotTo(HaveOccurred()) + b.Values["k"] = a.Values["k"] // attacker copies ciphertext into another stack + _, err = b.Get("k", priv) + Expect(err).To(HaveOccurred(), "cross-stack transplant must fail") + } +} + +func TestSameRecipients(t *testing.T) { + RegisterTestingT(t) + a, _ := genEd25519Recipient(t) + b, _ := genRSARecipient(t) + Expect(SameRecipients([]string{a, b}, []string{b, a + " comment"})).To(Succeed()) // order/comment-independent + err := SameRecipients([]string{a, b}, []string{a}) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("count differs")) + err = SameRecipients([]string{a}, []string{b}) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("not present in both")) + Expect(SameRecipients([]string{"not-a-key"}, []string{a})).To(HaveOccurred()) +} + +func TestScopeFile_Delete(t *testing.T) { + RegisterTestingT(t) + authA, priv := genEd25519Recipient(t) + f, err := NewScopeFile("teststack", "pr", []string{authA}) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Set("k", "v")).To(Succeed()) + Expect(f.Delete("k")).To(BeTrue()) + _, err = f.Get("k", priv) + Expect(err).To(HaveOccurred()) + Expect(f.Delete("absent")).To(BeFalse()) +} + +func TestGet_RejectsPassphraseProtectedKey(t *testing.T) { + RegisterTestingT(t) + priv, pub, err := ciphers.GenerateEd25519KeyPair() + Expect(err).NotTo(HaveOccurred()) + sshPub, err := ssh.NewPublicKey(pub) + Expect(err).NotTo(HaveOccurred()) + authorized := string(ssh.MarshalAuthorizedKey(sshPub)) + + block, err := ssh.MarshalPrivateKeyWithPassphrase(priv, "", []byte("s3cret")) + Expect(err).NotTo(HaveOccurred()) + encryptedPEM := string(pem.EncodeToMemory(block)) + + f, err := NewScopeFile("teststack", "pr", []string{authorized}) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Set("k", "v")).To(Succeed()) + _, err = f.Get("k", encryptedPEM) + Expect(err).To(HaveOccurred()) + Expect(err.Error()).To(ContainSubstring("passphrase-protected")) +} + +func TestScopeNameFromFile(t *testing.T) { + RegisterTestingT(t) + Expect(ScopeNameFromFile("/x/.sc/stacks/s/secrets.pr.yaml")).To(Equal("pr")) + Expect(ScopeNameFromFile("/x/secrets.yaml")).To(Equal("")) + Expect(ScopeNameFromFile("/x/other.pr.yaml")).To(Equal("")) +} diff --git a/pkg/api/secrets/scoped/scopes.go b/pkg/api/secrets/scoped/scopes.go new file mode 100644 index 00000000..0e857595 --- /dev/null +++ b/pkg/api/secrets/scoped/scopes.go @@ -0,0 +1,229 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) Simple Container + +// Package scoped implements the additive per-scope secret store +// (secrets..yaml): committed-encrypted files whose values are sealed to a +// per-scope recipient set, so a pull_request-triggered CI job can be given a key +// that opens only the "pr" scope rather than the whole-file store. It reuses the +// sibling ciphers package (RSA-OAEP + X25519 sealed box); recipients are SSH +// public keys, values are bound to their (scope,key) via associated data. Old +// binaries never read these files (see the fail-closed schemaVersion guard on the +// legacy store); the whole-file store in the parent package is untouched. +package scoped + +import ( + "os" + "regexp" + "sort" + + "github.com/pkg/errors" + "gopkg.in/yaml.v3" +) + +// CurrentScopesSchemaVersion is the highest scopes.yaml / secrets..yaml +// schema version this build understands. A store declaring a higher version is +// refused (fail-closed) — mirrors the whole-file store guard so a newer file is +// never read as empty and then clobbered. +const CurrentScopesSchemaVersion = 1 + +// ScopesFileName is the governance file at the .sc config root. +const ScopesFileName = "scopes.yaml" + +var ( + // ErrRecipientNotAllowed indicates the decrypting key is not a recipient of a + // value (as opposed to the value being corrupt or the wrong scope). + ErrRecipientNotAllowed = errors.New("key is not a recipient of this scoped secret") + // ErrScopesVersionUnsupported is fatal on every read path (fail-closed). + ErrScopesVersionUnsupported = errors.New("unsupported scoped secrets schema version") + + // scopeNameRe / secretKeyRe restrict names to a NUL-free charset: the AAD + // binding (valueAAD) uses NUL separators, and these are also used as filename + // components, so control characters and separators are rejected. + scopeNameRe = regexp.MustCompile(`^[a-z0-9][a-z0-9-]{0,62}$`) + secretKeyRe = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$`) +) + +// Fingerprint returns the stable SHA256 SSH fingerprint of an authorized public +// key ("SHA256:…"), the identifier used to key recipients inside a scope file. +func Fingerprint(authorizedKey string) (string, error) { + return recipientFingerprint(authorizedKey) +} + +// SameRecipients reports, as an error, whether two recipient lists denote the same +// set of recipients — compared by recipient ID (SSH fingerprint or normalized +// awskms:// URL) so ordering and comments do not matter. Used to detect drift +// between a scope file and scopes.yaml. +func SameRecipients(a, b []string) error { + fps := func(list []string) (map[string]struct{}, error) { + m := make(map[string]struct{}, len(list)) + for _, r := range list { + fp, err := recipientID(r) + if err != nil { + return nil, err + } + m[fp] = struct{}{} + } + return m, nil + } + am, err := fps(a) + if err != nil { + return err + } + bm, err := fps(b) + if err != nil { + return err + } + if len(am) != len(bm) { + return errors.Errorf("recipient count differs (%d vs %d)", len(am), len(bm)) + } + for fp := range am { + if _, ok := bm[fp]; !ok { + return errors.Errorf("recipient %s not present in both sets", fp) + } + } + return nil +} + +// ValidateScopeName rejects scope names that are unsafe as a filename component or +// AAD field. +func ValidateScopeName(scope string) error { + if !scopeNameRe.MatchString(scope) { + return errors.Errorf("invalid scope name %q (allowed: %s)", scope, scopeNameRe.String()) + } + return nil +} + +// ValidateSecretKey rejects secret keys that are unsafe as an AAD field. +func ValidateSecretKey(key string) error { + if !secretKeyRe.MatchString(key) { + return errors.Errorf("invalid secret key %q (allowed: %s)", key, secretKeyRe.String()) + } + return nil +} + +// Scopes is the .sc/scopes.yaml governance file: the authoritative per-scope +// recipient set. It is CODEOWNERS-gated at the repo level; `sc secrets lint` +// independently verifies each scope file's recipients against it so a scope file +// edited out-of-band (bypassing review) is still caught. +type Scopes struct { + SchemaVersion int `yaml:"schemaVersion"` + Scopes map[string]Scope `yaml:"scopes"` +} + +// Scope is one named recipient set. +type Scope struct { + Description string `yaml:"description,omitempty"` + Recipients []string `yaml:"recipients"` +} + +// LoadScopes reads and validates scopes.yaml. A missing file is not an error — +// it returns an empty, usable *Scopes (no scopes defined yet). +func LoadScopes(path string) (*Scopes, error) { + data, err := os.ReadFile(path) + if os.IsNotExist(err) { + return &Scopes{SchemaVersion: CurrentScopesSchemaVersion, Scopes: map[string]Scope{}}, nil + } + if err != nil { + return nil, errors.Wrapf(err, "failed to read %s", path) + } + var s Scopes + if err := yaml.Unmarshal(data, &s); err != nil { + return nil, errors.Wrapf(err, "failed to parse %s", path) + } + if s.SchemaVersion > CurrentScopesSchemaVersion { + return nil, errors.Wrapf(ErrScopesVersionUnsupported, "%s declares version %d, this build supports up to %d", path, s.SchemaVersion, CurrentScopesSchemaVersion) + } + if s.Scopes == nil { + s.Scopes = map[string]Scope{} + } + for name := range s.Scopes { + if err := ValidateScopeName(name); err != nil { + return nil, errors.Wrapf(err, "in %s", path) + } + } + return &s, nil +} + +// Save writes scopes.yaml with a stable field order. +func (s *Scopes) Save(path string) error { + if s.SchemaVersion == 0 { + s.SchemaVersion = CurrentScopesSchemaVersion + } + data, err := yaml.Marshal(s) + if err != nil { + return errors.Wrap(err, "failed to marshal scopes") + } + if err := writeFileAtomic(path, data, 0o644); err != nil { + return errors.Wrapf(err, "failed to write %s", path) + } + return nil +} + +// Recipients returns the authoritative recipient set for a scope, or an error if +// the scope is not declared (a value can only be encrypted to a governed set). +func (s *Scopes) Recipients(scope string) ([]string, error) { + sc, ok := s.Scopes[scope] + if !ok { + return nil, errors.Errorf("scope %q is not declared in %s", scope, ScopesFileName) + } + if len(sc.Recipients) == 0 { + return nil, errors.Errorf("scope %q has no recipients in %s", scope, ScopesFileName) + } + return sc.Recipients, nil +} + +// Allow adds a recipient to a scope (creating the scope if needed) and returns +// whether it was newly added. +func (s *Scopes) Allow(scope, recipient string) (bool, error) { + if err := ValidateScopeName(scope); err != nil { + return false, err + } + if _, err := recipientID(recipient); err != nil { + return false, err + } + if err := validateEncryptableRecipient(recipient); err != nil { + return false, err + } + if s.Scopes == nil { + s.Scopes = map[string]Scope{} + } + sc := s.Scopes[scope] + fp, _ := recipientID(recipient) + for _, existing := range sc.Recipients { + if efp, _ := recipientID(existing); efp == fp { + return false, nil + } + } + sc.Recipients = append(sc.Recipients, recipient) + sort.Strings(sc.Recipients) + s.Scopes[scope] = sc + return true, nil +} + +// Disallow removes a recipient from a scope and returns whether it was present. +// Removing a recipient does NOT revoke access to values already committed in +// history — callers must warn to rotate those values. +func (s *Scopes) Disallow(scope, recipient string) (bool, error) { + sc, ok := s.Scopes[scope] + if !ok { + return false, errors.Errorf("scope %q is not declared", scope) + } + fp, err := recipientID(recipient) + if err != nil { + return false, err + } + // Fresh slice — never filter in place: sc is a struct copy but shares the map + // value's backing array, so sc.Recipients[:0] would corrupt any retained slice. + kept := make([]string, 0, len(sc.Recipients)) + removed := false + for _, existing := range sc.Recipients { + if efp, _ := recipientID(existing); efp == fp { + removed = true + continue + } + kept = append(kept, existing) + } + sc.Recipients = kept + s.Scopes[scope] = sc + return removed, nil +} diff --git a/pkg/api/secrets/scoped/store.go b/pkg/api/secrets/scoped/store.go new file mode 100644 index 00000000..07c115a7 --- /dev/null +++ b/pkg/api/secrets/scoped/store.go @@ -0,0 +1,362 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) Simple Container + +package scoped + +import ( + "crypto" + "encoding/base64" + "fmt" + "os" + "path/filepath" + "sort" + "strings" + + "github.com/pkg/errors" + "gopkg.in/yaml.v3" + + "github.com/simple-container-com/api/pkg/api/secrets" + "github.com/simple-container-com/api/pkg/api/secrets/ciphers" +) + +// scopeFilePrefix / scopeFileSuffix bracket the scope name in a scope file's base +// name: secrets..yaml. +const ( + scopeFilePrefix = "secrets." + scopeFileSuffix = ".yaml" +) + +// EncryptedValue is one secret in envelope form: the value is AEAD-encrypted ONCE +// under a random data key (Ciphertext), and that data key is wrapped per recipient +// (Wraps, keyed by recipient ID — a SHA256 SSH fingerprint for an ssh key, or a +// normalized awskms:// URL for a KMS recipient). All recipients therefore decrypt +// the SAME value — a tampered slot yields a decrypt failure, never a different +// plaintext — and the value carries a single whole-message MAC (no chunk splicing). +// Committed as-is (opaque values, diffable structure). +type EncryptedValue struct { + Ciphertext string `yaml:"ciphertext"` + Wraps map[string][]string `yaml:"wraps"` +} + +// ScopeFile is a committed, encrypted secrets..yaml. Structure (keys, +// recipients) is readable; values are opaque. It is self-contained: the recipient +// list lets `sc secrets lint` verify the value fingerprints without a private key. +type ScopeFile struct { + SchemaVersion int `yaml:"schemaVersion"` + Stack string `yaml:"stack"` + Scope string `yaml:"scope"` + Recipients []string `yaml:"recipients"` + Values map[string]EncryptedValue `yaml:"values"` +} + +// ScopeFileName returns the base name for a scope: secrets..yaml. +func ScopeFileName(scope string) string { + return scopeFilePrefix + scope + scopeFileSuffix +} + +// ScopeNameFromFile extracts the scope from a scope file's path, or "" if the base +// name is not secrets..yaml. +func ScopeNameFromFile(path string) string { + base := filepath.Base(path) + if len(base) <= len(scopeFilePrefix)+len(scopeFileSuffix) { + return "" + } + if base[:len(scopeFilePrefix)] != scopeFilePrefix || base[len(base)-len(scopeFileSuffix):] != scopeFileSuffix { + return "" + } + return base[len(scopeFilePrefix) : len(base)-len(scopeFileSuffix)] +} + +// StackNameFromFile returns the stack a scope file belongs to — the name of its +// parent directory (.sc/stacks//secrets..yaml). Used to verify the +// file's self-declared stack against its actual location. +func StackNameFromFile(path string) string { + return filepath.Base(filepath.Dir(path)) +} + +// NewScopeFile creates an empty scope file bound to a stack, a scope, and its +// recipient set. The stack + scope are bound into every value's AAD, so a value +// cannot be transplanted to another stack or scope. +func NewScopeFile(stack, scope string, recipients []string) (*ScopeFile, error) { + if err := ValidateScopeName(scope); err != nil { + return nil, err + } + if strings.TrimSpace(stack) == "" { + return nil, errors.New("cannot create a scope file with no stack") + } + if len(recipients) == 0 { + return nil, errors.Errorf("cannot create scope %q with no recipients", scope) + } + return &ScopeFile{ + SchemaVersion: CurrentScopesSchemaVersion, + Stack: stack, + Scope: scope, + Recipients: append([]string(nil), recipients...), + Values: map[string]EncryptedValue{}, + }, nil +} + +// LoadScopeFile reads a scope file, fails closed on a too-new version, and verifies +// the in-file scope name matches the filename AND the in-file stack matches the +// parent directory — so copying secrets.prod.yaml into another stack's dir, or +// renaming it to another scope, is rejected here even before the per-value AAD +// binding would fail on decrypt. +func LoadScopeFile(path string) (*ScopeFile, error) { + data, err := os.ReadFile(path) + if err != nil { + return nil, errors.Wrapf(err, "failed to read scope file %s", path) + } + var f ScopeFile + if err := yaml.Unmarshal(data, &f); err != nil { + return nil, errors.Wrapf(err, "failed to parse scope file %s", path) + } + if f.SchemaVersion > CurrentScopesSchemaVersion { + return nil, errors.Wrapf(ErrScopesVersionUnsupported, "%s declares version %d, this build supports up to %d", path, f.SchemaVersion, CurrentScopesSchemaVersion) + } + if err := ValidateScopeName(f.Scope); err != nil { + return nil, errors.Wrapf(err, "in %s", path) + } + if fromName := ScopeNameFromFile(path); fromName != "" && fromName != f.Scope { + return nil, errors.Errorf("scope file %s declares scope %q but its filename says %q (renamed file?)", path, f.Scope, fromName) + } + if f.Stack == "" { + return nil, errors.Errorf("scope file %s has no stack field", path) + } + if fromDir := StackNameFromFile(path); fromDir != f.Stack { + return nil, errors.Errorf("scope file %s declares stack %q but lives under stack dir %q (moved file?)", path, f.Stack, fromDir) + } + if f.Values == nil { + f.Values = map[string]EncryptedValue{} + } + return &f, nil +} + +// Save writes the scope file with a stable key/recipient order for clean diffs. +func (f *ScopeFile) Save(path string) error { + if f.SchemaVersion == 0 { + f.SchemaVersion = CurrentScopesSchemaVersion + } + sort.Strings(f.Recipients) + data, err := yaml.Marshal(f) + if err != nil { + return errors.Wrap(err, "failed to marshal scope file") + } + if err := writeFileAtomic(path, data, 0o644); err != nil { + return errors.Wrapf(err, "failed to write scope file %s", path) + } + return nil +} + +// Set encrypts value under key to every recipient of this scope file. It fails if +// the file has no recipients, so a value is never written unencrypted or to an +// empty audience. +func (f *ScopeFile) Set(key, value string) error { + if err := ValidateSecretKey(key); err != nil { + return err + } + if len(f.Recipients) == 0 { + return errors.Errorf("scope file %q has no recipients", f.Scope) + } + enc, err := encryptForRecipients(f.Recipients, f.Stack, f.Scope, key, value) + if err != nil { + return err + } + if f.Values == nil { + f.Values = map[string]EncryptedValue{} + } + f.Values[key] = enc + return nil +} + +// Get decrypts key with privateKey (an unencrypted PEM SSH private key), verifying +// the (stack,scope,key) binding. Returns ErrRecipientNotAllowed (wrapped) if the +// key is not a recipient, and a distinct not-found error if the key is absent. This +// is the SSH-only path used for resealing (allow/disallow); the CLI `get`/`doctor` +// and deploy-time resolution use Open, which also tries KMS. +func (f *ScopeFile) Get(key, privateKey string) (string, error) { + enc, ok := f.Values[key] + if !ok { + return "", errors.Errorf("secret %q not found in scope %q", key, f.Scope) + } + // Surface clear parse errors (e.g. passphrase-protected) rather than a bare + // "not a recipient". + if _, _, err := privateKeyFingerprint(privateKey); err != nil { + return "", err + } + val, owned, err := NewOpener([]string{privateKey}, false).OpenValue(f.Stack, f.Scope, key, enc) + if err != nil { + return "", err + } + if !owned { + return "", errors.Wrapf(ErrRecipientNotAllowed, "key is not a recipient of %q in scope %q", key, f.Scope) + } + return val, nil +} + +// Open decrypts key using an Opener, which may hold several SSH keys and/or permit +// KMS Decrypt via the ambient AWS credentials. It returns (value, true, nil) on +// success, ("", false, nil) if the opener is not a usable recipient (least-privilege +// skip), and ("", true, err) on an integrity failure. It is the path used by the +// CLI and by deploy-time resolution. +func (f *ScopeFile) Open(key string, o *Opener) (string, bool, error) { + enc, ok := f.Values[key] + if !ok { + return "", false, errors.Errorf("secret %q not found in scope %q", key, f.Scope) + } + return o.OpenValue(f.Stack, f.Scope, key, enc) +} + +// Delete removes a key. Returns whether it was present. +func (f *ScopeFile) Delete(key string) bool { + if _, ok := f.Values[key]; !ok { + return false + } + delete(f.Values, key) + return true +} + +// Keys returns the secret names in sorted order. +func (f *ScopeFile) Keys() []string { + keys := make([]string, 0, len(f.Values)) + for k := range f.Values { + keys = append(keys, k) + } + sort.Strings(keys) + return keys +} + +// Reencrypt re-seals every value to newRecipients, using opener (which must hold a +// CURRENT recipient — an SSH key or KMS access) to decrypt each value first. Used by +// allow/disallow to roll the recipient set. Because it takes an Opener, a scope whose +// only current recipient is a KMS key can still be resealed by an operator with +// kms:Decrypt. It is all-or-nothing: if any value cannot be decrypted or re-sealed the +// file is left untouched, so a partial reseal never drops a recipient's access +// silently. Note: this does NOT rewrite git history — a removed recipient can still +// read prior committed versions, so callers must warn to rotate values on removal. +func (f *ScopeFile) Reencrypt(newRecipients []string, opener *Opener) error { + if len(newRecipients) == 0 { + return errors.Errorf("refusing to reseal scope %q to an empty recipient set", f.Scope) + } + // Decrypt everything first against the current recipient set. + plain := make(map[string]string, len(f.Values)) + for key := range f.Values { + v, owned, err := f.Open(key, opener) + if err != nil { + return errors.Wrapf(err, "cannot reseal scope %q: failed to decrypt %q", f.Scope, key) + } + if !owned { + return errors.Errorf("cannot reseal scope %q: the provided key/credentials are not a current recipient of %q", f.Scope, key) + } + plain[key] = v + } + // Build the new sealed set in a scratch file so a mid-way error can't corrupt f. + next := &ScopeFile{SchemaVersion: f.SchemaVersion, Stack: f.Stack, Scope: f.Scope, Recipients: append([]string(nil), newRecipients...), Values: map[string]EncryptedValue{}} + for key, val := range plain { + if err := next.Set(key, val); err != nil { + return errors.Wrapf(err, "cannot reseal scope %q: failed to re-encrypt %q", f.Scope, key) + } + } + f.Recipients = next.Recipients + f.Values = next.Values + return nil +} + +// VerifyConsistency is the offline (no private key) integrity check `sc secrets +// lint` runs: every value must be sealed to exactly the declared recipient set, +// keys/scope must be well-formed. It does NOT verify recipients against +// scopes.yaml — the caller does that so the drift error can name scopes.yaml. +func (f *ScopeFile) VerifyConsistency() error { + if err := ValidateScopeName(f.Scope); err != nil { + return err + } + if strings.TrimSpace(f.Stack) == "" { + return errors.Errorf("scope %q has no stack", f.Scope) + } + if len(f.Recipients) == 0 { + return errors.Errorf("scope %q has no recipients", f.Scope) + } + // Map each recipient ID to how its wrap must be shaped: an SSH recipient's wrap + // is a block of its key type (checkable via its public key); a KMS recipient's + // wrap is an opaque KMS ciphertext blob (checkable only for base64 + a minimum + // length that excludes a plaintext data key). + sshPub := make(map[string]crypto.PublicKey, len(f.Recipients)) + kmsIDs := make(map[string]struct{}, len(f.Recipients)) + for _, r := range f.Recipients { + id, err := recipientID(r) + if err != nil { + return err + } + if isKMSRecipient(r) { + if err := validateKMSRecipient(r); err != nil { + return errors.Wrapf(err, "scope %q recipient %s", f.Scope, id) + } + kmsIDs[id] = struct{}{} + continue + } + pub, err := ciphers.ParsePublicKey(secrets.TrimPubKey(r)) + if err != nil { + return errors.Wrapf(err, "scope %q recipient %s", f.Scope, id) + } + sshPub[id] = pub + } + nRecipients := len(sshPub) + len(kmsIDs) + for key, ev := range f.Values { + if err := ValidateSecretKey(key); err != nil { + return err + } + // The value blob: base64, and a plausibly-shaped envelope AEAD (nonce+tag), + // so a hand-edited plaintext value is rejected offline without a key. + blob, err := base64.StdEncoding.DecodeString(ev.Ciphertext) + if err != nil { + return errors.Wrapf(err, "scope %q key %q value ciphertext is not base64 (plaintext leak?)", f.Scope, key) + } + if err := ciphers.ValidEnvelopeBlob(blob); err != nil { + return errors.Wrapf(err, "scope %q key %q value ciphertext", f.Scope, key) + } + if len(ev.Wraps) != nRecipients { + return errors.Errorf("scope %q key %q data key wrapped for %d recipients, expected %d", f.Scope, key, len(ev.Wraps), nRecipients) + } + for id, chunks := range ev.Wraps { + if len(chunks) == 0 { + return errors.Errorf("scope %q key %q has an empty data-key wrap for recipient %s", f.Scope, key, id) + } + if _, isKMS := kmsIDs[id]; isKMS { + // A KMS wrap is a single opaque ciphertext blob; verify base64 + a + // minimum length so a plaintext data key smuggled into a KMS slot fails + // offline (its EncryptionContext binding is enforced by KMS at decrypt). + if len(chunks) != 1 { + return errors.Errorf("scope %q key %q KMS wrap for %s must be exactly one blob, got %d", f.Scope, key, id, len(chunks)) + } + raw, err := base64.StdEncoding.DecodeString(chunks[0]) + if err != nil { + return errors.Wrapf(err, "scope %q key %q KMS wrap for %s is not base64", f.Scope, key, id) + } + if len(raw) < kmsMinCiphertextLen { + return errors.Errorf("scope %q key %q KMS wrap for %s is %d bytes, below the %d-byte minimum (plaintext?)", f.Scope, key, id, len(raw), kmsMinCiphertextLen) + } + continue + } + pub, ok := sshPub[id] + if !ok { + return errors.Errorf("scope %q key %q wrapped for unknown recipient %s (not in recipients list)", f.Scope, key, id) + } + // Each SSH wrap is the 32-byte DEK sealed to the recipient — a single block + // of the recipient's key type (RSA modulus size, or an X25519 sealed box). + for i, chunk := range chunks { + raw, err := base64.StdEncoding.DecodeString(chunk) + if err != nil { + return errors.Wrapf(err, "scope %q key %q recipient %s wrap %d is not base64", f.Scope, key, id, i) + } + if err := ciphers.ValidateCiphertextShape(pub, raw); err != nil { + return errors.Wrapf(err, "scope %q key %q recipient %s wrap %d", f.Scope, key, id, i) + } + } + } + } + return nil +} + +// String renders a short human summary (scope + counts), never values. +func (f *ScopeFile) String() string { + return fmt.Sprintf("scope %q: %d value(s), %d recipient(s)", f.Scope, len(f.Values), len(f.Recipients)) +} diff --git a/pkg/cmd/cmd_secrets/cmd_scope.go b/pkg/cmd/cmd_secrets/cmd_scope.go new file mode 100644 index 00000000..7fe8cc4c --- /dev/null +++ b/pkg/cmd/cmd_secrets/cmd_scope.go @@ -0,0 +1,518 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) Simple Container + +package cmd_secrets + +import ( + "fmt" + "io" + "os" + "path/filepath" + "sort" + "strings" + + "github.com/pkg/errors" + "github.com/spf13/cobra" + + "github.com/simple-container-com/api/pkg/api" + "github.com/simple-container-com/api/pkg/api/secrets/scoped" +) + +// scopeCmd carries the shared flags for the `secrets scope` verbs. Scoped secrets +// live in .sc/stacks//secrets..yaml, governed by .sc/scopes.yaml — +// a per-scope recipient set so a pull_request CI job can hold a key that opens +// only its scope, not the whole-file store. +type scopeCmd struct { + *secretsCmd + scope string + stack string + keyFile string +} + +// scDir returns the .sc config directory for the current repo. +func (s *scopeCmd) scDir() string { + return filepath.Join(s.Root.Provisioner.Cryptor().Workdir(), api.ScConfigDirectory) +} + +// privateKey resolves the private key used to decrypt scoped values, in order: +// 1. --key-file (an explicit PEM file); +// 2. env SC_KEY_ (e.g. SC_KEY_PR) — the per-scope CI key described by the +// rollout — then the generic SC_SCOPE_KEY; +// 3. the ambient cryptor key (SIMPLE_CONTAINER_CONFIG / profile) for local use. +// +// This lets a pull_request scan job hold ONLY the scope key without also carrying +// a full SIMPLE_CONTAINER_CONFIG. +func (s *scopeCmd) privateKey() (string, error) { + if s.keyFile != "" { + b, err := os.ReadFile(s.keyFile) + if err != nil { + return "", errors.Wrapf(err, "failed to read --key-file %s", s.keyFile) + } + return string(b), nil + } + if s.scope != "" { + envName := "SC_KEY_" + strings.ToUpper(strings.ReplaceAll(s.scope, "-", "_")) + if v := os.Getenv(envName); strings.TrimSpace(v) != "" { + return v, nil + } + } + if v := os.Getenv("SC_SCOPE_KEY"); strings.TrimSpace(v) != "" { + return v, nil + } + pk := s.Root.Provisioner.Cryptor().PrivateKey() + if strings.TrimSpace(pk) == "" { + return "", errors.New("no private key available: set --key-file, SC_KEY_ / SC_SCOPE_KEY, or SIMPLE_CONTAINER_CONFIG") + } + return pk, nil +} + +func NewScopeCmd(sCmd *secretsCmd) *cobra.Command { + cmd := &cobra.Command{ + Use: "scope", + Short: "Manage per-scope secrets (secrets..yaml) with per-scope recipients", + Long: "Per-scope secrets let a narrow CI context (e.g. pull_request scan jobs) hold a key " + + "that decrypts only one scope instead of the whole-file store. Recipients are governed " + + "by .sc/scopes.yaml (CODEOWNERS-gated); values live in .sc/stacks//secrets..yaml.", + SilenceUsage: true, + } + cmd.AddCommand( + newScopeSetCmd(sCmd), + newScopeGetCmd(sCmd), + newScopeListCmd(sCmd), + newScopeDeleteCmd(sCmd), + newScopeAllowCmd(sCmd), + newScopeDisallowCmd(sCmd), + newScopeLintCmd(sCmd), + newScopeDoctorCmd(sCmd), + ) + return cmd +} + +// addScopeStackFlags registers --scope (required) and -s/--stack (required) on a +// value-level command. +func (s *scopeCmd) addScopeStackFlags(cmd *cobra.Command) { + cmd.Flags().StringVar(&s.scope, "scope", "", "scope name (e.g. pr)") + cmd.Flags().StringVarP(&s.stack, "stack", "s", "", "stack name (secrets..yaml under .sc/stacks/)") + _ = cmd.MarkFlagRequired("scope") + _ = cmd.MarkFlagRequired("stack") +} + +// openForWrite loads (or creates) the scope file for --stack/--scope, and verifies +// its recipients match the authoritative scopes.yaml set — refusing to write into +// a file whose audience has drifted (reconcile via allow/disallow first). +func (s *scopeCmd) openForWrite() (*scoped.ScopeFile, *scoped.Scopes, string, error) { + if err := scoped.ValidateScopeName(s.scope); err != nil { + return nil, nil, "", err + } + sc, err := scoped.LoadScopes(scoped.ScopesPath(s.scDir())) + if err != nil { + return nil, nil, "", err + } + recipients, err := sc.Recipients(s.scope) + if err != nil { + return nil, nil, "", errors.Wrapf(err, "declare the scope first with `sc secrets scope allow`") + } + path := scoped.ScopeFilePath(s.scDir(), s.stack, s.scope) + var f *scoped.ScopeFile + if _, statErr := os.Stat(path); statErr == nil { + if f, err = scoped.LoadScopeFile(path); err != nil { + return nil, nil, "", err + } + if err := scoped.SameRecipients(f.Recipients, recipients); err != nil { + return nil, nil, "", errors.Wrapf(err, "%s recipients drifted from %s — reconcile with `sc secrets scope allow/disallow`", filepath.Base(path), scoped.ScopesFileName) + } + } else if os.IsNotExist(statErr) { + if f, err = scoped.NewScopeFile(s.stack, s.scope, recipients); err != nil { + return nil, nil, "", err + } + } else { + return nil, nil, "", errors.Wrapf(statErr, "failed to stat %s", path) + } + return f, sc, path, nil +} + +func newScopeSetCmd(sCmd *secretsCmd) *cobra.Command { + s := &scopeCmd{secretsCmd: sCmd} + cmd := &cobra.Command{ + Use: "set KEY [VALUE]", + Short: "Seal a value into a scope (VALUE from arg, or '-'/omitted reads stdin)", + Long: "Seal a value into a scope. The value is taken from the VALUE argument, or " + + "read from stdin when VALUE is omitted or '-'. When read from stdin, a single " + + "trailing newline is stripped (the usual echo/heredoc artifact); pipe binary or " + + "exact-match data via the VALUE argument if that matters.", + Args: cobra.RangeArgs(1, 2), + RunE: func(cmd *cobra.Command, args []string) error { + key := args[0] + value, err := readValueArg(cmd, args) + if err != nil { + return err + } + f, _, path, err := s.openForWrite() + if err != nil { + return err + } + if err := f.Set(key, value); err != nil { + return err + } + if err := f.Save(path); err != nil { + return err + } + fmt.Fprintf(cmd.OutOrStdout(), "sealed %q into scope %q (%d recipient(s))\n", key, s.scope, len(f.Recipients)) + return nil + }, + } + s.addScopeStackFlags(cmd) + return cmd +} + +func newScopeGetCmd(sCmd *secretsCmd) *cobra.Command { + s := &scopeCmd{secretsCmd: sCmd} + cmd := &cobra.Command{ + Use: "get KEY", + Short: "Decrypt one scoped value with a scope key or ambient AWS (KMS) credentials", + Args: cobra.ExactArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + if err := scoped.ValidateScopeName(s.scope); err != nil { + return err + } + // An SSH key is optional: a KMS-recipient scope is opened via the ambient + // AWS credential chain (e.g. an OIDC role) with no private key at all. + pk, pkErr := s.privateKey() + var keys []string + if pkErr == nil && strings.TrimSpace(pk) != "" { + keys = append(keys, pk) + } + path := scoped.ScopeFilePath(s.scDir(), s.stack, s.scope) + f, err := scoped.LoadScopeFile(path) + if err != nil { + return err + } + val, owned, err := f.Open(args[0], scoped.NewOpener(keys, true)) + if err != nil { + return err + } + if !owned { + if pkErr != nil { + return errors.Wrapf(pkErr, "no SSH key and no KMS recipient could open %q in scope %q", args[0], s.scope) + } + return errors.Errorf("key is not a recipient of %q in scope %q (and no KMS recipient was openable)", args[0], s.scope) + } + fmt.Fprintln(cmd.OutOrStdout(), val) + return nil + }, + } + s.addScopeStackFlags(cmd) + cmd.Flags().StringVar(&s.keyFile, "key-file", "", "PEM private key to decrypt with (else SC_KEY_ / SC_SCOPE_KEY / ambient config / ambient AWS for KMS recipients)") + return cmd +} + +func newScopeListCmd(sCmd *secretsCmd) *cobra.Command { + s := &scopeCmd{secretsCmd: sCmd} + cmd := &cobra.Command{ + Use: "list", + Short: "List secret names in a scope (values are never printed)", + Args: cobra.NoArgs, + RunE: func(cmd *cobra.Command, args []string) error { + path := scoped.ScopeFilePath(s.scDir(), s.stack, s.scope) + f, err := scoped.LoadScopeFile(path) + if err != nil { + return err + } + for _, k := range f.Keys() { + fmt.Fprintln(cmd.OutOrStdout(), k) + } + return nil + }, + } + s.addScopeStackFlags(cmd) + return cmd +} + +func newScopeDeleteCmd(sCmd *secretsCmd) *cobra.Command { + s := &scopeCmd{secretsCmd: sCmd} + cmd := &cobra.Command{ + Use: "delete KEY", + Short: "Remove a value from a scope", + Args: cobra.ExactArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + path := scoped.ScopeFilePath(s.scDir(), s.stack, s.scope) + f, err := scoped.LoadScopeFile(path) + if err != nil { + return err + } + if !f.Delete(args[0]) { + return errors.Errorf("secret %q not found in scope %q", args[0], s.scope) + } + return f.Save(path) + }, + } + s.addScopeStackFlags(cmd) + return cmd +} + +func newScopeAllowCmd(sCmd *secretsCmd) *cobra.Command { + s := &scopeCmd{secretsCmd: sCmd} + cmd := &cobra.Command{ + Use: "allow RECIPIENT", + Short: "Add a recipient — SSH pubkey or awskms://?region= — to a scope (updates scopes.yaml and reseals its files)", + Args: cobra.ExactArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + return s.reconcileRecipients(cmd, args[0], true) + }, + } + cmd.Flags().StringVar(&s.scope, "scope", "", "scope name (e.g. pr)") + _ = cmd.MarkFlagRequired("scope") + return cmd +} + +func newScopeDisallowCmd(sCmd *secretsCmd) *cobra.Command { + s := &scopeCmd{secretsCmd: sCmd} + cmd := &cobra.Command{ + Use: "disallow RECIPIENT", + Short: "Remove a recipient — SSH pubkey or awskms:// URL — from a scope (reseals; prints rotate-values warning)", + Args: cobra.ExactArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + return s.reconcileRecipients(cmd, args[0], false) + }, + } + cmd.Flags().StringVar(&s.scope, "scope", "", "scope name (e.g. pr)") + _ = cmd.MarkFlagRequired("scope") + return cmd +} + +// reconcileRecipients updates scopes.yaml then reseals every scope file of the +// scope to the new recipient set. Resealing requires the ambient key to be a +// current recipient (it decrypts to re-encrypt). +func (s *scopeCmd) reconcileRecipients(cmd *cobra.Command, pubKey string, allow bool) error { + if err := scoped.ValidateScopeName(s.scope); err != nil { + return err + } + scopesPath := scoped.ScopesPath(s.scDir()) + sc, err := scoped.LoadScopes(scopesPath) + if err != nil { + return err + } + if allow { + changed, aErr := sc.Allow(s.scope, pubKey) + if aErr != nil { + return aErr + } + if !changed { + fmt.Fprintf(cmd.OutOrStdout(), "recipient already present in scope %q; nothing to do\n", s.scope) + return nil + } + } else { + removed, dErr := sc.Disallow(s.scope, pubKey) + if dErr != nil { + return dErr + } + if !removed { + fmt.Fprintf(cmd.OutOrStdout(), "recipient not present in scope %q; nothing to do\n", s.scope) + return nil + } + // Refuse to strand a scope with no recipients — the resulting store would be + // undecryptable and every value orphaned. Deleting the scope is the explicit + // path for that. + if left := sc.Scopes[s.scope].Recipients; len(left) == 0 { + return errors.Errorf("refusing to remove the last recipient of scope %q; delete the scope's files and its %s entry instead", s.scope, scoped.ScopesFileName) + } + } + recipients, err := sc.Recipients(s.scope) + if err != nil { + return err + } + // Phase 1: load + reseal every scope file of this scope IN MEMORY. The private + // key is fetched lazily — only files that actually hold values need decrypting, + // so declaring the first recipient of an empty scope needs no key. Nothing is + // written until every reseal succeeds, so a mid-way decrypt/parse failure can + // never leave some files resealed and scopes.yaml/other files behind (drift). + files, err := scoped.ListScopeFiles(s.scDir()) + if err != nil { + return err + } + type pendingSave struct { + f *scoped.ScopeFile + path string + } + var pending []pendingSave + var opener *scoped.Opener + for _, path := range files { + if scoped.ScopeNameFromFile(path) != s.scope { + continue + } + f, lErr := scoped.LoadScopeFile(path) + if lErr != nil { + return lErr + } + if len(f.Values) > 0 { + if opener == nil { + // Reseal decrypts current values first: build an Opener from any SSH key + // we have PLUS KMS (so a scope whose current recipient is a KMS key can + // still be resealed by an operator with kms:Decrypt). A missing SSH key is + // not fatal here — KMS may open it; Reencrypt fails closed if neither can. + var keys []string + if pk, kErr := s.privateKey(); kErr == nil && strings.TrimSpace(pk) != "" { + keys = append(keys, pk) + } + opener = scoped.NewOpener(keys, true) + } + if err := f.Reencrypt(recipients, opener); err != nil { + return err + } + } else { + f.Recipients = recipients + } + pending = append(pending, pendingSave{f: f, path: path}) + } + // Phase 2: persist. Write the resealed files first, then scopes.yaml last, so a + // reader never sees scopes.yaml advertise a recipient a file hasn't been + // resealed for. + for _, p := range pending { + if err := p.f.Save(p.path); err != nil { + return err + } + } + if err := sc.Save(scopesPath); err != nil { + return err + } + fmt.Fprintf(cmd.OutOrStdout(), "scope %q now has %d recipient(s); resealed %d file(s)\n", s.scope, len(recipients), len(pending)) + if !allow { + fmt.Fprintf(cmd.OutOrStderr(), "WARNING: removing a recipient does NOT revoke access to values already committed in git history. Rotate every value in scope %q now.\n", s.scope) + } + return nil +} + +func newScopeLintCmd(sCmd *secretsCmd) *cobra.Command { + s := &scopeCmd{secretsCmd: sCmd} + cmd := &cobra.Command{ + Use: "lint", + Short: "Verify every scope file: encryption, recipient set vs scopes.yaml, scope/filename binding", + Args: cobra.NoArgs, + RunE: func(cmd *cobra.Command, args []string) error { + scDir := s.scDir() + sc, err := scoped.LoadScopes(scoped.ScopesPath(scDir)) + if err != nil { + return err + } + files, err := scoped.ListScopeFiles(scDir) + if err != nil { + return err + } + var problems []string + // keyScopes[stack][key] = scopes that define it, to catch cross-scope + // duplicates (the resolver hard-fails on these at deploy; lint catches + // them first). + keyScopes := map[string]map[string][]string{} + for _, path := range files { + f, lErr := scoped.LoadScopeFile(path) + if lErr != nil { + problems = append(problems, lErr.Error()) + continue + } + if vErr := f.VerifyConsistency(); vErr != nil { + problems = append(problems, vErr.Error()) + } + want, rErr := sc.Recipients(f.Scope) + if rErr != nil { + problems = append(problems, fmt.Sprintf("%s: %s", filepath.Base(path), rErr.Error())) + } else if dErr := scoped.SameRecipients(f.Recipients, want); dErr != nil { + problems = append(problems, fmt.Sprintf("%s: recipients drift vs %s: %s", filepath.Base(path), scoped.ScopesFileName, dErr.Error())) + } + if keyScopes[f.Stack] == nil { + keyScopes[f.Stack] = map[string][]string{} + } + for _, k := range f.Keys() { + keyScopes[f.Stack][k] = append(keyScopes[f.Stack][k], f.Scope) + } + } + // A key must live in exactly one mode/scope so deploy-time resolution is + // deterministic: flag a key present in >1 scope, or in both a scope and + // the stack's legacy secrets.yaml (mode A, which wins silently at deploy). + for stack, keys := range keyScopes { + var legacy map[string]string + legacyPath := filepath.Join(scoped.StackDir(scDir, stack), api.SecretsDescriptorFileName) + if _, statErr := os.Stat(legacyPath); statErr == nil { + if d, rErr := api.ReadDescriptor(legacyPath, &api.SecretsDescriptor{}); rErr == nil { + legacy = d.Values + } + } + for k, scopes := range keys { + if len(scopes) > 1 { + sort.Strings(scopes) + problems = append(problems, fmt.Sprintf("stack %q: key %q is defined in multiple scopes %v (ambiguous at deploy)", stack, k, scopes)) + } + if _, inLegacy := legacy[k]; inLegacy { + problems = append(problems, fmt.Sprintf("stack %q: key %q is in both scope %q and the legacy secrets.yaml (mode A wins silently)", stack, k, scopes[0])) + } + } + } + if len(problems) > 0 { + for _, p := range problems { + fmt.Fprintf(cmd.OutOrStderr(), "✗ %s\n", p) + } + return errors.Errorf("scoped secrets lint failed: %d problem(s)", len(problems)) + } + fmt.Fprintf(cmd.OutOrStdout(), "✓ %d scope file(s) OK\n", len(files)) + return nil + }, + } + return cmd +} + +func newScopeDoctorCmd(sCmd *secretsCmd) *cobra.Command { + s := &scopeCmd{secretsCmd: sCmd} + cmd := &cobra.Command{ + Use: "doctor", + Short: "Report which scopes the current key or AWS (KMS) credentials can open", + Args: cobra.NoArgs, + RunE: func(cmd *cobra.Command, args []string) error { + // SSH key is optional — a KMS recipient is opened via ambient AWS creds. + pk, pkErr := s.privateKey() + var keys []string + if pkErr == nil && strings.TrimSpace(pk) != "" { + keys = append(keys, pk) + } + opener := scoped.NewOpener(keys, true) + files, err := scoped.ListScopeFiles(s.scDir()) + if err != nil { + return err + } + for _, path := range files { + f, lErr := scoped.LoadScopeFile(path) + if lErr != nil { + fmt.Fprintf(cmd.OutOrStdout(), "? scope=? file=%s (unreadable: %v)\n", filepath.Base(path), lErr) + continue + } + status := "no" + if len(f.Keys()) == 0 { + status = "empty" + } else if _, owned, gErr := f.Open(f.Keys()[0], opener); gErr != nil { + status = "ERR" + } else if owned { + status = "YES" + } + fmt.Fprintf(cmd.OutOrStdout(), "%-5s scope=%s file=%s\n", status, f.Scope, filepath.Base(path)) + } + return nil + }, + } + cmd.Flags().StringVar(&s.keyFile, "key-file", "", "PEM private key to test with (else SC_KEY_ / SC_SCOPE_KEY / ambient config / ambient AWS for KMS)") + return cmd +} + +// readValueArg returns the value from args[1], or reads stdin when args[1] is +// absent or "-". +func readValueArg(cmd *cobra.Command, args []string) (string, error) { + if len(args) == 2 && args[1] != "-" { + return args[1], nil + } + data, err := io.ReadAll(cmd.InOrStdin()) + if err != nil { + return "", errors.Wrap(err, "failed to read value from stdin") + } + // Strip a SINGLE trailing newline — the usual `echo`/heredoc artifact — rather than + // all trailing newlines, so a value with intentional trailing newlines (e.g. a PEM + // key piped via `set KEY - < key.pem`) keeps all but the last. Also drop a trailing + // CR so a CRLF line ending (Windows / some editors) does not leave a stray \r. + return strings.TrimSuffix(strings.TrimSuffix(string(data), "\n"), "\r"), nil +} diff --git a/pkg/cmd/cmd_secrets/cmd_scope_test.go b/pkg/cmd/cmd_secrets/cmd_scope_test.go new file mode 100644 index 00000000..9fda610d --- /dev/null +++ b/pkg/cmd/cmd_secrets/cmd_scope_test.go @@ -0,0 +1,191 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) Simple Container + +package cmd_secrets + +import ( + "bytes" + "os" + "path/filepath" + "strings" + "testing" + + . "github.com/onsi/gomega" + "golang.org/x/crypto/ssh" + + "github.com/simple-container-com/api/pkg/api/secrets" + "github.com/simple-container-com/api/pkg/api/secrets/ciphers" + "github.com/simple-container-com/api/pkg/cmd/root_cmd" + "github.com/simple-container-com/api/pkg/provisioner" + "github.com/simple-container-com/api/pkg/provisioner/placeholders" +) + +// execScope drives the real `secrets scope` command tree against a temp workdir. +// A fresh command tree is built per call so cobra flag state does not leak. +func execScope(t *testing.T, workdir, stdin string, args ...string) (string, error) { + t.Helper() + g := NewWithT(t) + cryptor, err := secrets.NewCryptor(workdir) + g.Expect(err).NotTo(HaveOccurred()) + p, err := provisioner.New(provisioner.WithCryptor(cryptor), provisioner.WithPlaceholders(placeholders.New())) + g.Expect(err).NotTo(HaveOccurred()) + cmd := NewScopeCmd(&secretsCmd{Root: &root_cmd.RootCmd{Provisioner: p}}) + var out bytes.Buffer + cmd.SetOut(&out) + cmd.SetErr(&out) + cmd.SetIn(strings.NewReader(stdin)) + cmd.SetArgs(args) + execErr := cmd.Execute() + return out.String(), execErr +} + +func testRecipient(t *testing.T) (authorized, privatePEM string) { + t.Helper() + g := NewWithT(t) + priv, pub, err := ciphers.GenerateEd25519KeyPair() + g.Expect(err).NotTo(HaveOccurred()) + sshPub, err := ssh.NewPublicKey(pub) + g.Expect(err).NotTo(HaveOccurred()) + pem, err := ciphers.MarshalEd25519PrivateKey(priv) + g.Expect(err).NotTo(HaveOccurred()) + return string(ssh.MarshalAuthorizedKey(sshPub)), pem +} + +func TestScopeCmd_SetGetListLintDoctor(t *testing.T) { + RegisterTestingT(t) + workdir := t.TempDir() + authorized, keyPEM := testRecipient(t) + + // allow (no key needed — no files yet) + out, err := execScope(t, workdir, "", "allow", "--scope", "pr", authorized) + Expect(err).NotTo(HaveOccurred(), out) + + // set (arg value + stdin value) + _, err = execScope(t, workdir, "", "set", "--scope", "pr", "-s", "integrail", "defectdojo-api-key", "dd-123") + Expect(err).NotTo(HaveOccurred()) + _, err = execScope(t, workdir, "stdin-secret", "set", "--scope", "pr", "-s", "integrail", "cf-token", "-") + Expect(err).NotTo(HaveOccurred()) + + // list + out, _ = execScope(t, workdir, "", "list", "--scope", "pr", "-s", "integrail") + Expect(out).To(ContainSubstring("defectdojo-api-key")) + Expect(out).To(ContainSubstring("cf-token")) + + // get via SC_SCOPE_KEY env (the CI path, no ambient config) + t.Setenv("SC_SCOPE_KEY", keyPEM) + out, err = execScope(t, workdir, "", "get", "--scope", "pr", "-s", "integrail", "defectdojo-api-key") + Expect(err).NotTo(HaveOccurred()) + Expect(strings.TrimSpace(out)).To(Equal("dd-123")) + out, _ = execScope(t, workdir, "", "get", "--scope", "pr", "-s", "integrail", "cf-token") + Expect(strings.TrimSpace(out)).To(Equal("stdin-secret")) + + // lint clean + out, err = execScope(t, workdir, "", "lint") + Expect(err).NotTo(HaveOccurred(), out) + Expect(out).To(ContainSubstring("OK")) + + // doctor: openable with the scope key + out, _ = execScope(t, workdir, "", "doctor") + Expect(out).To(ContainSubstring("YES")) +} + +func TestScopeCmd_LintCatchesCrossScopeDuplicate(t *testing.T) { + RegisterTestingT(t) + workdir := t.TempDir() + authorized, _ := testRecipient(t) + + _, err := execScope(t, workdir, "", "allow", "--scope", "pr", authorized) + Expect(err).NotTo(HaveOccurred()) + _, err = execScope(t, workdir, "", "allow", "--scope", "prod", authorized) + Expect(err).NotTo(HaveOccurred()) + _, err = execScope(t, workdir, "", "set", "--scope", "pr", "-s", "integrail", "shared", "a") + Expect(err).NotTo(HaveOccurred()) + _, err = execScope(t, workdir, "", "set", "--scope", "prod", "-s", "integrail", "shared", "b") + Expect(err).NotTo(HaveOccurred()) + + out, err := execScope(t, workdir, "", "lint") + Expect(err).To(HaveOccurred()) + Expect(out).To(ContainSubstring("multiple scopes")) +} + +func TestScopeCmd_DisallowLastRecipientRefused(t *testing.T) { + RegisterTestingT(t) + workdir := t.TempDir() + authorized, _ := testRecipient(t) + + _, err := execScope(t, workdir, "", "allow", "--scope", "pr", authorized) + Expect(err).NotTo(HaveOccurred()) + _, err = execScope(t, workdir, "", "set", "--scope", "pr", "-s", "integrail", "k", "v") + Expect(err).NotTo(HaveOccurred()) + + out, err := execScope(t, workdir, "", "disallow", "--scope", "pr", authorized) + Expect(err).To(HaveOccurred()) + Expect(err.Error() + out).To(ContainSubstring("last recipient")) +} + +func TestScopeCmd_ReconcileFailsClosedWithoutKey(t *testing.T) { + RegisterTestingT(t) + workdir := t.TempDir() + authA, _ := testRecipient(t) + authB, _ := testRecipient(t) + + _, err := execScope(t, workdir, "", "allow", "--scope", "pr", authA) + Expect(err).NotTo(HaveOccurred()) + _, err = execScope(t, workdir, "", "set", "--scope", "pr", "-s", "integrail", "k", "v") + Expect(err).NotTo(HaveOccurred()) + + // Adding recipient B needs to reseal the populated file, which needs a key that + // can decrypt it — none is available here, so allow must fail and persist nothing. + _, err = execScope(t, workdir, "", "allow", "--scope", "pr", authB) + Expect(err).To(HaveOccurred()) + + // No drift: the scope file's recipients still match scopes.yaml (A only). + out, lintErr := execScope(t, workdir, "", "lint") + Expect(lintErr).NotTo(HaveOccurred(), out) +} + +func TestScopeCmd_SetRefusesUndeclaredScope(t *testing.T) { + RegisterTestingT(t) + workdir := t.TempDir() + // No allow first → scope not in scopes.yaml → set must refuse. + out, err := execScope(t, workdir, "", "set", "--scope", "pr", "-s", "integrail", "k", "v") + Expect(err).To(HaveOccurred()) + Expect(err.Error() + out).To(ContainSubstring("scope")) +} + +// TestScopeCmd_KMSRecipientGovernance covers the v2 KMS-recipient additions at the +// CLI layer without any AWS call: allow accepts an awskms:// URL and persists it, +// a malformed KMS URL is rejected at governance time, and disallow removes it. The +// KMS crypto itself (wrap/unwrap, EncryptionContext binding) is covered by the +// scoped package's unit tests with a fake KMS client. +func TestScopeCmd_KMSRecipientGovernance(t *testing.T) { + RegisterTestingT(t) + workdir := t.TempDir() + sshAuth, _ := testRecipient(t) + kmsRec := "awskms://alias/sc-ci-pr?region=us-east-1" + + // SSH break-glass recipient + KMS recipient in the same scope. + out, err := execScope(t, workdir, "", "allow", "--scope", "pr", sshAuth) + Expect(err).NotTo(HaveOccurred(), out) + out, err = execScope(t, workdir, "", "allow", "--scope", "pr", kmsRec) + Expect(err).NotTo(HaveOccurred(), out) + + // The KMS recipient landed in scopes.yaml. + scopesYAML, rErr := os.ReadFile(filepath.Join(workdir, ".sc", "scopes.yaml")) + Expect(rErr).NotTo(HaveOccurred()) + Expect(string(scopesYAML)).To(ContainSubstring(kmsRec)) + + // A malformed KMS URL (no region, not an ARN) is rejected at allow time. + _, err = execScope(t, workdir, "", "allow", "--scope", "pr", "awskms://alias/no-region") + Expect(err).To(HaveOccurred()) + + // lint passes (recipients declared; no scope files with values yet). + out, err = execScope(t, workdir, "", "lint") + Expect(err).NotTo(HaveOccurred(), out) + + // disallow removes the KMS recipient (no scope files → no reseal / no KMS call). + out, err = execScope(t, workdir, "", "disallow", "--scope", "pr", kmsRec) + Expect(err).NotTo(HaveOccurred(), out) + scopesYAML, _ = os.ReadFile(filepath.Join(workdir, ".sc", "scopes.yaml")) + Expect(string(scopesYAML)).NotTo(ContainSubstring(kmsRec)) +} diff --git a/pkg/cmd/cmd_secrets/cmd_secrets.go b/pkg/cmd/cmd_secrets/cmd_secrets.go index f154ac4c..ff7f36a2 100644 --- a/pkg/cmd/cmd_secrets/cmd_secrets.go +++ b/pkg/cmd/cmd_secrets/cmd_secrets.go @@ -33,6 +33,7 @@ func NewSecretsCmd(rootCmd *root_cmd.RootCmd) *cobra.Command { NewAddCmd(sCmd), NewDeleteCmd(sCmd), NewInitCmd(sCmd), + NewScopeCmd(sCmd), ) return cmd } diff --git a/pkg/provisioner/provision.go b/pkg/provisioner/provision.go index 1d78675e..6cd01ade 100644 --- a/pkg/provisioner/provision.go +++ b/pkg/provisioner/provision.go @@ -14,6 +14,7 @@ import ( "github.com/samber/lo" "github.com/simple-container-com/api/pkg/api" + "github.com/simple-container-com/api/pkg/api/secrets/scoped" ) func (p *provisioner) Provision(ctx context.Context, params api.ProvisionParams) error { @@ -121,7 +122,10 @@ func (p *provisioner) ReadStacks(ctx context.Context, cfg *api.ConfigFile, param p.log.Debug(ctx, "Secrets descriptor not found for %s", stackName) } - if secretsDesc, err := p.readSecretsDescriptor(stacksDir, stackName); err != nil && (!readOpts.IgnoreSecretsMissing || lo.Contains(readOpts.RequireSecretConfigs, stackName)) { + if secretsDesc, err := p.readSecretsDescriptor(ctx, stacksDir, stackName); err != nil && (errors.Is(err, scoped.ErrScopedIntegrity) || errors.Is(err, scoped.ErrScopedUnavailable) || !readOpts.IgnoreSecretsMissing || lo.Contains(readOpts.RequireSecretConfigs, stackName)) { + // A scoped integrity failure (tamper / corrupt / ambiguous) OR a transient + // backend outage (KMS throttle) is fatal even under IgnoreSecretsMissing — a + // secret we could not resolve is never treated as "simply absent". return err } else if secretsDesc != nil { // SECURITY: Never log actual secrets descriptor content - contains credential values @@ -191,20 +195,91 @@ func (p *provisioner) readServerDescriptor(rootDir string, stackName string) (*a } } -func (p *provisioner) readSecretsDescriptor(rootDir string, stackName string) (*api.SecretsDescriptor, error) { +func (p *provisioner) readSecretsDescriptor(ctx context.Context, rootDir string, stackName string) (*api.SecretsDescriptor, error) { descFilePath := path.Join(rootDir, stackName, api.SecretsDescriptorFileName) + legacyExists := true if _, err := os.Stat(descFilePath); errors.Is(err, os.ErrNotExist) { - return nil, errors.Wrapf(err, "file not found: %q", descFilePath) + legacyExists = false } - return p.readSecretsDescriptorFromFile(descFilePath) + return p.readSecretsDescriptorFromFile(ctx, descFilePath, legacyExists) } -func (p *provisioner) readSecretsDescriptorFromFile(descFilePath string) (*api.SecretsDescriptor, error) { - if desc, err := api.ReadSecretsDescriptor(descFilePath); err != nil { - return nil, errors.Wrapf(err, "failed to read secrets descriptor from %q", descFilePath) - } else { - return desc, nil +func (p *provisioner) readSecretsDescriptorFromFile(ctx context.Context, descFilePath string, legacyExists bool) (*api.SecretsDescriptor, error) { + desc := &api.SecretsDescriptor{} + if legacyExists { + d, err := api.ReadSecretsDescriptor(descFilePath) + if err != nil { + return nil, errors.Wrapf(err, "failed to read secrets descriptor from %q", descFilePath) + } + desc = d + } + // Additively merge any per-scope secrets (secrets..yaml) that a candidate + // key — the ambient config key OR a CI scope key (SC_KEY_ / SC_SCOPE_KEY) — + // is a recipient of. Repos without scope files are unaffected. Whole-file values + // win on conflict, so a scoped value never changes an existing ${secret:} result; + // only new keys are added. The pull_request clamp is cryptographic — a scope key + // that is not a recipient of secrets.prod.yaml cannot open it. Integrity failures + // (tamper / corrupt / ambiguous) are tagged scoped.ErrScopedIntegrity and MUST NOT + // be swallowed by IgnoreSecretsMissing (see ReadStacks). + scopedVals, sErr := scoped.ResolveScopedValues(path.Dir(descFilePath), p.scopeCandidateKeys()) + if sErr != nil { + return nil, sErr + } + for k, v := range scopedVals { + if _, exists := desc.Values[k]; exists { + // The legacy whole-file store wins on conflict (an actor who can only write a + // scope file cannot override a legacy secret). `sc secrets scope lint` flags + // this collision, but lint is not always a required check — so warn at deploy + // too, or an operator who set a scoped value silently gets the legacy one. + if p.log != nil { + p.log.Warn(ctx, "scoped secret %q is shadowed by the legacy secrets.yaml for this stack; the legacy value is used. Remove one (see `sc secrets scope lint`).", k) + } + continue + } + if desc.Values == nil { + desc.Values = map[string]string{} + } + desc.Values[k] = v + } + // A stack with neither a legacy secrets.yaml nor any openable scoped value has no + // secrets for this key — preserve the previous "not found" behavior (ignorable + // under IgnoreSecretsMissing) rather than returning an empty descriptor. + if !legacyExists && len(desc.Values) == 0 && len(desc.Auth) == 0 { + return nil, errors.Wrapf(os.ErrNotExist, "file not found: %q (and no openable scoped secrets)", descFilePath) + } + return desc, nil +} + +// scopeCandidateKeys gathers every private key that might open a scope file: the +// ambient cryptor key (from SIMPLE_CONTAINER_CONFIG / profile) plus CI scope keys +// supplied without a full config — a generic SC_SCOPE_KEY and any per-scope +// SC_KEY_ (e.g. SC_KEY_PR). This lets a pull_request job resolve scoped +// secrets holding ONLY its scope key. +func (p *provisioner) scopeCandidateKeys() []string { + var keys []string + if p.cryptor != nil { + if pk := p.cryptor.PrivateKey(); strings.TrimSpace(pk) != "" { + keys = append(keys, pk) + } + } + if v := os.Getenv("SC_SCOPE_KEY"); strings.TrimSpace(v) != "" { + keys = append(keys, v) + } + // Accept only SC_KEY_ where maps back to a valid scope name + // (uppercase, '-'→'_'), so an unrelated SC_KEY_* env var is not blindly tried as + // a decryption key. A job with several scope keys resolves the union of its scopes. + for _, e := range os.Environ() { + name, val, ok := strings.Cut(e, "=") + if !ok || strings.TrimSpace(val) == "" || !strings.HasPrefix(name, "SC_KEY_") { + continue + } + scopeName := strings.ToLower(strings.ReplaceAll(strings.TrimPrefix(name, "SC_KEY_"), "_", "-")) + if scoped.ValidateScopeName(scopeName) != nil { + continue + } + keys = append(keys, val) } + return keys } func (p *provisioner) readClientDescriptor(rootDir string, stackName string) (*api.ClientDescriptor, error) { diff --git a/pkg/provisioner/scoped_provision_test.go b/pkg/provisioner/scoped_provision_test.go new file mode 100644 index 00000000..d394b47e --- /dev/null +++ b/pkg/provisioner/scoped_provision_test.go @@ -0,0 +1,138 @@ +// SPDX-License-Identifier: MIT +// Copyright (c) Simple Container + +package provisioner + +import ( + "context" + "fmt" + "os" + "path/filepath" + "strings" + "testing" + + . "github.com/onsi/gomega" + "github.com/pkg/errors" + "golang.org/x/crypto/ssh" + + "github.com/simple-container-com/api/pkg/api/secrets/ciphers" + "github.com/simple-container-com/api/pkg/api/secrets/scoped" +) + +// captureLogger records Warn lines so a test can assert the deploy-time +// legacy-shadows-scoped warning fires. It satisfies api/logger.Logger structurally. +type captureLogger struct{ warns []string } + +func (l *captureLogger) Error(_ context.Context, f string, a ...any) {} +func (l *captureLogger) Warn(_ context.Context, f string, a ...any) { + l.warns = append(l.warns, fmt.Sprintf(f, a...)) +} +func (l *captureLogger) Info(_ context.Context, f string, a ...any) {} +func (l *captureLogger) Debug(_ context.Context, f string, a ...any) {} +func (l *captureLogger) SetLogLevel(ctx context.Context, _ int) context.Context { return ctx } +func (l *captureLogger) Silent(ctx context.Context) context.Context { return ctx } + +// Test_readSecretsDescriptor_Scoped exercises the deploy-time read path for scoped +// secrets: a stack with ONLY secrets..yaml (no legacy secrets.yaml), resolved +// via a CI scope key supplied through the environment (no SIMPLE_CONTAINER_CONFIG), +// and the fail-closed integrity propagation. +func Test_readSecretsDescriptor_Scoped(t *testing.T) { + RegisterTestingT(t) + + // An "admin/CI" ed25519 recipient of the pr scope. + priv, pub, err := ciphers.GenerateEd25519KeyPair() + Expect(err).NotTo(HaveOccurred()) + sshPub, err := ssh.NewPublicKey(pub) + Expect(err).NotTo(HaveOccurred()) + authorized := string(ssh.MarshalAuthorizedKey(sshPub)) + pem, err := ciphers.MarshalEd25519PrivateKey(priv) + Expect(err).NotTo(HaveOccurred()) + + stacksDir := t.TempDir() + stackDir := filepath.Join(stacksDir, "teststack") + Expect(os.MkdirAll(stackDir, 0o755)).To(Succeed()) + + f, err := scoped.NewScopeFile("teststack", "pr", []string{authorized}) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Set("defectdojo-api-key", "dd-secret")).To(Succeed()) + scopePath := filepath.Join(stackDir, scoped.ScopeFileName("pr")) + Expect(f.Save(scopePath)).To(Succeed()) + // NB: no secrets.yaml written — this is a scoped-only stack. + + // Supply the scope key the way a pull_request job would (env, no full config). + t.Setenv("SC_SCOPE_KEY", pem) + p := &provisioner{} // no cryptor: key must come from the env + + // P1a + P1b: scoped-only stack resolves via the env scope key. + desc, err := p.readSecretsDescriptor(context.Background(), stacksDir, "teststack") + Expect(err).NotTo(HaveOccurred()) + Expect(desc).NotTo(BeNil()) + Expect(desc.Values["defectdojo-api-key"]).To(Equal("dd-secret")) + + // P2: a corrupt scope file is a hard error (ErrScopedIntegrity), never a silent + // skip — even though nothing references the value. + Expect(os.WriteFile(scopePath, []byte("schemaVersion: 999\nscope: pr\n"), 0o644)).To(Succeed()) + _, err = p.readSecretsDescriptor(context.Background(), stacksDir, "teststack") + Expect(err).To(HaveOccurred()) + Expect(errors.Is(err, scoped.ErrScopedIntegrity)).To(BeTrue()) +} + +// Test_readSecretsDescriptor_NoSecretsStillNotFound confirms a stack with neither a +// legacy secrets.yaml nor any openable scoped value still reports not-found (the +// pre-scoped behavior that IgnoreSecretsMissing relies on). +func Test_readSecretsDescriptor_NoSecretsStillNotFound(t *testing.T) { + RegisterTestingT(t) + stacksDir := t.TempDir() + Expect(os.MkdirAll(filepath.Join(stacksDir, "empty"), 0o755)).To(Succeed()) + + p := &provisioner{} + _, err := p.readSecretsDescriptor(context.Background(), stacksDir, "empty") + Expect(err).To(HaveOccurred()) + Expect(errors.Is(err, os.ErrNotExist)).To(BeTrue()) + Expect(errors.Is(err, scoped.ErrScopedIntegrity)).To(BeFalse()) +} + +// Test_readSecretsDescriptor_ScopedShadowedByLegacy verifies the phase-2 deploy-time +// behavior: when a key exists in BOTH the legacy secrets.yaml and an openable scope, +// the legacy value wins (fail-safe direction) AND a warning is emitted so the operator +// is not silently surprised. +func Test_readSecretsDescriptor_ScopedShadowedByLegacy(t *testing.T) { + RegisterTestingT(t) + + priv, pub, err := ciphers.GenerateEd25519KeyPair() + Expect(err).NotTo(HaveOccurred()) + sshPub, err := ssh.NewPublicKey(pub) + Expect(err).NotTo(HaveOccurred()) + authorized := string(ssh.MarshalAuthorizedKey(sshPub)) + pem, err := ciphers.MarshalEd25519PrivateKey(priv) + Expect(err).NotTo(HaveOccurred()) + + stacksDir := t.TempDir() + stackDir := filepath.Join(stacksDir, "teststack") + Expect(os.MkdirAll(stackDir, 0o755)).To(Succeed()) + + // Legacy secrets.yaml defines shared-key. + Expect(os.WriteFile(filepath.Join(stackDir, "secrets.yaml"), + []byte("values:\n shared-key: legacy-value\n"), 0o644)).To(Succeed()) + + // A scope file ALSO defines shared-key (with a different value). + f, err := scoped.NewScopeFile("teststack", "pr", []string{authorized}) + Expect(err).NotTo(HaveOccurred()) + Expect(f.Set("shared-key", "scoped-value")).To(Succeed()) + Expect(f.Save(filepath.Join(stackDir, scoped.ScopeFileName("pr")))).To(Succeed()) + + t.Setenv("SC_SCOPE_KEY", pem) + log := &captureLogger{} + p := &provisioner{log: log} + + desc, err := p.readSecretsDescriptor(context.Background(), stacksDir, "teststack") + Expect(err).NotTo(HaveOccurred()) + // Legacy wins on conflict. + Expect(desc.Values["shared-key"]).To(Equal("legacy-value")) + // And a shadow warning was emitted, naming the key (never the value). + joined := strings.Join(log.warns, "\n") + Expect(joined).To(ContainSubstring("shared-key")) + Expect(joined).To(ContainSubstring("shadowed")) + Expect(joined).NotTo(ContainSubstring("legacy-value")) + Expect(joined).NotTo(ContainSubstring("scoped-value")) +}