Skip to content

chore(seismic)!: replace nginx and certbot with an unprivileged Caddy - #81

Merged
samlaf merged 1 commit into
seismicfrom
sl/sei-629-chore-replace-nginx-and-certbot-with-an-unprivileged-caddy
Oct 9, 2026
Merged

samlaf merged 1 commit into
seismicfrom
sl/sei-629-chore-replace-nginx-and-certbot-with-an-unprivileged-caddy

Conversation

@samlaf

@samlaf samlaf commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

The public proxy ran as root: the nginx master, certbot (a root Python process that talks to the network and rewrites nginx's config) and the nginx-ssl-setup script. In-guest root can read root_key out of the custodian's memory, so the most exposed process on the node should hold no privilege it does not need. Caddy takes and renews its certificate in-process, as User=caddy with only CAP_NET_BIND_SERVICE, from a static Caddyfile in the measured image.

  • certificates: Let's Encrypt, falling back to ZeroSSL, over TLS-ALPN-01 only, so nothing listens on :80; no admin endpoint, HTTP/3 off
  • routes match their exact path (trailing slash allowed) and forward to /, since every backend serves one endpoint whatever the path; any other path is a 404, and WebSocket upgrades reach /ws only
  • caddy.service: ordered after tdx-init and persistent-luks-setup and required by nothing, with NoNewPrivileges, ProtectSystem=strict, PrivateTmp, PrivateDevices and RestrictAddressFamilies; all Caddy state lives in /persistent/caddy, which replaces /persistent/nginx
  • caddy v2.11.7 built from source in mkosi.build, with a pinned Go (go1.26.9) fetched through GOTOOLCHAIN, since Debian's is too old; static and reproducible
  • enclave pin to 74a6185: Caddy pastes {$DOMAIN_NAME} into its config as raw text, so tdx-init's domain validation is what keeps an operator's domain from injecting Caddyfile syntax
  • python3, nginx and certbot leave the image
  • CI validates the Caddyfile with the image's own caddy and expects four source pins; image.json and the release notes record the caddy commit
  • README: the endpoint table describes Caddy, and /summit is read-only JSON-RPC (getDepositSignature is on summit's localhost admin listener)

BREAKING CHANGE: existing nodes take a fresh certificate on their first boot on this image, which counts against Let's Encrypt's 5-a-week limit for an exact name set. Nothing answers on :80.

Refs: SEI-629

The public proxy ran as root: the nginx master, certbot (a root Python
process that talks to the network and rewrites nginx's config) and the
nginx-ssl-setup script. In-guest root can read root_key out of the
custodian's memory, so the most exposed process on the node should hold
no privilege it does not need. Caddy takes and renews its certificate
in-process, as User=caddy with only CAP_NET_BIND_SERVICE, from a static
Caddyfile in the measured image.

- certificates: Let's Encrypt, falling back to ZeroSSL, over TLS-ALPN-01
  only, so nothing listens on :80; no admin endpoint, HTTP/3 off
- routes match their exact path (trailing slash allowed) and forward to
  /, since every backend serves one endpoint whatever the path; any
  other path is a 404, and WebSocket upgrades reach /ws only
- caddy.service: ordered after tdx-init and persistent-luks-setup and
  required by nothing, with NoNewPrivileges, ProtectSystem=strict,
  PrivateTmp, PrivateDevices and RestrictAddressFamilies; all Caddy
  state lives in /persistent/caddy, which replaces /persistent/nginx
- caddy v2.11.7 built from source in mkosi.build, with a pinned Go
  (go1.26.9) fetched through GOTOOLCHAIN, since Debian's is too old;
  static and reproducible
- enclave pin to 74a6185: Caddy pastes {$DOMAIN_NAME} into its config
  as raw text, so tdx-init's domain validation is what keeps an
  operator's domain from injecting Caddyfile syntax
- python3, nginx and certbot leave the image
- CI validates the Caddyfile with the image's own caddy and expects four
  source pins; image.json and the release notes record the caddy commit
- README: the endpoint table describes Caddy, and /summit is read-only
  JSON-RPC (getDepositSignature is on summit's localhost admin listener)

BREAKING CHANGE: existing nodes take a fresh certificate on their first
boot on this image, which counts against Let's Encrypt's 5-a-week limit
for an exact name set. Nothing answers on :80.

Refs: SEI-629
@samlaf
samlaf requested a review from a team as a code owner October 8, 2026 22:03
@linear-code

linear-code Bot commented Oct 8, 2026

Copy link
Copy Markdown

SEI-629

@samlaf
samlaf merged commit cf5d261 into seismic Oct 9, 2026
5 checks passed
@samlaf
samlaf deleted the sl/sei-629-chore-replace-nginx-and-certbot-with-an-unprivileged-caddy branch October 9, 2026 21:25
samlaf added a commit to SeismicSystems/seismic that referenced this pull request Oct 9, 2026
Describes SeismicSystems/seismic-images#81

architecture.md: the process table, lifecycle diagram, ports table (:443
only), keys table and the /persistent section describe Caddy, and a
design-rationale entry explains an unprivileged Caddy over nginx and
certbot. network-founding.md, trust-model.md and the node-processes and
key-families diagrams follow. The seismic-tee CLI's comments and its
configure summary name the node's HTTPS proxy generically.

Refs: SEI-629
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant