Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 19 additions & 12 deletions .github/workflows/seismic.yml
Original file line number Diff line number Diff line change
Expand Up @@ -93,14 +93,12 @@ jobs:
bash -n \
modules/seismic/mkosi.build \
modules/seismic/mkosi.postinst \
modules/seismic/mkosi.extra/usr/bin/nginx-ssl-setup \
modules/seismic/mkosi.extra/usr/bin/persistent-luks-setup \
scripts/seismic/founding_inputs.sh \
scripts/seismic/image_json.sh
shellcheck -x \
modules/seismic/mkosi.build \
modules/seismic/mkosi.postinst \
modules/seismic/mkosi.extra/usr/bin/nginx-ssl-setup \
modules/seismic/mkosi.extra/usr/bin/persistent-luks-setup \
scripts/seismic/founding_inputs.sh \
scripts/seismic/image_json.sh
Expand All @@ -113,8 +111,8 @@ jobs:

source = pathlib.Path("modules/seismic/sources.yaml").read_text()
pins = re.findall(r"^\s+git_reference:\s*([0-9a-f]+)\s*$", source, re.MULTILINE)
if len(pins) != 3 or any(len(pin) != 40 for pin in pins):
raise SystemExit("sources.yaml must contain three full 40-character git pins")
if len(pins) != 4 or any(len(pin) != 40 for pin in pins):
raise SystemExit("sources.yaml must contain four full 40-character git pins")

build:
name: build Seismic image
Expand Down Expand Up @@ -169,11 +167,10 @@ jobs:
exit 1
}
done <<'EOF'
etc/caddy/Caddyfile
etc/systemd/system/attestation.service
etc/systemd/system/certbot-renew.service
etc/systemd/system/certbot-renew.timer
etc/systemd/system/caddy.service
etc/systemd/system/custodian.service
etc/systemd/system/nginx-ssl-setup.service
etc/systemd/system/persistent-luks-setup.service
etc/systemd/system/reth.service
etc/systemd/system/summit-keygen.service
Expand All @@ -182,14 +179,13 @@ jobs:
etc/systemd/system/summit.target
etc/systemd/system/tdx-init.service
etc/systemd/system/minimal.target.wants/attestation.service
etc/systemd/system/minimal.target.wants/certbot-renew.timer
etc/systemd/system/minimal.target.wants/caddy.service
etc/systemd/system/minimal.target.wants/custodian.service
etc/systemd/system/minimal.target.wants/nginx-ssl-setup.service
etc/systemd/system/minimal.target.wants/persistent-luks-setup.service
etc/systemd/system/minimal.target.wants/reth.service
etc/systemd/system/minimal.target.wants/summit.target
etc/systemd/system/minimal.target.wants/tdx-init.service
usr/bin/nginx-ssl-setup
usr/bin/caddy
usr/bin/persistent-luks-setup
usr/bin/seismic-attestation-service
usr/bin/seismic-custodian-service
Expand Down Expand Up @@ -252,6 +248,16 @@ jobs:
&& find . -type f -print0 | sort -z | xargs -0 sha256sum) \
> build/initrd-files.sha256

# A node parses the Caddyfile only after the config POST, so a mistake
# in it would first show as a node without HTTPS. The image's own caddy
# checks it here, with placeholder values for what domain.env provides.
- name: Validate the Caddyfile
run: |
root="$RUNNER_TEMP/initrd-root"
XDG_DATA_HOME="$RUNNER_TEMP/caddy" XDG_CONFIG_HOME="$RUNNER_TEMP/caddy" \
DOMAIN_NAME=node.example.com DOMAIN_EMAIL=ops@example.com \
"$root/usr/bin/caddy" validate --config "$root/etc/caddy/Caddyfile" --adapter caddyfile

- name: Upload checksums for reproducibility comparison
if: >-
github.event_name == 'pull_request' ||
Expand Down Expand Up @@ -379,7 +385,7 @@ jobs:

# The release notes: the same facts as image.json, for a human reading
# the release — the blob URL and the measurements URL together, what
# each founding input is, and the three source commits (this
# each founding input is, and the four source commits (this
# repository's own commit is the tag; the pins live in sources.yaml at
# that commit, so no asset repeats them beyond image.json).
- name: Write the release notes
Expand All @@ -389,7 +395,7 @@ jobs:
run: |
fact() { jq -r "$1" build/image.json; }
vhd_blob_url=$(fact '.targets["azure-tdx"].vhd_blob_url')
reth=$(fact .sources.seismic_reth); summit=$(fact .sources.summit); enclave=$(fact .sources.enclave)
reth=$(fact .sources.seismic_reth); summit=$(fact .sources.summit); enclave=$(fact .sources.enclave); caddy=$(fact .sources.caddy)
cat > "$RUNNER_TEMP/notes.md" <<EOF
Seismic node image \`$IMAGE\`, built by this repository at $GITHUB_SHA.

Expand Down Expand Up @@ -419,6 +425,7 @@ jobs:
| [\`seismic-reth\`](https://github.com/SeismicSystems/seismic-reth/commit/$reth) | \`${reth:0:7}\` |
| [\`summit\`](https://github.com/SeismicSystems/summit/commit/$summit) | \`${summit:0:7}\` |
| [\`enclave\`](https://github.com/SeismicSystems/enclave/commit/$enclave) | \`${enclave:0:7}\` — attestation service, custodian service, tdx-init |
| [\`caddy\`](https://github.com/caddyserver/caddy/commit/$caddy) | \`${caddy:0:7}\` — the public HTTPS proxy |

### Build provenance

Expand Down
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ A network founded on an image has genesis artifacts that only that image's own c

Both genesis files are inputs to a founding, never a founded network: every value in the starter, and the chain id and allocations in `reth-genesis.json`, is a per-network choice, made before anything is derived from them.

`make release-assets` (the measurements, then `make founding-inputs`) gathers the same files from a local build into `build/`, with the same `SHA256SUMS`, so a rebuild is compared to a release with `sha256sum -c --ignore-missing SHA256SUMS`. The release notes name the three source commits the image was built from; the commit of this repository is the tag itself, and [`sources.yaml`](modules/seismic/sources.yaml) at that commit is the record a rebuild starts from.
`make release-assets` (the measurements, then `make founding-inputs`) gathers the same files from a local build into `build/`, with the same `SHA256SUMS`, so a rebuild is compared to a release with `sha256sum -c --ignore-missing SHA256SUMS`. The release notes name the four source commits the image was built from; the commit of this repository is the tag itself, and [`sources.yaml`](modules/seismic/sources.yaml) at that commit is the record a rebuild starts from.

### `image.json`

Expand All @@ -85,7 +85,7 @@ Where the image's bytes are and what they are, in one machine-readable file, so
{
"image": "seismic_2026-09-22.2ee71c",
"commit": "<the commit of this repository that built it>",
"sources": {"seismic_reth": "<sha>", "summit": "<sha>", "enclave": "<sha>"},
"sources": {"seismic_reth": "<sha>", "summit": "<sha>", "enclave": "<sha>", "caddy": "<sha>"},
"targets": {
"azure-tdx": {
"vhd_blob_url": "https://seismicimages.blob.core.windows.net/dev/seismic_2026-09-22.2ee71c.vhd",
Expand All @@ -97,36 +97,36 @@ Where the image's bytes are and what they are, in one machine-readable file, so
}
```

`targets` is keyed by attestation type, one entry per cloud the image is published for — Azure only today; a GCP image adds an entry, not a schema. Per target: the artifact the nodes boot from, the storage account's ARM ID (Azure's managed-disk import refuses to read a blob from another subscription or resource group without it, and the URL names the account but neither of those; a SAS URL would avoid the requirement but expires, so it has no place in a release), the measurements asset for that target, and the sha256 of the artifact where it is reproducible — the `.efi`, since the VHD's wrapping is not. Once per file: the image name (the tag, and the stem of `measurement_id`), this repository's commit, and the three source pins from `sources.yaml`. The publish job writes it (`make image-json`, [`scripts/seismic/image_json.sh`](scripts/seismic/image_json.sh)) after the VHD is pushed, asking Azure for the account's ID rather than carrying it in this repo, and renders the release notes from it. It refuses a measurements file and a `SHA256SUMS` that name different UKIs, so the two halves of a release cannot come from different builds. It is the one asset `make release-assets` does not produce, since the ID is a fact about where the bytes were put, not about the build.
`targets` is keyed by attestation type, one entry per cloud the image is published for — Azure only today; a GCP image adds an entry, not a schema. Per target: the artifact the nodes boot from, the storage account's ARM ID (Azure's managed-disk import refuses to read a blob from another subscription or resource group without it, and the URL names the account but neither of those; a SAS URL would avoid the requirement but expires, so it has no place in a release), the measurements asset for that target, and the sha256 of the artifact where it is reproducible — the `.efi`, since the VHD's wrapping is not. Once per file: the image name (the tag, and the stem of `measurement_id`), this repository's commit, and the four source pins from `sources.yaml`. The publish job writes it (`make image-json`, [`scripts/seismic/image_json.sh`](scripts/seismic/image_json.sh)) after the VHD is pushed, asking Azure for the account's ID rather than carrying it in this repo, and renders the release notes from it. It refuses a measurements file and a `SHA256SUMS` that name different UKIs, so the two halves of a release cannot come from different builds. It is the one asset `make release-assets` does not produce, since the ID is a fact about where the bytes were put, not about the build.

## What's in the image

The Seismic node stack: [`seismic-reth`](https://github.com/SeismicSystems/seismic-reth),
[`summit`](https://github.com/SeismicSystems/summit), and from
[`enclave`](https://github.com/SeismicSystems/enclave) `tdx-init`, the custodian and the
attestation service, built from the commits pinned in
[`sources.yaml`](modules/seismic/sources.yaml), plus nginx and certbot from Debian.
attestation service, and [Caddy](https://github.com/caddyserver/caddy) as the public HTTPS
proxy, all built from the commits pinned in [`sources.yaml`](modules/seismic/sources.yaml).
[`modules/seismic/readme.md`](modules/seismic/readme.md) covers how they are assembled:
units, users and groups, directories.

## Exposed HTTPS endpoints

nginx terminates TLS (Let's Encrypt) and reverse-proxies the following paths to in-TEE services. See [`modules/seismic/mkosi.extra/etc/nginx/node-template.conf`](modules/seismic/mkosi.extra/etc/nginx/node-template.conf).
Caddy terminates TLS on `:443` with a certificate it takes and renews itself (Let's Encrypt, or ZeroSSL when that fails), and reverse-proxies the following paths to in-TEE services. Each backend serves one endpoint whatever the request path, so a route matches its path exactly, with or without a trailing slash, and forwards to `/`; anything else (`/rpc/foo`, `/rpcfoo`) is a 404. Only `/ws` accepts WebSocket upgrades. See [`modules/seismic/mkosi.extra/etc/caddy/Caddyfile`](modules/seismic/mkosi.extra/etc/caddy/Caddyfile).

| Route | Backend | Purpose | Public? |
| ----------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------- | -------------------------------------- |
| `/rpc` | `reth` `:8545` | Ethereum JSON-RPC (shielded tx support via TxSeismic) | ✅ intended public |
| `/ws` | `reth` `:8546` | Ethereum WebSocket RPC | ✅ intended public |
| `/summit` | `summit` `:3030` | Consensus REST API, incl. `/summit/get_deposit_signature/...` used by the staking UI | ⚠️ **overly broad** — see warning below |
| `/summit` | `summit` `:3030` | Consensus JSON-RPC: read-only queries (health, checkpoints, headers, validator balances, deposits, state proofs) | ⚠️ no rate limiting — see below |
| `/attestation` | `attestation-service` `:7878` | Attestation API (tx-io attestation evidence, health/LUKS status, admission-chain status, peer root-key bootstrap) | ✅ same surface as `:7878`, which is open to anyone |
| `/metrics/reth` | `reth` `:9001` | Prometheus metrics | ⚠️ unauthenticated |
| `/metrics/summit` | `summit` `:9002` | Prometheus metrics | ⚠️ unauthenticated |

### ⚠️ Known sharp edges

Everything above listens on the single public `:443`. The intended long-term fix is to split nginx into two tiers — a public server block with only the endpoints that should reach the open internet (`/rpc`, `/ws`, a narrowed `/summit/get_deposit_signature`, `/attestation`), and an internal server block on a separate port with the rest (`/metrics/*`, the full `/summit/*` query surface). The deploy tooling would then configure cloud firewall rules (Azure NSG / GCP firewall) to allow the public port from `0.0.0.0/0` and restrict the internal port to the VPC CIDR. Until that split lands, the concrete issues are:
Everything above listens on the single public `:443`. The intended long-term fix is to split the proxy into two tiers — a public site with only the endpoints that should reach the open internet (`/rpc`, `/ws`, `/attestation`, and whatever of `/summit` the staking UI needs), and an internal site on a separate port with the rest (`/metrics/*`, the rest of `/summit`). The deploy tooling would then configure cloud firewall rules (Azure NSG / GCP firewall) to allow the public port from `0.0.0.0/0` and restrict the internal port to the VPC CIDR. Until that split lands, the concrete issues are:

- **`/summit/*` is a blanket proxy.** Summit exposes a JSON-RPC surface (via `jsonrpsee`) with ~20 methods: mostly read-only state queries (`getCheckpoint`, `getValidatorBalance`, `getDeposit`, etc. — analogous to `eth_*` reads and safe to expose), plus `getDepositSignature` which causes a BLS signature in the enclave, plus `sendGenesis` on the genesis-setup API which must *never* be public at runtime. Narrowing requires JSON-RPC-method-level filtering (all calls are `POST /`, so you can't gate by URL path alone). `getDepositSignature` additionally has no rate limiting — trivially DoS-able — and should gain `limit_req` regardless of network controls.
- **`/summit` is a blanket proxy with no rate limiting.** Summit's public listener serves only read-only queries (`health`, `getCheckpoint`, `getValidatorBalance`, `getDeposit`, `getStateProof`, … — analogous to `eth_*` reads). Its signing method, `getDepositSignature`, is on a localhost-only admin listener, and `sendGenesis` on a separate genesis-setup server; the proxy reaches neither. Narrowing `/summit` further needs JSON-RPC-method-level filtering, since every call is `POST /`.
- **`/metrics/*` is unauthenticated.** Operationally safe on a locked-down network (the cloud firewall rule above is the right fix), but anyone who can reach the port can scrape sync status, peer info, and resource usage. Do *not* expose the internal port to the open internet without adding basic auth or an IP allowlist on top.

## Our diff vs upstream
Expand Down
22 changes: 22 additions & 0 deletions modules/seismic/mkosi.build
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ SOURCES_YAML="$SRCDIR/modules/seismic/sources.yaml"
ENCLAVE_COMMIT=$(mkosi-chroot yq -r '.enclave.git_reference' < "$SOURCES_YAML")
RETH_COMMIT=$(mkosi-chroot yq -r '.seismic_reth.git_reference' < "$SOURCES_YAML")
SUMMIT_COMMIT=$(mkosi-chroot yq -r '.summit.git_reference' < "$SOURCES_YAML")
CADDY_COMMIT=$(mkosi-chroot yq -r '.caddy.git_reference' < "$SOURCES_YAML")

# We manage our own rust toolchain instead of using the one provided by the Debian Snapshot.
# This decouples us from Debian's release cycle and installs deterministically the version
Expand Down Expand Up @@ -144,3 +145,24 @@ make_git_package \
"https://github.com/SeismicSystems/summit" \
"$SUMMIT_BUILD_CMD" \
"target/release/summit:/usr/bin/summit"

# Build caddy. Debian's golang-go is too old for it and only bootstraps:
# GOTOOLCHAIN makes it fetch the pinned Go release, which the checksum
# database verifies, the way rustup pins Rust above. It must be at least the
# `go` line of caddy's go.mod. CGO_ENABLED=0 links caddy statically, so it
# does not depend on the image's libc.
CADDY_BUILD_CMD="
git checkout $CADDY_COMMIT
export GOTOOLCHAIN='go1.26.9'
export GOPATH='/go'
export GOCACHE='/go/cache'
export CGO_ENABLED='0'

go build -trimpath -ldflags '-s -w -buildid=' -o build/caddy ./cmd/caddy
"
make_git_package \
"caddy" \
"$CADDY_COMMIT" \
"https://github.com/caddyserver/caddy" \
"$CADDY_BUILD_CMD" \
"build/caddy:/usr/bin/caddy"
18 changes: 3 additions & 15 deletions modules/seismic/mkosi.conf
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,7 @@ ImageId=seismic
# 1. Signed-PCR-policy LUKS enrollment (the BLOCKER header in
# persistent-luks-setup) must land first — else every release
# rebases roothash → PCR 11 → bricks unlock on first update.
# 2. Audit and relocate /usr/ writes from runtime scripts. Main
# offender is certbot's /etc/letsencrypt/; converges with
# the nginx+certbot → RA-TLS TODO below.
# 3. Uniform `User=` across reth/summit/nginx (enclave already
# done). Verity's value is in narrowing in-guest privilege
# pivots; with services-as-root, an RCE in any of them is
# RCE-into-root-readable enclave memory, and verity buys
# little. With per-service UIDs, the standard pivots
# (overwrite /usr/bin/X, swap a .so) are what verity closes.
# 2. Audit and relocate /usr/ writes from runtime scripts.

[Build]
Environment=KERNEL_CONFIG_SNIPPETS_SEISMIC=modules/seismic/kernel/config.d
Expand All @@ -56,12 +48,6 @@ Packages=cryptsetup
# TPM2 libraries the attestation service links against
libtss2-esys-3.0.2-0t64
libtss2-tctildr0t64
# TODO(samlaf): replace nginx+certbot with RA-TLS (e.g. flashbots'
# cvm-reverse-proxy) which embeds the TDX attestation quote in the
# TLS cert, removing the need for Let's Encrypt and CAs entirely.
nginx
certbot
python3-certbot-nginx
# TODO: document what lz4 is used for
lz4

Expand All @@ -80,3 +66,5 @@ BuildPackages=build-essential
pkg-config
# yq is used by mkosi.build to read source pins from sources.yaml
yq
# bootstraps the pinned Go toolchain mkosi.build builds caddy with
golang-go
Loading