Skip to content

feat(node): surface healthy provider counts on /debug/node_config - #3850

Draft
haiyuechen-nearone wants to merge 1 commit into
3764-foreign-chain-health-metricfrom
3764-debug-node-config-health
Draft

feat(node): surface healthy provider counts on /debug/node_config#3850
haiyuechen-nearone wants to merge 1 commit into
3764-foreign-chain-health-metricfrom
3764-debug-node-config-health

Conversation

@haiyuechen-nearone

@haiyuechen-nearone haiyuechen-nearone commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Closes #3764. Stacked on #3849 (top of the stack), merge bottom-up:

  1. feat(node): probe foreign-chain RPC providers on startup #3848 — startup probe + per-provider logs
  2. feat(node): expose foreign-chain RPC provider health metric #3849 — per-chain configured / healthy Prometheus gauges
  3. feat(node): surface healthy provider counts on /debug/node_config #3850 (this PR) — healthy counts on /debug/node_config

The startup probe publishes its per-chain healthy-provider counts over a watch channel; the web server overlays them onto /debug/node_config as foreign_chains_provider_health (empty until the probe completes, always present, mirroring foreign_chains_provider_counts).

@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch from 30b2b8f to 9f1b2bb Compare July 17, 2026 08:03
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch 2 times, most recently from 420b59d to 56ea9d2 Compare July 17, 2026 08:23
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch 2 times, most recently from 00a321c to 8ff2845 Compare July 17, 2026 08:35
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch from 56ea9d2 to 0e54756 Compare July 17, 2026 08:35
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch from 0e54756 to e1eb9d4 Compare July 20, 2026 09:57
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch 2 times, most recently from 585df70 to fefd8f9 Compare July 20, 2026 11:05
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch from e1eb9d4 to d1cf5a8 Compare July 20, 2026 11:05
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch from fefd8f9 to f64152b Compare July 20, 2026 11:19
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch from d1cf5a8 to 94b2bab Compare July 20, 2026 11:19
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch from f64152b to fa93861 Compare July 20, 2026 16:28
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch from 94b2bab to 318ecc3 Compare July 20, 2026 16:28
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch from fa93861 to 68f3d87 Compare July 20, 2026 16:47
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch from 318ecc3 to 1b3e1e6 Compare July 20, 2026 16:47
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch from 68f3d87 to dcadf7c Compare July 20, 2026 16:55
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch from 1b3e1e6 to 214d78d Compare July 20, 2026 16:55
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch from dcadf7c to f8a075a Compare July 20, 2026 17:09
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch 2 times, most recently from a30098d to e3328e2 Compare July 21, 2026 09:27
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch from f8a075a to b6a3b10 Compare July 21, 2026 09:31
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch from e3328e2 to 48f7886 Compare July 22, 2026 12:52
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch from b6a3b10 to 09b60e6 Compare July 22, 2026 13:00
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch from 48f7886 to 287686c Compare July 22, 2026 13:11
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch from 09b60e6 to 71cd909 Compare July 22, 2026 13:16
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch from 287686c to 938f549 Compare July 22, 2026 13:18
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch from 71cd909 to 71b6738 Compare July 22, 2026 13:19
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch from 938f549 to 82fea65 Compare July 22, 2026 13:25
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch from 71b6738 to ffb1f09 Compare July 22, 2026 13:26
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch 4 times, most recently from a327d7a to 996daa8 Compare July 22, 2026 15:09
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch from ffb1f09 to 7bc83e5 Compare July 22, 2026 15:12
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch from 996daa8 to de1619f Compare July 22, 2026 16:55
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch from 7bc83e5 to 828e978 Compare July 22, 2026 16:55
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch from de1619f to 996daa8 Compare July 22, 2026 17:08
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch 2 times, most recently from 51ed578 to a2e6f10 Compare July 22, 2026 17:28
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-foreign-chain-health-metric branch from 996daa8 to e7a8755 Compare July 22, 2026 17:47
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch 2 times, most recently from 4388a56 to 098e64d Compare July 22, 2026 18:04
@haiyuechen-nearone haiyuechen-nearone self-assigned this Jul 22, 2026
@haiyuechen-nearone
haiyuechen-nearone force-pushed the 3764-debug-node-config-health branch from 098e64d to ec5a9c1 Compare July 22, 2026 18:07
@haiyuechen-nearone
haiyuechen-nearone marked this pull request as ready for review July 22, 2026 18:10
@claude

claude Bot commented Jul 22, 2026

Copy link
Copy Markdown

Pull request overview

Wires a tokio::sync::watch channel from run_startup_health_check to the /debug/node_config handler so per-chain healthy-provider counts published by the startup probe surface alongside the existing configured counts. New ProviderHealthSnapshot type (BTreeMap<String, i64> keyed by chain.label()) is overlaid at request time; the field is always present, defaulting to {} before/without a probe.

Changes:

  • New ProviderHealthSnapshot type in foreign_chain_health.rs; run_startup_health_check / set_healthy_metric now accept a watch::Sender and publish alongside the Prometheus gauge.
  • WebServerState gains a watch::Receiver<ProviderHealthSnapshot>; debug_node_config clones the base response and overlays the current snapshot.
  • NodeConfigResponse gains a foreign_chains_provider_health field.
  • run.rs and tests.rs create/plumb the watch channel; test callers updated.
  • E2E cluster helper fetch_node_config added; startup foreign-chain-health E2E asserts the field content on probed and skipped nodes.

Reviewed changes

Per-file summary
File Description
crates/node/src/foreign_chain_health.rs Adds ProviderHealthSnapshot; threads a watch::Sender through the probe; updates unit tests to construct a channel and assert on the receiver.
crates/node/src/web.rs Adds foreign_chains_provider_health field, receiver on WebServerState, overlay in debug_node_config; new/updated unit tests.
crates/node/src/run.rs Creates the watch channel and passes sender to the probe / receiver to the web server.
crates/node/src/tests.rs Wires a dummy watch receiver into the one-node test harness.
crates/e2e-tests/src/cluster.rs New fetch_node_config helper.
crates/e2e-tests/tests/startup_foreign_chain_health.rs Asserts the healthy overlay on /debug/node_config for probed and skipped nodes.

Findings

Non-blocking (nits, follow-ups, suggestions):

  • crates/node/src/web.rs:103,124NodeConfigResponse stores foreign_chains_provider_health as a field, but the value set by From<ConfigFile> is always the empty default and is always overwritten at request time in debug_node_config (web.rs:189-192). This is a small design smell: the field exists only to satisfy Serialize. Consider either (a) constructing the response from the base struct + snapshot at handler time (no default field on NodeConfigResponse), or (b) documenting on the field that its stored value is unused and set per-request. Not a correctness issue, but future edits could easily be confused into believing the stored value flows to callers.
  • crates/node/src/foreign_chain_health.rs:114let _ = health_publisher.send(counts); silently drops the SendError, which for watch::Sender only happens when all receivers have been dropped. That's benign here (web server holds the receiver for the process lifetime), but if that invariant ever changes this failure mode becomes invisible. A debug! on the discarded error would be cheap insurance without changing behavior.
  • crates/node/src/foreign_chain_health.rs:81 — Golden-supplied probes on NetworkKind::Local don't publish a snapshot when nothing was probed (probe returned early via Skip variants). This is consistent with the existing "skipped chain → empty map on /debug/node_config" behavior asserted by the E2E test for nodes 1 and 2, so no action needed — but worth noting explicitly in the doc comment of run_startup_health_check that snapshots are only published after a probe actually runs (not on Skip*).
  • crates/e2e-tests/src/cluster.rs:777 — No timeout on reqwest::get. The default reqwest client has no timeout; a hung web-server response would hang the test until the harness-level timeout fires. Consider using a reqwest::Client with an explicit timeout for consistency with other health/metric helpers in the file.

✅ Approved

@haiyuechen-nearone
haiyuechen-nearone marked this pull request as draft July 23, 2026 08:19
The startup probe publishes its per-chain healthy-provider counts over a watch
channel; the web server overlays them onto `/debug/node_config` as
`foreign_chains_provider_health` (empty until the probe completes, always
present, mirroring `foreign_chains_provider_counts`).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant