[2/n][guardian-integration] in-process e2e harness by 0xsiddharthks · Pull Request #466 · MystenLabs/hashi

0xsiddharthks · 2026-04-17T16:41:29Z

Summary

lift Enclave (+ EnclaveConfig / EnclaveState / Scratchpad) from main.rs into a new enclave.rs lib module
— @mskd12 this is just a simple move. no changes made to the code.
- the only change is that enclave_btc_keypair and hashi_btc_master_pubkey went from private to pub(crate).
- This is required because init.rs::tests reads these fields directly via enclave.config.<field>.get()
- moving these to enclave.rs allows the e2e-tests to construct and derive an enclave in-process
New test-utils feature on hashi-guardian exposing constructors for partially/fully-initialized enclaves.
add new guardian client for hashi

Tests harness changes

e2e-tests::GuardianHarness runs an in-process guardian over real gRPC.
TestNetworksBuilder::with_guardian() orchestrates start → DKG → finalize.
test_bitcoin_withdrawal_with_guardian_e2e_flow does full e2e flow with guardian

mskd12 · 2026-04-28T16:01:16Z

 aws-sdk-s3 = "1.12.0"
 aws-credential-types = "1"

+aws-smithy-mocks = { version = "0.1", optional = true }


nit: any way to remove the mock library from the main dependency list? IIUC it is just used by the test harness, right?

Yeah, afaik we can remove it, but then we have to pass --features test-utils everytime we want to run a test.

but i think we might be able to manage that via nextest profiles.

let me check.

bmwill

not major issues, mostly just some nits or asks to add TODOs so we don't forget things.

bmwill · 2026-04-28T14:57:40Z

+        run_bitcoin_withdrawal_e2e(true).await
+    }
+
+    async fn run_bitcoin_withdrawal_e2e(with_guardian: bool) -> Result<()> {


I suppose this makes sense while we are getting the integration hooked up. We may want to evaluate if we want to have all tests run with guardian by default eventually.

bmwill · 2026-04-28T15:01:06Z

+    /// Start an operator-init'd guardian. Withdrawal RPCs stay gated until
+    /// [`Self::finalize`] completes provisioner-init.
+    pub async fn start(network: Network) -> Result<Self> {


At which stage is the guardian pubkey available? Here or after finalize? we do need to commit to it, and likely a url of some such, onchain and it would probably be beneficial to be able to have the key available at contract init time.

From a conversation we had today, the committing onchain will be done post this series

bmwill · 2026-04-28T20:25:39Z

+            .await
+            .context("bind guardian harness listener")?;
+        let addr: SocketAddr = listener.local_addr()?;
+        let endpoint = format!("http://{addr}");


Just a note, do we want to use traditional TLS termination or require mTLS from the validators to the guardian?

bmwill · 2026-04-28T20:27:52Z

+    nodes[0]
+        .wait_for_mpc_key(std::time::Duration::from_secs(120))
+        .await?;


We do we need this if we fetch the key from onchain below?

bmwill · 2026-04-28T20:28:10Z

+        .onchain_state()
+        .current_committee()
+        .ok_or_else(|| anyhow::anyhow!("no current committee after DKG"))?;
+    let master_pubkey = hashi.get_onchain_mpc_pubkey()?;


s/master/mpc

bmwill · 2026-04-28T20:30:21Z

+    /// URL of the `hashi-guardian` gRPC endpoint. When not set, the guardian
+    /// integration is bypassed.
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub guardian_endpoint: Option<String>,


maybe add a TODO that this is temporary and will be removed once we commit it onchain?

bmwill · 2026-04-28T20:31:22Z

+pub struct GuardianClient {
+    endpoint: String,
+    channel: Channel,
+}


We should instrument this with metrics like we do our other gRPC client. Happy to add a TODO and add them as a follow up PR.

bmwill · 2026-04-28T20:32:46Z

+    /// TODO: Investigate if it can be moved to std::sync::Mutex
+    pub shares: tokio::sync::Mutex<Vec<Share>>,


yeah we should have this be std::sync::Mutex

bmwill · 2026-04-28T20:35:08Z

+pub mod enclave;
+pub mod getters;
+pub mod heartbeat;
+pub mod init;
+pub mod rpc;
 pub mod s3_logger; // used by the monitor
+pub mod setup;
+pub mod withdraw;


were these files just not getting compiled?

bmwill · 2026-04-28T20:51:22Z

-mod getters;
-mod heartbeat;
-mod init;
-mod rpc;
-mod s3_logger;
-mod setup;
-mod withdraw;


ah this is where they were.

`GuardianHarness::finalize` and `create_fully_initialized_enclave` were running the same "drive an operator-init'd enclave to fully-initialized state" sequence (set BTC keys, build init state, state.init, mark logged) inline. Lift it into `hashi_guardian::test_utils::finalize_enclave` and call it from both. The harness's `finalize` shrinks to a single helper call plus the `is_fully_initialized` post-condition. No visibility or API changes to the `Enclave` types.

The `network` field was set in `start()` but never read internally; the getter was marked `#[allow(dead_code)]`. The enclave already owns the network, and `pub` already opts out of dead-code lint, so the attribute on a `pub` API was a smell.

Extract `run_bitcoin_withdrawal_e2e(with_guardian: bool)` as the shared body for the deposit → withdrawal → confirm flow. Both `test_bitcoin_withdrawal_e2e_flow` and `test_bitcoin_withdrawal_with_guardian_e2e_flow` now reduce to one-line wrappers that pass `false` / `true`. Forces the two flows to stay in lockstep — any future change to the deposit/withdrawal path is automatically exercised under both configurations. The guardian variant gains the post-deposit balance assertion and the withdrawal_request_id / withdrawal_txid log lines that were previously only in the no-guardian variant; behavior is otherwise unchanged. `setup_test_networks` is no longer called from the bitcoin-withdrawal test (its setup is now inlined into the parameterized helper) but remains in use by the other 3 e2e tests.

Reuse setup_test_networks via a shared `_with(builder)` helper so the guardian withdrawal flow doesn't re-implement the network-init logging. Trim doc comments on guardian_harness, finalize_enclave, and create_fully_initialized_enclave.

Collapse the wrapper + helper pair into one function that always takes a builder. The 3 existing call sites pass `TestNetworksBuilder::new() .with_nodes(4)` explicitly, which removes the wrapper-of-a-wrapper and keeps any future setup customization at the call site.

0xsiddharthks requested a review from bmwill as a code owner April 17, 2026 16:41

0xsiddharthks marked this pull request as draft April 17, 2026 17:21

0xsiddharthks closed this Apr 17, 2026

0xsiddharthks reopened this Apr 19, 2026

0xsiddharthks force-pushed the siddharth/guardian-e2e-harness branch from aac0a18 to 2fd7fb1 Compare April 19, 2026 23:42

0xsiddharthks changed the title ~~[6/n][guardian-integration] in-process e2e harness + guardian-enabled withdrawal test~~ [2/n][guardian-integration] in-process e2e harness + TestNetworksBuilder.with_guardian() Apr 19, 2026

0xsiddharthks changed the base branch from siddharth/guardian-restart-safety to siddharth/deploy-guardian April 19, 2026 23:42

0xsiddharthks force-pushed the siddharth/guardian-e2e-harness branch from 2fd7fb1 to f58edae Compare April 19, 2026 23:51

0xsiddharthks force-pushed the siddharth/deploy-guardian branch from 370b474 to 942504b Compare April 23, 2026 10:08

0xsiddharthks force-pushed the siddharth/guardian-e2e-harness branch from f58edae to c124d70 Compare April 23, 2026 10:09

0xsiddharthks force-pushed the siddharth/deploy-guardian branch from 942504b to 91bcf58 Compare April 23, 2026 11:34

0xsiddharthks force-pushed the siddharth/guardian-e2e-harness branch from c124d70 to 2fcf91f Compare April 23, 2026 11:36

0xsiddharthks force-pushed the siddharth/deploy-guardian branch from 91bcf58 to e0a17e2 Compare April 23, 2026 11:46

0xsiddharthks force-pushed the siddharth/guardian-e2e-harness branch from 2fcf91f to 56510fb Compare April 23, 2026 11:54

0xsiddharthks marked this pull request as ready for review April 23, 2026 15:10

0xsiddharthks force-pushed the siddharth/deploy-guardian branch from e0a17e2 to 1422d8d Compare April 26, 2026 21:22

0xsiddharthks force-pushed the siddharth/guardian-e2e-harness branch from 56510fb to 1bb33df Compare April 26, 2026 21:22

0xsiddharthks force-pushed the siddharth/deploy-guardian branch from 1422d8d to 89064c8 Compare April 26, 2026 22:01

0xsiddharthks force-pushed the siddharth/guardian-e2e-harness branch 4 times, most recently from 96f3774 to 49100e6 Compare April 27, 2026 03:27

0xsiddharthks requested review from Bridgerz, dlukegordon and mskd12 April 27, 2026 21:22

mskd12 reviewed Apr 28, 2026

View reviewed changes

Comment thread crates/hashi-guardian/src/enclave.rs Outdated

0xsiddharthks force-pushed the siddharth/guardian-e2e-harness branch from f5289b5 to 67c9469 Compare April 28, 2026 16:35

0xsiddharthks changed the title ~~[2/n][guardian-integration] in-process e2e harness + TestNetworksBuilder.with_guardian()~~ [2/n][guardian-integration] in-process e2e harness Apr 28, 2026

bmwill approved these changes Apr 28, 2026

View reviewed changes

Base automatically changed from siddharth/deploy-guardian to main April 28, 2026 21:28

0xsiddharthks added 8 commits April 28, 2026 14:39

[guardian] in-process e2e harness + TestNetworksBuilder.with_guardian()

d9d08bf

[guardian] use workspace tokio-stream

4d8d06a

[guardian] trim docs and dedupe e2e setup

4c8fd40

Reuse setup_test_networks via a shared `_with(builder)` helper so the guardian withdrawal flow doesn't re-implement the network-init logging. Trim doc comments on guardian_harness, finalize_enclave, and create_fully_initialized_enclave.

[guardian] add is_*_set() predicates and re-privatize btc key fields

dffff49

0xsiddharthks force-pushed the siddharth/guardian-e2e-harness branch from 67c9469 to dffff49 Compare April 28, 2026 21:44

0xsiddharthks enabled auto-merge (squash) April 28, 2026 21:45

0xsiddharthks merged commit c735437 into main Apr 28, 2026
11 checks passed

0xsiddharthks deleted the siddharth/guardian-e2e-harness branch April 28, 2026 21:57

zhouwfang pushed a commit that referenced this pull request Apr 30, 2026

[2/n][guardian-integration] in-process e2e harness (#466)

4c8cac6

		/// TODO: Investigate if it can be moved to std::sync::Mutex
		pub shares: tokio::sync::Mutex<Vec<Share>>,

Conversation

0xsiddharthks commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Tests harness changes

Uh oh!

mskd12 Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bmwill left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

0xsiddharthks commented Apr 17, 2026 •

edited

Loading

mskd12 Apr 28, 2026 •

edited

Loading