test(tests-bypass-email): Use mocked email on testing #318
test(tests-bypass-email): Use mocked email on testing #318christopherferreira9 wants to merge 18 commits into
Conversation
…se a mocked publicKey updates integration tests to also follow the same logic for using a real email or use a mocked email
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
commit: |
…catch fixture drift
|
|
||
| test: | ||
| name: Test | ||
| name: Unit Tests |
There was a problem hiding this comment.
Renamed for a clear read on the pipeline since it hasn't been extracted to the dedicated test workflow.
| use_real_email: false | ||
| secrets: inherit | ||
|
|
||
| all-checks-pass: |
There was a problem hiding this comment.
single job to have a mandatory success status. Easier to maintain through code.
|
|
||
| on: | ||
| schedule: | ||
| - cron: '0 8 * * *' |
There was a problem hiding this comment.
Currently scheduled to run daily using using real email keeping the original behavior and reducing the amount of flakiness due to rate limiting from the email service probider.
| # Pin the bundler/paymaster to staging so the AA stack matches the | ||
| # staging KMS above (the SDK default host is prod). | ||
| NEXT_PUBLIC_ZERODEV_AA_HOST: https://staging-meta-aa-provider.onrender.com | ||
| USE_REAL_EMAIL: ${{ inputs.use_real_email }} |
There was a problem hiding this comment.
Demo is built with this env variable which is then passed downstream to the wagmi options.
| const otpCode = extractOtpCodeFromMagicLinkUrl(emailContent) | ||
| // Step 6: Resolve OTP code — use fixture value in mock mode, poll inbox in real mode | ||
| const otpCode = | ||
| magicLinkSession.otpCode ?? |
There was a problem hiding this comment.
when in mock mode, otpCode comes directly from the fixture. The fixture handles the real vs mocked email logic.
Summary
Previously, all E2E and integration tests required hitting Guerrilla Mail and Turnkey to complete OTP flows, making them brittle and slow due to external rate limits. This PR introduces a USE_REAL_EMAIL flag that switches tests between real and mocked backends — PR runs use mocked mode (no external dependencies), and a new daily scheduled workflow runs with real email to catch live-service regressions.
SDK side changes
packages/core/src/core/createZeroDevWalletCore.tsAdded dangerouslyOverrideOtpSignerPublicKey?: string to ZeroDevWalletConfigCore. When present, it's forwarded to encryptOtpAttempt as dangerouslyOverrideSignerPublicKey, bypassing the production-pinned TLS Fetcher key check. The spread conditional (...(config.x && { ... })) ensures the property is only passed when explicitly set — it never appears in a normal integrator's call.
packages/react/src/core/connector.tsSame option added to ConnectorCoreParams. It's forwarded into createZeroDevWallet() using the same conditional spread — so an integrator who doesn't set it gets no property at all, not undefined.
Both additions are marked @internal with a "Never set this in production" JSDoc comment.
Tests added
packages/core/src/utils/encryptOtpAttempt.test.ts— new test: generates a fresh ephemeral key pair (not the production pinned key), builds a test bundle signed with it, and asserts that calling encryptOtpAttempt without the override throws does not match pinned signing key. This proves the default path enforces the production key.packages/react/src/connector.test.ts— two new tests in a dangerouslyOverrideOtpSignerPublicKey security guard describe block:Workflow changes
ci.ymlbuildjob for parallel execution in favor of faster feedback from tests.all-checks-passpassing.test.yml- new workflowReusable workflow extracted from
ci.yml. It contains:pnpm test:integrationBoth of these workflows accept a newly added input called
use_real_emailwith the default value beingtrue.ci.ymlthen sendsuse_real_emailasfalsein order to remove the dependency on the email service on PRs and main runs (we may consider using the real email service for main runs - to be discussed).test.yml- new workflowRuns test.yml daily at 08:00 UTC with use_real_email: true (real Guerrilla Mail + Turnkey). Also triggerable manually via workflow_dispatch with the option to toggle email mode.
Concerns
1. Mock fidelity
As the payload returned by the backend can change, fixtures can become stale and results would not be reliable. A
fixture-shape.test.tstest has been created to catch this drift on real email mode. On a future development, the PR t update the fixture can be made automatically on failure.2. The rejection test tests the mock, not the logic
"should reject an invalid OTP code" in mock mode stacks setupNodeMocks({ rejectOtpVerify: true }), which unconditionally returns HTTP 400 regardless of what was sent. It doesn't verify that a semantically wrong OTP (wrong code, wrong encryption) is actually detected — it verifies that an HTTP 400 throws an exception. This test is still valued as we have UI checks - we should rename this test if we intend to keep it.
3. dangerouslyOverrideSignerPublicKey is a scattered pattern
Every integration test call site that invokes encryptOtpAttempt directly must manually pass dangerouslyOverrideSignerPublicKey: isRealEmail() ? undefined : MOCK_OTP_SIGNER_PUBLIC_KEY. This is currently in three places. Anyone writing a new integration test that calls encryptOtpAttempt directly will get a cryptic key-mismatch error in mock mode until they discover the pattern. This can be mitigated.
4. Fork PRs have no integration test coverage at all
call-tests is skipped on fork PRs (no secret access). all-checks-pass treats skipped as OK, so external contributors get zero integration or E2E feedback. A fork PR that breaks the OTP flow in a way that unit tests don't catch can pass all required checks. I personally think this is acceptable.
5. otp-init-shape.test.ts is permanently skipped in mock mode
This test exists specifically to validate the shape of the real backend's OTP init response and verify the production pinned key. Skipping it in mock mode means the CI run that runs on every PR never exercises it. It only runs in the daily scheduled run. If the production key rotates or the bundle shape changes, a PR that breaks the integration won't be caught at PR time.
6. Passkeys can't be tested when the mock mode is enabled