test(tests-bypass-email): Use mocked email on testing #318
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| name: Scheduled Tests | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: '0 8 * * *' | ||
|
Author
There was a problem hiding this comment. Currently scheduled to run daily using using real email keeping the original behavior and reducing the amount of flakiness due to rate limiting from the email service probider. |
||
| workflow_dispatch: | ||
| inputs: | ||
| use_real_email: | ||
| description: 'Use real email service (mail.tm) for OTP tests' | ||
| required: false | ||
| default: true | ||
| type: boolean | ||
|
|
||
| jobs: | ||
| scheduled-tests: | ||
| name: Scheduled Tests (Real Email) | ||
| uses: ./.github/workflows/test.yml | ||
| with: | ||
| use_real_email: ${{ github.event_name == 'workflow_dispatch' && inputs.use_real_email || true }} | ||
| secrets: inherit | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,130 @@ | ||
| name: Tests | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| use_real_email: | ||
| description: 'Use real email service (mail.tm) for OTP tests' | ||
| required: false | ||
| default: true | ||
| type: boolean | ||
| secrets: | ||
| ZD_PROJECT_ID: | ||
| required: true | ||
| workflow_dispatch: | ||
| inputs: | ||
| use_real_email: | ||
| description: 'Use real email service (mail.tm) for OTP tests' | ||
| required: false | ||
| default: true | ||
| type: boolean | ||
|
|
||
| jobs: | ||
| integration-test: | ||
| name: Integration Tests | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Annotate email mode | ||
| run: | | ||
| if [ "${{ inputs.use_real_email }}" = "true" ]; then | ||
| echo "::notice::Email mode: real (Guerrilla Mail + Turnkey)" | ||
| else | ||
| echo "::notice::Email mode: mock (no external dependencies)" | ||
| fi | ||
|
|
||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup pnpm | ||
| uses: pnpm/action-setup@v4 | ||
|
|
||
| - name: Setup Node.js | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: 20 | ||
| cache: pnpm | ||
|
|
||
| - name: Install dependencies | ||
| run: pnpm install --frozen-lockfile | ||
|
|
||
| - name: Run integration tests | ||
| run: pnpm test:integration | ||
| env: | ||
| ZD_PROJECT_ID: ${{ secrets.ZD_PROJECT_ID }} | ||
| USE_REAL_EMAIL: ${{ inputs.use_real_email }} | ||
|
|
||
| e2e-test: | ||
| name: Browser E2E Tests | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Annotate email mode | ||
| run: | | ||
| if [ "${{ inputs.use_real_email }}" = "true" ]; then | ||
| echo "::notice::Email mode: real (Guerrilla Mail + Turnkey)" | ||
| else | ||
| echo "::notice::Email mode: mock (no external dependencies)" | ||
| fi | ||
|
|
||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Setup pnpm | ||
| uses: pnpm/action-setup@v4 | ||
|
|
||
| - name: Setup Node.js | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: 20 | ||
| cache: pnpm | ||
|
|
||
| - name: Install dependencies | ||
| run: pnpm install --frozen-lockfile | ||
|
|
||
| - name: Build SDK | ||
| run: pnpm build | ||
|
|
||
| - name: Install Playwright browsers | ||
| run: pnpm exec playwright install --with-deps chromium | ||
|
|
||
| - name: Build and start demo app | ||
| working-directory: apps/zerodev-signer-demo | ||
| run: | | ||
| if [ "$USE_REAL_EMAIL" = "false" ]; then | ||
| export NEXT_PUBLIC_DANGEROUS_OTP_SIGNER_KEY=$(cat ../../e2e/fixtures/test-signer-public-key.txt) | ||
| fi | ||
| pnpm build | ||
| pnpm start & | ||
| env: | ||
| NEXT_PUBLIC_ZERODEV_PROJECT_ID: ${{ secrets.ZD_PROJECT_ID }} | ||
| NEXT_PUBLIC_KMS_PROXY_BASE_URL: https://kms.staging.zerodev.app/api/v1 | ||
| # Pin the bundler/paymaster to staging so the AA stack matches the | ||
| # staging KMS above (the SDK default host is prod). | ||
| NEXT_PUBLIC_ZERODEV_AA_HOST: https://staging-meta-aa-provider.onrender.com | ||
| USE_REAL_EMAIL: ${{ inputs.use_real_email }} | ||
|
Author
There was a problem hiding this comment. Demo is built with this env variable which is then passed downstream to the wagmi options. |
||
|
|
||
| - name: Wait for demo app | ||
| run: | | ||
| for i in $(seq 1 15); do | ||
| if curl -sf http://localhost:3000 > /dev/null 2>&1; then | ||
| echo "Demo app is ready" | ||
| exit 0 | ||
| fi | ||
| echo "Waiting for demo app... ($i/15)" | ||
| sleep 2 | ||
| done | ||
| echo "Demo app failed to start" | ||
| exit 1 | ||
|
|
||
| - name: Run E2E tests | ||
| run: pnpm test:e2e | ||
| env: | ||
| DEMO_APP_URL: http://localhost:3000 | ||
| USE_REAL_EMAIL: ${{ inputs.use_real_email }} | ||
|
|
||
| - name: Upload Playwright report | ||
| uses: actions/upload-artifact@v4 | ||
| if: always() | ||
| with: | ||
| name: playwright-report | ||
| path: playwright-report/ | ||
| retention-days: 14 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| # E2E Tests | ||
|
|
||
| ## Two-mode testing strategy | ||
|
|
||
| Browser tests run in two modes controlled by the `USE_REAL_EMAIL` environment variable. | ||
|
|
||
| | | Mock mode (`USE_REAL_EMAIL=false`) | Real-email mode (`USE_REAL_EMAIL=true`) | | ||
| |---|---|---| | ||
| | **When** | PR CI, local development | Nightly scheduled run | | ||
| | **Email** | Intercepted via `page.route()` | mail.tm temporary inbox | | ||
| | **Auth session** | Fake JWT (never hits backend) | Real session from ZeroDev KMS | | ||
| | **Signing** | Fake signature stub | Real ZeroDev KMS signing | | ||
| | **Wallet address** | Indeterminate (no real account) | Real counterfactual address | | ||
| | **Project config** | Not enforced | Enforced (chains, sponsorship) | | ||
| | **Speed** | Fast, no network waits | Slower, dependent on external services | | ||
|
|
||
| ### What mock mode does NOT cover | ||
|
|
||
| Mock mode verifies UI flow and component wiring. It deliberately bypasses: | ||
|
|
||
| - **Session validation** — the fake JWT is never checked against the database; user/org do not exist. | ||
| - **Project configuration** — allowed chains, permitted operations, and gas sponsorship rules are never evaluated. | ||
| - **Cryptographic signing** — the returned signature is a deterministic stub; it is not valid for any message and is not tied to any key or wallet address. | ||
| - **Real wallet address** — no account is created; wallet address is meaningless in mock mode. | ||
|
|
||
| These gaps exist because mocking auth cascades: once auth is mocked the backend rejects all subsequent authenticated requests, so signing endpoints must also be mocked. This is an accepted tradeoff — the nightly real-email run covers correctness end-to-end. | ||
|
|
||
| ### What mock mode does cover | ||
|
|
||
| - OTP entry form renders and accepts input | ||
| - Magic-link entry form renders and accepts input | ||
| - Error states (wrong code, expired session) — by adjusting fixture responses | ||
| - Post-auth UI (wallet address display, sign message, logout flow) | ||
| - Component composition in `@zerodev/wallet-react-kit` | ||
|
|
||
| ## Directory structure | ||
|
|
||
| ``` | ||
| e2e/ | ||
| ├── browser/ # Playwright tests (UI + integration) | ||
| │ ├── otp.spec.ts | ||
| │ ├── magic-link.spec.ts | ||
| │ └── post-auth.spec.ts | ||
| ├── integration/ # Vitest tests against real backend (no UI) | ||
| ├── fixtures/ | ||
| │ ├── auth.ts # Playwright fixture extensions (OTP, magic-link, authenticated sessions) | ||
| │ ├── test-otp-bundle.json # Pre-signed EncryptionTargetEnvelope for mock mode | ||
| │ ├── test-signer-public-key.txt # enclaveQuorumPublic hex for the test key pair | ||
| │ └── README.md # Fixture regeneration instructions | ||
| ├── helpers/ | ||
| │ ├── mock-backend.ts # page.route() intercepts for auth + signing | ||
| │ ├── mock-session.ts # Fake session JWT builder | ||
| │ └── env-utils.ts # isRealEmail() helper | ||
| └── scripts/ | ||
| └── generate-test-otp-bundle.ts # One-time fixture generator | ||
| ``` | ||
|
|
||
| ## Running tests locally | ||
|
|
||
| ```bash | ||
| # Mock mode (default — no email service needed) | ||
| pnpm test:e2e:headed | ||
|
|
||
| # Real-email mode | ||
| USE_REAL_EMAIL=true pnpm test:e2e:headed | ||
| ``` | ||
|
|
||
| `USE_REAL_EMAIL` unset is treated as `false` — mock mode is the default so tests work out of the box without any configuration. | ||
|
|
||
| ## CI workflows | ||
|
|
||
| - **`ci.yml`** calls `test.yml` with `use_real_email: false` on every PR and push to `main`/`develop`. | ||
| - **`scheduled-tests.yml`** calls `test.yml` with `use_real_email: true` daily at 08:00 UTC. | ||
|
|
||
| ## Fixture maintenance | ||
|
|
||
| The pre-signed OTP bundle in `e2e/fixtures/` must be regenerated when `encryptOtpAttempt` validation logic or the `EncryptionTargetEnvelope` shape changes. See [`e2e/fixtures/README.md`](fixtures/README.md) for instructions. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Renamed for a clear read on the pipeline since it hasn't been extracted to the dedicated test workflow.