E2E: one-time global auth before test start

[phyllis-sy-wu]

WHY are these changes introduced?

E2E tests currently authenticate the CLI and browser per worker on every test run. This PR centralizes authentication into a single globalSetup step that runs once before tests start.

This lays the foundation for:

• Parallel workers (future PR): Without global auth, each worker would need to authenticate independently, causing redundant OAuth flows and potential rate limiting
• Per-test store creation (future PR): Browser needs pre-authenticated cookies for admin.shopify.com to create dev stores via the store creation form

WHAT is this pull request doing?

Adds a Playwright globalSetup that authenticates once before tests start, then reuses the session:

1. CLI auth (setup/global-auth.ts): Spawns shopify auth login via PTY, completes OAuth in a headless browser (with passkey/WebAuthn bypass), waits for "Logged in"
2. Browser session establishment: Visits admin.shopify.com and dev.shopify.com to establish cookies for both domains (not just accounts.shopify.com)
3. Session reuse (setup/auth.ts): Copies the pre-authenticated CLI session files (XDG dirs) and loads browser storageState — no re-authentication needed

Also includes a CodeQL fix: URL checks for accounts.shopify.com redirects now use new URL().hostname comparison instead of substring matching.

Design decisions: session persistence and caching

Approach: Follows Playwright's recommended storageState pattern — global setup authenticates once and saves browser cookies to a JSON file. Workers load the saved state into their browser context.

Session caching across runs (local dev only):
Before re-authenticating, globalSetup checks if a valid cached session exists from a previous run by loading the saved storageState and verifying the browser is still logged into admin.shopify.com. If valid, skips the full OAuth flow (~30s → ~3s). If expired, re-authenticates and overwrites the cache. On CI (fresh runner each time), there is no cache, so it always authenticates fresh. Caching auth tokens in GitHub Actions cache was considered but rejected since this is a public repo — storing OAuth tokens in GitHub's cache storage is a security risk.

Directory structure:

.e2e-tmp/
├── global-auth/                    ← stable dir, all auth artifacts together
│   ├── XDG_CONFIG_HOME/            ← CLI tokens (used on Linux CI)
│   ├── XDG_DATA_HOME/
│   ├── XDG_STATE_HOME/
│   ├── XDG_CACHE_HOME/
│   └── browser-storage-state.json  ← browser cookies (used everywhere)
├── e2e-{worker0}/                  ← per-worker isolated dirs
└── e2e-{worker1}/

• global-auth/: Stable name (not random), reused across runs. All auth state in one place — no scattered files.
• Per-worker dirs: Random name via mkdtemp, isolated XDG dirs so parallel workers don't corrupt each other's CLI state. Workers copy CLI tokens from global-auth/ into their own dir at startup.

Cross-platform note: On macOS, the CLI's conf package ignores XDG env vars and writes to ~/Library/Preferences/. The XDG dirs in global-auth/ are only populated on Linux. Browser storageState works on all platforms.

Files changed:

File	Change
setup/global-auth.ts	New: one-time auth + session caching + browser cookie establishment
setup/auth.ts	Copies session from global setup instead of re-authenticating
setup/browser.ts	Loads storageState from global setup
setup/env.ts	Adds globalLog() helper for pre-test logging
playwright.config.ts	Adds globalSetup entry

How to test your changes?

# Run all tests
DEBUG=1 pnpm --filter e2e exec playwright test

# Quick test with a single spec
DEBUG=1 pnpm --filter e2e exec playwright test app-deploy

# Watch browser behavior (headed mode)
E2E_HEADED=1 DEBUG=1 pnpm --filter e2e exec playwright test

# Second run — should reuse cached session
DEBUG=1 pnpm --filter e2e exec playwright test

# Force re-auth (clear cache)
rm -rf .e2e-tmp/global-auth
DEBUG=1 pnpm --filter e2e exec playwright test

Example: session caching saves ~30-40s on subsequent runs

First run — full auth (~1.5 min total):

$ E2E_HEADED=1 DEBUG=1 pnpm --filter e2e exec playwright test app-deploy
[e2e][auth] global setup starting
[e2e][auth] no cached session found

To run this command, log in to Shopify.
User verification code: QLFS-BNXL
👉 Open this link to start the auth process: https://accounts.shopify.com/activate-with-code?...
✔ Logged in.
✔ Current account: genghis-khan-identity-1-2025-03-31@shopify.com.
[e2e][auth] browser sessions established for admin + dev dashboard
[e2e][auth] global setup done, config at .e2e-tmp/global-auth/XDG_CONFIG_HOME

  ✓  1 tests/app-deploy.spec.ts › App deploy › deploy and verify version exists (41.1s)

  1 passed (1.5m)

Second run — cached session (~48s total, auth skipped):

$ E2E_HEADED=1 DEBUG=1 pnpm --filter e2e exec playwright test app-deploy
[e2e][auth] global setup starting
[e2e][auth] reusing cached session

  ✓  1 tests/app-deploy.spec.ts › App deploy › deploy and verify version exists (44.3s)

  1 passed (48.4s)

Checklist

• I've considered possible cross-platform impacts (Mac, Linux, Windows)
• I've considered possible documentation changes
• I've considered analytics changes to measure impact
• The change is user-facing, so I've added a changelog entry with pnpm changeset add

[phyllis-sy-wu]

• E2E: parallel workers #7309
• E2E: per-test store isolation with automatic teardown #7279
• E2E: temporary time limit bump #7282
• E2E: improve error context in assertions #7281
• E2E: one-time global auth before test start #7272 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[phyllis-sy-wu]

No issues found

---

Review assisted by pair-review

[Copilot]

Pull request overview

Introduces a Playwright globalSetup flow to authenticate the Shopify CLI and browser once per test run, then reuse the resulting CLI session files and browser storageState across worker-scoped fixtures.

Changes:

• Added setup/global-auth.ts global setup to perform one-time CLI OAuth login and persist browser storage state.
• Updated worker fixtures to reuse global session artifacts (copy XDG auth dirs + load storageState) instead of re-authenticating.
• Added a globalLog() helper for pre-worker logging gated behind DEBUG=1.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
packages/e2e/setup/global-auth.ts	New global setup that performs one-time login and writes reusable session artifacts.
packages/e2e/setup/auth.ts	Reuses global CLI session by copying XDG dirs; retains fallback per-worker login.
packages/e2e/setup/browser.ts	Loads Playwright storageState from global setup when available.
packages/e2e/setup/env.ts	Adds globalLog() for debug logging during global setup.
packages/e2e/playwright.config.ts	Registers the new Playwright globalSetup.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ryancbahan]

The idea here is great! I'd encourage you to consider how other libraries and tools implement persistence between runs/manage sessions, though. This is a problem that has a lot of prior art to lean on. I don't think we'll end up needing multiple tmp dirs per-state. And I think some clarity in the pr description around choice of state prsistence and caching approach is important here.

[phyllis-sy-wu]

@ryancbahan Thanks for the feedback!

I've updated the PR description with a "Design decisions" section covering the persistence approach. Please let me know if it’s still unclear.

Code changes based on this review:

• Session caching: Added cross-run caching for local dev. Before re-authenticating, globalSetup loads the saved browser state and checks if it's still valid (~3s). If so, skips the full OAuth flow (~30s). I also explored doing the same on CI, but since CLI is a public repo, we can't safely cache auth tokens in GitHub Actions. So CI still re-authenticates fresh each run.
• Consolidated auth dir: All auth artifacts now live in a stable global-auth/ directory under .e2e-tmp/ (instead of random mkdtemp dirs that accumulated across runs). Browser state JSON and XDG dirs are both inside it for a cleaner structure.

Things I kept:

• Follows Playwright's recommended storageState pattern — global setup authenticates once, saves browser cookies to a JSON file, workers load it via storageState.
• Per-worker temp dirs remain — workers copy auth into their own isolated XDG dirs because the CLI writes to config at runtime, so parallel workers need separate copies. This is the minimum for parallel execution.

More details in the updated PR description.

[ryancbahan]

do you need any of this at all if you run global auth once before the tests run, then don't call login() or any auth commands in any of the tests?

[phyllis-sy-wu]

Yes! This copying step is needed for parallel worker isolation.

To be clear: this separation is for CLI command isolation, not for browser or login needs.

It’s correct that browser auth works fine with a single shared browser-storage-state.json — Playwright handles that natively. The problem is that our tests run CLI commands which read and write to config directories at runtime (storing app state, caching API responses, etc.). If multiple workers share the same config directory, they’d write to the same files simultaneously, causing race conditions and corrupted JSON.

.e2e-tmp/
├── global-auth/                          ← created once by globalSetup
│   ├── XDG_CONFIG_HOME/                  ← CLI tokens (source of truth)
│   ├── XDG_DATA_HOME/
│   ├── XDG_STATE_HOME/
│   ├── XDG_CACHE_HOME/
│   └── browser-storage-state.json        ← shared read-only by all workers ✅
│
├── e2e-worker0/                          ← worker 0's isolated copy
│   ├── XDG_CONFIG_HOME/                  ← copied from global-auth, written to by CLI commands
│   ├── XDG_DATA_HOME/                    ← worker 0's app deploy writes here
│   └── ...
│
└── e2e-worker1/                          ← worker 1's isolated copy
    ├── XDG_CONFIG_HOME/                  ← copied from global-auth, written to by CLI commands
    ├── XDG_DATA_HOME/                    ← worker 1's app dev writes here
    └── ...

Global auth creates one set of CLI tokens in global-auth/. Each worker copies those into its own isolated XDG dirs so parallel CLI commands don’t interfere with each other.

The CLI writes to these local stores during commands:

Store	Written during	Conflict risk if shared
shopify-cli-kit	Many commands (session tokens, current session ID, cached responses)	HIGH — global fields like currentSessionId get overwritten; last write wins
shopify-cli-app	app dev, app deploy (cached app info: appId, storeFqdn)	Medium — entries are keyed per app dir, but conf rewrites the whole JSON per write, so concurrent writes can still race at the filesystem level
shopify-cli-store	Store operations	Medium — same file-level race risk as above

The critical one is cliKitStore — it stores the current session ID in a single field. If worker 0 and worker 1 write at the same time, one overwrites the other, causing auth failures.

---

Another original reason to keep auth.ts was a fallback path (authenticating directly without global setup) for local dev when running a single test without globalSetup. But both auth.ts and global-auth.ts use the same login logic — if global-auth fails, the fallback in auth.ts would fail the same way. So the fallback is effectively dead code on this branch. Only the copy step is strictly needed, and we could converge auth.ts and global-auth.ts in a follow-up PR. Does that make sense?

[ryancbahan]

I think it's worth exploring whether you can auth once per job run at the top level and pass the session in-memory. There's two reasons I think this approach merits consideration:

1. Storing on filesystem won't be useful in the GH actions runs since we spawn a new container each time
2. writing to disk means we need to pay more attention to if those files get exposed as build artifacts (which we don't want).

[phyllis-sy-wu]

I think it's worth exploring whether you can auth once per job run at the top level and pass the session in-memory.

Playwright communicates between globalSetup and workers via process.env + filesystem — there’s no built-in in-memory mechanism across worker processes. The CLI also reads tokens from disk (via the conf package), not memory, so filesystem is required for CLI commands to work.

1. Storing on filesystem won't be useful in the GH actions runs since we spawn a new container each time

You're right that cross-run caching doesn't help (fresh container). Within a single run, filesystem is still needed for Playwright's storageState and CLI token access. The caching is purely a local-dev optimization so that iteration locally can be quicker.

2. writing to disk means we need to pay more attention to if those files get exposed as build artifacts (which we don't want).

Auth files live in .e2e-tmp/ (gitignored). CI only uploads playwright-report/ and test-results/ — tokens are not exposed.

More details on why per-worker copies are needed in my reply to the auth.ts comment above.