test: enforce api test boundary lint rules#20713
Merged
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Contributor
|
API preview: https://pr-20713-api.vm6.ai |
de0cd1f to
10b2340
Compare
6464415 to
c9f41eb
Compare
Enforce the API test boundary model: no service-level API tests, no direct DB access from tests or helpers, and no vitest usage of /api/test/* fixture routes. Tests construct state through production APIs with MSW-mocked providers, isolate by random IDs, and assert through product read surfaces. Deletes 13 test-only contract modules; production-unreachable state moves to a narrow non-HTTP fixture layer under src/test-fixtures/. Keep the deployed-E2E infrastructure (cli-auth-test provisioning, test-oauth provider routes, Slack/Telegram mock/state/probe routes and client redirection) behind a single documented E2E_ROUTES registry, still env-gated to development/preview. Scope global-sweep crons in tests: MemoryViewer fixtures opt back out after each test, drain pacing and the firewall refresh timeout become optional env reads, and bounded-sweep assertions poll to convergence instead of asserting one tick's global counters. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
c9f41eb to
0399e10
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
src/signals/services/**/*.test.ts, sibling*.serviceimports from API tests, and directsignals/external/dbimports from API tests are rejected.test-relationship-state,test-usage-state,test-generation-state,test-memory-state,test-chat-thread-state,test-chat-messages-state,test-workflow-trigger-state,test-email-state,test-webhooks-state,test-billing-redeem-state,test-onboarding-status-state,test-user-export-state.apps/api/src/test-fixtures/(14 small doc-commented modules, e.g. identical-created_atchat messages for cursor tie-breaks, platformvm0_api_keysrows,usage_pricingconfig, aged email-outbox rows, expired credit grants, legacy pre-event-sourcing threads). No HTTP route, no contract.setFirewallAuthRefreshTimeoutMsForTests) with an optionalFIREWALL_AUTH_REFRESH_TIMEOUT_MSenv read, and make the email-outbox drain's Resend pacing overridable viaEMAIL_OUTBOX_DRAIN_DELAY_MS(tests disable the pacing instead of widening timeouts).services/connector-catalog-artifacts/directory (its only importer was a deleted service-level test).Global-sweep cron isolation
Removing fixture teardown surfaced a class of cross-file interference: several crons (
summarize-memory,drain-email-outbox,refresh-storage-presigned-urls) sweep shared tables globally, so one file's leftover fixtures leak into another file's sweep. Fixes, all through product surfaces:cron-summarize-memory.test.ts,run-lifecycle.bdd.test.ts) now opt their fixture users back out via the same product feature-switch API inonTestFinished, keeping later sweeps scoped.GET /api/zero/memory/activity) instead of the global cron response.refreshed <= 3) and then polls repeated cron ticks until its fixture rows converge — the same way production converges — instead of asserting a single tick's global counts.Deployed E2E infrastructure is explicitly out of scope
An earlier revision of this PR removed all test routes from the global route table and deleted the Slack/Telegram/OAuth mock routes. That broke
deploy-api("Generate E2E test tokens" calls/api/cli/auth/test-tokenon the preview) and would breakcli-e2e-01-serial/cli-e2e-02-playwright/ runner suites plus thetest-oauthconnectors inpackages/connectors, whose provider endpoints are/api/test/oauth-provider/*on the deployed preview. Those consumers run against a remote deployment where in-process MSW is impossible.This PR therefore splits the surface:
cli-auth-test,test-oauth-provider-*×5,test-slack-mock,test-slack-state,test-slack-dispatch-probe,test-telegram-mock,test-telegram-state,test-telegram-dispatch-probe,test-zero-agent-state), re-mounted through a single documented registryapps/api/src/signals/e2e-routes.ts(E2E_ROUTES). All remain env-gated byisTestEndpointAllowed(development / preview-with-bypass only; 404 in production). TheE2E_SLACK_MOCK_ENABLED/E2E_TELEGRAM_MOCK_ENABLEDpreview redirection in the Slack/Telegram clients is restored for the same reason. The lint guard still preventssrc/signals/route.tsfrom importing./routes/test-*or./routes/cli-auth-testdirectly —e2e-routes.tsis the only sanctioned aggregation point.Current test-only API surface
E2E_ROUTES(13 env-gated deployed-E2E modules); zero vitest-fixture routes.test-automations-state,test-banking-state,test-computer-use-state,test-cron-cleanup-sandboxes-state,test-cron-sync-skills-state,test-system-storage-presigned-url-cache-state,test-usage-insight-state,test-workflow-skill-storage-presigned-url-cache-state), tracked as follow-up.helpers/zero-integrations-slack.ts,helpers/zero-telegram.ts,helpers/api-bdd-firewall.ts/api-bdd-auth-device.tsprovisioning); migrating those to connector-OAuth + MSW is follow-up work.Coverage notes
test-relationship-statewere ported intosrc/test-fixtures/relationship-memory.ts(non-HTTP), so the alias-race, injection-preview, and injection-window tests keep their coverage while the fixture route stays deleted.GET /api/zero/usage/record; webhook dedup proven via no-duplicate-run). Tests that only exercised fixture internals or production-unreachable defensive branches were deleted with in-place justification.Verification
pnpm -F api lintpnpm -F @vm0/api-contracts check-typesNODE_OPTIONS=--max-old-space-size=4096 pnpm -F api check-typespnpm knip(clean)pnpm format,git diff --checkpnpm -F api exec vitest runlocally against PostgreSQL: 152 files / 2010 tests, green in consecutive runsLocal-dev caveat
Random-ID isolation on a persistent local database trades teardown for accumulation: global-sweep crons (summarize-memory, aggregate-insights, presigned-URL refresh) slowly gain work from leftover fixtures across runs. CI is unaffected (fresh database per run). The affected tests now scope their fixtures (MemoryViewer opt-out) or pre-drain/converge through repeated cron ticks; if a long-lived local database still gets slow,
scripts/prepare.sh/ a DB reseed clears the backlog.