Skip to content

Remote integration nightly does not build required server distribution #2481

Description

@ateles-agent

Summary

The Remote integration nightly workflow installs dependencies and immediately runs npm run test:remote:critical, but those tests import server modules from dist/server/**. The test global setup correctly fails closed when dist/ is absent, so a clean scheduled or manually dispatched run exits before executing any remote assertions.

Observed in workflow runs 35733498969 and 35758085854:

[neotoma] dist/ is missing. Run `npm test` (pretest builds server) or `npm run build:server` before `npx vitest run`.

Scope

Use the repository's existing server build mechanism in the workflow before the remote critical suite. Keep the change generic and limited to CI setup; do not change test behavior, production runtime behavior, auth policy, or the remote suite's fail-closed global setup.

Acceptance criteria

  • A clean Remote integration nightly run builds the server distribution before npm run test:remote:critical.
  • Removing the build step reproduces the missing-dist failure before tests run.
  • With the build step present, the workflow reaches real remote assertions and can fail on a behavioral defect.
  • The workflow reuses npm run build:server; it does not introduce a parallel build path.
  • Workflow lint and ordinary CI checks pass.

Prior art

Swarm specification

This section is maintained by the Ateles swarm. Each lens agent owns exactly one subsection below; the human-written description above these markers is never modified.

Design basis

Design basis: docs/foundation/principles.md#extend-the-mechanism-that-already-generalizes — reuse the existing npm run build:server path in CI rather than inventing a parallel dist producer; red-control acceptance also binds principle 4 (a test that cannot fail on the watched defect is decoration).

Product / Scope (PM)

Problem. The Remote integration nightly workflow never builds dist/, so test:remote:critical fail-closes in global setup and never runs remote assertions — the nightly cannot detect real behavioral regressions.

In scope

  • Add the repository’s existing npm run build:server step to .github/workflows/remote_integration_nightly.yml after npm ci and before npm run test:remote:critical.
  • Keep the change limited to CI workflow setup.
  • Preserve the suite’s fail-closed global setup when dist/ is absent (that behavior is correct; the defect is missing the build).

Out of scope

Acceptance criteria

  • A clean Remote integration nightly run executes npm run build:server before npm run test:remote:critical.
  • Effect (red control): with the build step removed (or dist/ absent after install), the job fails in global setup with the missing-dist message before remote assertions run.
  • Effect (green path): with the build step present, the job proceeds past global setup into real remote assertions and can fail on a behavioral defect (not only on missing artifacts).
  • The workflow reuses npm run build:server; no parallel build script or alternate dist producer is introduced.
  • Workflow lint (actionlint on the workflow file) and ordinary CI checks pass.
  • Cross-surface parity: N/A — capability is exposed only on the GitHub Actions workflow surface (not MCP/REST/CLI/SDK).

Priority & sequencing

Open questions: none blocking.

Engineering

Prior art check. PR #2478 (fix(ci): build server before remote nightly tests, open, Closes #2481) already implements this exact change — a single 3-line addition to .github/workflows/remote_integration_nightly.yml, no other files touched. This section specifies the same minimal change so a build PR can proceed independently of #2478's disposition (merge #2478, or reimplement identically if it's closed/superseded).

File to touch

  • .github/workflows/remote_integration_nightly.yml — the only file in scope. No source, test, or config files outside this workflow.

Concrete change

Insert a new step between the existing Install dependencies (npm ci) step and the existing Run remote critical suite (npm run test:remote:critical) step:

      - name: Build server (required by integration tests)
        run: npm run build:server
  • Step name mirrors the convention used in other CI lanes that build before consuming dist/server/**.
  • Invokes the existing npm run build:server script verbatim — no new script, no inline build logic, no alternate dist producer.
  • No changes to job-level env, permissions, triggers (schedule/workflow_dispatch), matrix, or any other step in the file.
  • Do not touch the remote suite's global setup (fail-closed dist/ check) — that logic stays exactly as-is; this only ensures dist/ exists before it runs.

Data/contract changes

None. This is CI orchestration only — no entity, schema, API, or runtime contract changes. No cross-surface parity applies (workflow-only surface, per PM section).

Layering

  • Sits entirely at the CI-workflow layer (GitHub Actions YAML), above and outside application code.
  • Reuses the pre-existing build:server npm script (defined in package.json) as-is — do not modify that script or its underlying build config as part of this fix.
  • No coupling introduced between this workflow and other workflows beyond the shared script they already both call.

Build-step checklist (what the PR does, in order)

  1. Confirm current state on main/branch tip: run npm run test:remote:critical locally (or trust the two observed failing runs 35733498969 / 35758085854) to reproduce the missing-dist fail-closed error as the red control.
  2. Edit .github/workflows/remote_integration_nightly.yml: add the Build server (required by integration tests) step running npm run build:server, positioned after Install dependencies and before Run remote critical suite.
  3. Run actionlint .github/workflows/remote_integration_nightly.yml locally; fix any lint findings (there should be none — the added step matches the pattern used elsewhere in the repo).
  4. Verify green control: with the step present, run npm run build:server then npm run test:remote:critical locally and confirm the suite passes global setup and reaches real remote assertions (does not require all assertions to pass — Remote critical MCP resource tests do not establish an authenticated user #2479 covers a separate downstream fixture failure once assertions run).
  5. Run the repo's ordinary pre-commit/CI checks (type-check, lint, full test suite) — expect no impact since no non-workflow files changed.
  6. No generated/derived files depend on this workflow file, so no regeneration step applies (test catalog, openapi types, etc. are untouched).
  7. Open PR referencing closes #2481; PR body states the red/green control results per the effect-test requirement in PM acceptance criteria, and cites Design basis: docs/foundation/principles.md#extend-the-mechanism-that-already-generalizes.
  8. Do not open a duplicate PR if fix(ci): build server before remote nightly tests #2478 is still open and unmodified from the diff shown above — in that case the build step for this issue is to review/merge fix(ci): build server before remote nightly tests #2478 rather than re-author the same 3 lines.

QA / Test Plan

Surface under test. GitHub Actions workflow .github/workflows/remote_integration_nightly.yml only — no MCP / store / schema / CLI agent surface. No agentic_eval fixture (nothing to replay through the agent-facing harness). The reproducible regression guard is a workflow contract unit test (same pattern as tests/contract/deploy_workflow_config.test.ts), plus the PM red/green effect controls.

Prior art (do not duplicate).

Regression eval to commit in the same PR as the workflow fix

Id File Asserts
remote_integration_nightly_workflow tests/contract/remote_integration_nightly_workflow.test.ts (new) (1) a step run: equals exactly npm run build:server; (2) that step appears after the npm ci / Install-dependencies step and before npm run test:remote:critical; (3) no alternate dist producer is introduced in this workflow (tsc inline, npm run build without :server, custom compile scripts).

Run via ordinary Vitest contract/unit CI (not eval:tier1 / agentic_evals). Green on this file is the qa-report signal for the YAML fix.

Definition of done (checklist)

  • Contract test remote_integration_nightly_workflow committed and passing.
  • Workflow contains step Build server (required by integration tests) → npm run build:server between Install dependencies and Run remote critical suite (merge fix(ci): build server before remote nightly tests #2478 or identical diff).
  • Red control: after npm ci with dist/ absent, npm run test:remote:critical fails in global setup with [neotoma] dist/ is missing... before any remote assertion runs (reproduce via removing build step or deleting dist/).
  • Green control: npm run build:server then npm run test:remote:critical passes global setup and reaches real remote assertions (may still fail on Remote critical MCP resource tests do not establish an authenticated user #2479 fixture/auth — out of scope; failure must not be missing-dist).
  • actionlint .github/workflows/remote_integration_nightly.yml clean.
  • Ordinary CI (type-check / lint / unit+contract) green; no production runtime, auth, or fail-closed global-setup behavior changed.
  • PR body records red/green control results and cites Design basis docs/foundation/principles.md#extend-the-mechanism-that-already-generalizes.

Out of scope for this QA DoD

qa gate handoff. Sign off gate_status.qa only when the contract test is in the PR and ordinary CI is green on it (or #2478 equivalent). Leave pending if the PR ships YAML alone without the contract test.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingincidentlanius-triageApplied by Lanius triage workflow

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions