Learning proposal: (498f33dd)

[fro-bot]

Source PR: merge commit 498f33dd6b59d2dae0aa0fc5b135f81d9158caa4 (3 substantive review rounds, ending in a CONDITIONAL verdict with a blocking issue).

The accepted half first: the change replaced a shape-based scope criterion with a materiality-based one. "Has classification logic plus a main() that sets exitCode" is a silhouette test, and silhouette tests draw the wrong set — they admit modules that merely look like guards and exclude ones that are. Re-deriving inclusion from whether a module's mutants are material to the guarantee under test was the right correction, and the build-wiki-write-core.ts carve-out ("self-consistently green") was singled out as the sharpest reasoning in the diff.

The blocking issue is the durable lesson. Demoting private-leak-adapter.ts from mutate was defensible; removing private-leak-adapter.test.ts from testFiles was a separate lever pulled by reflex, and it stripped a still-mutated module of its only real source-tree exerciser. The chain the reviewer traced: private-leak.ts remains in mutate as a Tier 0 module; check-private-leak.ts imports checkPrivateLeak from the package's dist specifier, so check-private-leak.test.ts exercises dist/private-leak.js and never touches the instrumented source; private-leak-adapter.ts imported from ./private-leak.ts — a same-tree source import — and was therefore the live path. The plan document itself recorded 47 killed mutants attributable to that adapter test. What survived the cut was a single it() in an unrelated test file.

That is a general trap. In mutation configs, mutate and testFiles look coupled but are independent levers, and the import specifier decides whether instrumentation is exercised at all. A test that reaches a module through its published dist entry point is measuring the compiled shell, not the mutated source — it reports green while measuring nothing.

Proposed learning: mutate and testFiles are independent levers, and dist-vs-source specifiers determine what instrumentation actually sees. Before removing any entry from testFiles, enumerate every module still in mutate and confirm each retains at least one test that reaches it through a source-tree import. Tests that import through a package's dist or published specifier exercise the compiled artifact and contribute zero mutant kills, so they cannot be counted as coverage for a mutated module. Adopt materiality (does this mutant threaten the guarantee?) rather than module shape as the scoping criterion, and record for each mutated module which specific test file is its live source-path exerciser, so a future demotion cannot quietly orphan it.

[marcusrbrown]

Documented in docs/solutions/best-practices/a-mutation-score-can-measure-nothing-2026-09-08.md (#3868), merged with #3849.

The two are one learning by two mechanisms: a module-level const frozen per worker, and a test importing through compiled dist rather than instrumented source. Both leave a module in mutate reporting a number while measuring nothing, and both answer the same question — why is my mutation signal wrong. Splitting them would have produced two docs competing for one retrieval query.