fix(nevermore-cli): tag only the script template as the batch test entry - #782
Open
Quenty wants to merge 1 commit into
Open
fix(nevermore-cli): tag only the script template as the batch test entry#782Quenty wants to merge 1 commit into
Quenty wants to merge 1 commit into
Conversation
combine-test-places tagged every Script under ServerScriptService with `_BatchTest_<slug>`, and batch-test-runner executes `GetTagged(...)[1]`. With one script in a project those are the same script; with two, the runner loadstrings whichever the tag listed first and the package's real test bootstrap never runs. Nothing catches it downstream either -- a script that returns without erroring reports as a pass, so the batch reports 0ms and no jest output for a suite that was never asked to run. The `scriptPath` each triple already carries -- the target's `scriptTemplate`, and what the single-package path executes -- is now injected as a Script under ReplicatedStorage._BatchScripts and is the only thing tagged, which is what the file's own header said it did. The runner refuses an ambiguous tag rather than picking from it, so a project that reintroduces one gets told. Verified against a game with two server scripts: FAILED (0ms, no jest report) before, 645/707 passed after. Claude-Session: https://claude.ai/code/session_01Ei9RRAgYwL3MGtMkr6vyKE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
batch testcan run the wrong script and report it as a pass. A game in my orghas had zero test coverage in CI since the moment it added a second server
script — the batch ran that script instead of the test bootstrap, and CI's only
complaint was
0msandno jest report in output.Why
combine-test-places.luautags everyScriptunderServerScriptService:and
batch-test-runner.luauloadstringsCollectionService:GetTagged(...)[1].With one script in the project those are the same script. With two, the runner
takes whichever the tag lists first — and since a script that returns without
erroring looks exactly like a passing test bootstrap, the marker says PASS and
the only symptom is an empty section.
Meanwhile
scriptPath— the target'sscriptTemplate, the thing thesingle-package path actually executes — is passed all the way down to the lune
script and then ignored. The header comment says it becomes a Script under
ReplicatedStorage._BatchScripts; the code never did that.The change
combine-test-places.luauinjects thescriptTemplatefile as a Script underReplicatedStorage._BatchScriptsand tags only that, for every packagerather than only the first. It exits with a clear message if the file is
missing, instead of silently producing a place with nothing to run. A Script
under
ReplicatedStorageis inert, so it is a source carrier and nothing else.batch-test-runner.luaurefuses a tag that resolves to more than one instanceand names them, rather than picking one. That is the state that used to be
silent.
This also makes batch agree with the single-package path about what "the test
script" is, rather than agreeing by coincidence of project shape.
Verification
Against a real game whose
src/scripts/Serverholds two scripts, using a localbuild of the CLI:
FAILED (0ms)— 7 log messages, none of them jestPassed (645/707)in 1m21sThe 645/707 matches what that repo's CI reported the day before it added the
second script, so the fix restores exactly the suite that went missing.
npm testintools/nevermore-cli: 121 passed. No unit test added — theregression lives in a lune script that needs a real
.rbxlto exercise, and thetwo guards this adds (a hard exit at combine time, a refusal at run time) are
what would have caught it.
https://claude.ai/code/session_01Ei9RRAgYwL3MGtMkr6vyKE