Commit 5b2f43d
committed
fix(engine): fourteen defects found by running v0.21.0 on a live 30B, plus frontend assemblers
Everything here was found by driving the harness against a real local stack —
oMLX serving Qwen3-Coder-30B — rather than by reading the code. Each fix names
the measurement that produced it, and each carries a regression test that fails
without it.
── The gates were not gating ─────────────────────────────────────────────────
Executable acceptance criteria were discarded before they could run.
collapseToWorker rebuilt the surviving task without Criteria, so whenever the
splitter's tasks collapsed — the common shape on a small model — every criterion
the model authored was dropped. Measured: the 30B emitted correct criteria with
`go test ./...` as the verify command, and the run reached the reviewer with
nothing to check, indistinguishable from a task that passed.
A green test command that ran no tests counted as verification. classifySmoke
checked sr.OK before the no-tests check, and `go test ./...` on a tree with no
_test.go files exits ZERO — so the gate's own documented contract ("a 'nothing
to run' exit is NoTests, never Green") held only for runners that fail on empty.
Measured: a run told "both parts must have tests" wrote none and finished ✔ with
qa_gate green.
A forged criteria header could switch the criteria gate off. A worker that
merely echoed "## Acceptance criteria" suppressed the review-time gate, and with
nothing run CriteriaUnverifiedInOutput stayed false — the value that ALLOWS the
reviewer fast path. The section's provenance stamp is now required.
Criteria that could never run. Told only in the abstract to write a runnable
verify command, the 30B wrote `go test -v ./... | grep -E '(TestX|PASS|FAIL)'`;
the sanitizer refuses shell metacharacters, so every criterion on that board came
back UNVERIFIED. The splitter now gets the concrete ✓/✗ example, and its own
prompt budget — it is paid once per split, as the composer is paid once per run.
── The board could deadlock, and correction aimed at the wrong files ─────────
runWave is synchronous, so nothing is in flight at the top of the scheduler
loop — yet a task left in in_progress/in_review still made AgentWorkRemaining
report work in progress, and no path re-dispatches those columns. Measured: one
stranded task, four dependents frozen behind it, ~9 minutes of correct work
discarded and reported as a failure with the edit already on disk and compiling.
Orphans are re-queued, bounded by the existing attempt ceiling.
When the QA gate stayed red, the synthesized verdict carried the literal string
"qa_gate red" — and the corrective rewrite scopes its fix by mining file paths
out of those failure strings, so a verdict naming no file fell back to "whatever
finished most recently". Measured: three correction rounds scoped to pkg/tasks
while every compiler error was in cmd/server/main.go, which none of them was
allowed to touch.
A dev server was accepted as a verification. `npm run` is whitelisted because
`npm run lint` and `npm run typecheck` are exactly what the gate should run;
`npm run dev` shares that prefix and never returns. Measured: a splitter wrote
"npm run dev passes" as a tester task's acceptance, which is not a proposition
that can pass. Refusing the non-terminating scripts is what makes it safe to
admit `npm run`/`yarn run`/`pnpm run` as verify commands at all — so "the
project builds" can now be proven instead of reported UNVERIFIED.
── Routing was configured, displayed, and then not used ─────────────────────
runner.HasRole was wired only inside the escalation branch, so on any install
without a model_escalation ladder — nearly all of them — every feature that asks
"is this agent registered?" answered no.
normalizeExecRole applied execute.default_role only to an EMPTY role, but
Task.Normalize fills empty roles with "worker" first, so the chosen language
specialist was displayed in the composition and never used. Tasks now route to
the specialist that owns their files, which is also what lets a mixed Go + React
board run on two specialists at once.
doctor printed "agents inherit stack/global LLM" unconditionally — false as soon
as model_roles pins a role — and validated nothing, so a typo'd model passed at
100/100 and failed mid-run at the reviewer. Both routing keys are now shown,
every model they name is checked against what the endpoint serves, and an
unserved one is a readiness finding (measured: 100 → 86).
`slmcode agent list` showed 20 of 59 agents: block-defined agents were
registered by the orchestrator and invisible to the command that lists them.
── Greenfield work could not start, and duplicated itself when it did ───────
Greenfield Go and web paths were dropped: isGreenfieldCreatePath accepted
main.py but not main.go, and knew no cmd/, pkg/ or web/. Measured: a full-stack
request finished 0 of 7 tasks with nothing on disk, because every planned path
was discarded and a worker forbidden to invent paths had nothing it was allowed
to write. Fixing it made the same request produce a compiling Go backend and a
React frontend. The fix itself is upstream's — see the rebase note below; what
remains here are the tests that pin the behaviour.
Two prompt few-shot examples were being copied as real work: the splitter
planned "add Sum to calc.go" verbatim from its own example on a greenfield
request, and the composer's handoff carried "verify with go test ./..." into a
React run for the same reason. Both are now shapes, not content.
Duplicate workers on one file each cost a full wave and opened a file the
previous one had rewritten. Same-file worker tasks now merge — on the primary
target or the whole set — with dependencies rewritten onto the survivor.
Measured: one query went from six tasks to three.
Workers wrote against types they had never read. Rule 1 said "ws_read a file
before editing it" — nothing about USING one. Measured: a worker scoped to
cmd/server/main.go used task.Title/CreatedAt/UpdatedAt on a Task a sibling task
had defined with three other fields.
── Frontend assemblers ──────────────────────────────────────────────────────
Two packs that build React UI by INSTALLING components with the library's own
CLI and wiring them up, instead of writing them from scratch: shadcn/ui and
Untitled UI. Hand-writing a dialog with focus traps is where a 7-32B model
spends its runway; imports, props and layout is what it is good at.
Both ship enabled — nothing to install, nothing to apply. The method is chosen
from evidence and announced: the request wins first (name a library, or say
"from scratch"), then the project's own markers, then greenfield defaults to
assembling; an existing app with no markers keeps writing by hand.
Each pack carries an assembly reviewer that rejects a component hand-rolled when
the registry already had it, and per-library gates. Both CLIs were driven for
real to write the agents: shadcn's `init` needs an explicit -b or it stops on an
interactive menu even with -y, and Untitled UI matches names fuzzily and wrongly
(`buttons` installs app-store-buttons, and it does not error).
Shell access is scoped rather than opened. npx stays an executor; five
subcommands of two named packages are allowed, matched structurally so
`npx shadcn add`, `npx --yes shadcn@latest add` and the legacy shadcn-ui name all
work. An `add` naming a URL, an @registry or a local path is refused — both CLIs
accept those where a component name goes, and they resolve to a registry nobody
reviewed. Also fixed: `npx tsc --noEmit` was refused (the packs' own typecheck
and qa_gate — those tools live in node_modules, never on PATH), and a bare
`npx tsc` prefix also admitted `npx tsc-evil`, since matching is HasPrefix.
── Verification ─────────────────────────────────────────────────────────────
Local gate green on the frozen tree: gofmt, vet, golangci-lint (0 issues), the
full unit suite, -race, go mod tidy, the Studio UI (lint/typecheck/test/build),
govulncheck (no vulnerabilities), and coverage at 67.6% against a 63.0% floor.
Live end-to-end against oMLX + Qwen3-Coder-30B: a Go bug fix with criteria
verified 3 passed / 0 failed / 0 unverified, worktree isolation leaving the
origin untouched and committing no harness state, per-role model routing, and
the shadcn assembler installing a component with the CLI and wiring it into a
page that typechecks.
── Rebased onto 32 upstream commits ─────────────────────────────────────────
Three conflicts, resolved in upstream's favour where it had built the better
mechanism:
- Greenfield paths. Upstream added a state-aware rule — isGreenfieldRoot plus
looksLikeSourceTarget — that is permissive in a genuinely empty repository and
conservative once there is code in it. That is strictly better than the
unconditional allowlist here, which would have blessed a claimed-but-absent
cmd/server/main.go in an ESTABLISHED repo. The additions to
isGreenfieldCreatePath are dropped; the tests that cover the behaviour stay
and pass against upstream's rule.
- Per-task specialist routing. Upstream staffs each task at board level
(routeBoardToSpecialists), which sets task.Role. specializeExecRole only ever
replaces a GENERIC role, so it defers to that and fills in where board-level
staffing did not run — which is also what the frontend assembler hooks into.
- The QA-gate verdict. Upstream's integration-failure branch and the real
failure payload are both kept: the guard decides WHETHER to re-enter the
tester path, and the payload decides what it is told when it does.
Gate green on the rebased tree: lint, the full unit suite, -race, and coverage
at 69.0% against a 63.0% floor.1 parent 0f66e88 commit 5b2f43d
61 files changed
Lines changed: 4166 additions & 75 deletions
File tree
- cmd/slmcode
- docs
- pkg
- agents
- blocks
- bundled
- agents
- packs
- pipelines
- quality
- config
- loop
- orchestrator
- plan
- quality
- readiness
- sandbox
- workspace
- test/e2e
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
| 52 | + | |
52 | 53 | | |
53 | 54 | | |
54 | 55 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
| |||
150 | 151 | | |
151 | 152 | | |
152 | 153 | | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
153 | 162 | | |
154 | 163 | | |
155 | 164 | | |
| |||
187 | 196 | | |
188 | 197 | | |
189 | 198 | | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| 11 | + | |
11 | 12 | | |
12 | 13 | | |
13 | 14 | | |
| |||
494 | 495 | | |
495 | 496 | | |
496 | 497 | | |
497 | | - | |
| 498 | + | |
498 | 499 | | |
499 | 500 | | |
| 501 | + | |
500 | 502 | | |
501 | 503 | | |
502 | 504 | | |
| |||
832 | 834 | | |
833 | 835 | | |
834 | 836 | | |
| 837 | + | |
| 838 | + | |
| 839 | + | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
| 843 | + | |
| 844 | + | |
| 845 | + | |
| 846 | + | |
| 847 | + | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
| 851 | + | |
| 852 | + | |
| 853 | + | |
| 854 | + | |
| 855 | + | |
| 856 | + | |
| 857 | + | |
| 858 | + | |
| 859 | + | |
| 860 | + | |
| 861 | + | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
| 867 | + | |
| 868 | + | |
| 869 | + | |
| 870 | + | |
| 871 | + | |
| 872 | + | |
| 873 | + | |
| 874 | + | |
| 875 | + | |
| 876 | + | |
| 877 | + | |
| 878 | + | |
| 879 | + | |
| 880 | + | |
| 881 | + | |
| 882 | + | |
| 883 | + | |
| 884 | + | |
| 885 | + | |
| 886 | + | |
| 887 | + | |
| 888 | + | |
| 889 | + | |
| 890 | + | |
| 891 | + | |
| 892 | + | |
| 893 | + | |
| 894 | + | |
| 895 | + | |
| 896 | + | |
| 897 | + | |
| 898 | + | |
| 899 | + | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
60 | 60 | | |
61 | 61 | | |
62 | 62 | | |
63 | | - | |
| 63 | + | |
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
| |||
79 | 79 | | |
80 | 80 | | |
81 | 81 | | |
| 82 | + | |
| 83 | + | |
82 | 84 | | |
83 | 85 | | |
84 | 86 | | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
85 | 94 | | |
86 | 95 | | |
87 | 96 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
592 | 592 | | |
593 | 593 | | |
594 | 594 | | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
| 734 | + | |
| 735 | + | |
595 | 736 | | |
596 | 737 | | |
597 | 738 | | |
| |||
0 commit comments