WIP: discover kernel-source roots at runtime, and replace the two LLM source tiers with one agent review

.env.template

@@ -226,10 +226,11 @@ KNOWLEDGE_STORE_MODE=local
 # applies to dense TunableOp recording, while block-FP8 uses standard Roofline.
 # HYPERLOOM_GEMM_SHAPE_CAPTURE=1
 # HYPERLOOM_GEMM_SHAPE_CAPTURE_TIMEOUT_SEC=1800
-# Force the per-optimization wall-clock budget in minutes (default 60). The env
+# Force the per-optimization wall-clock budget in minutes (default 90). The env
 # wins over the payload value, which is LLM-authored, so an operator raising the
-# budget is not silently overridden.
-# KERNEL_OPT_BACKEND_BUDGET_MIN=60
+# budget is not silently overridden. forge-loop reserves half the window for
+# finalize, so this buys roughly half as much iteration as it reads.
+# KERNEL_OPT_BACKEND_BUDGET_MIN=90
 
 # --- Collective optimization lane (optional) -------------------------------
 # The Coordinator drives this lane itself at KERNEL entry; it is never an agent

docs/conceptual/kernel-execution-path.md

@@ -176,7 +176,7 @@ Optional:
 | `TRACELENS_INTERNAL_ROOT` | TraceLens internal extension; unset = open-source-only |
 | `KERNEL_OPT_MAX_PARALLEL` | Override the 8-concurrent-kernel default |
 | `INFERENCE_OPTIMIZER_KERNEL_OPT_MAX_PARTIAL` | Override partial-attempt retry cap (default 2) |
-| `KERNEL_OPT_BACKEND_BUDGET_MIN` | Force the per-optimization wall-clock budget in minutes (default 60); wins over the LLM-authored payload value |
+| `KERNEL_OPT_BACKEND_BUDGET_MIN` | Force the per-optimization wall-clock budget in minutes (default 90); wins over the LLM-authored payload value |
 
 Collective lane:
 

docs/reference/environment-variables.md

@@ -188,7 +188,7 @@ The following variables control the kernel optimization backend ladder.
 | `HYPERLOOM_GEMM_SHAPE_CAPTURE` | `1`                           | Enables automatic runtime GEMM-shape capture for eligible single-node dense vLLM Forge tuning when no explicit shape input is available. Block-FP8 first reuses shapes from the TraceLens-selected steady-state trace of a successful Roofline with exactly matching model, workload, server arguments, environment, and backend controls. Missing or stale evidence triggers the same standard Roofline/ProfileExecutor/TraceLens steady-state pipeline as a fallback. Set to `0` to preserve the no-capture path. |
 | `HYPERLOOM_GEMM_SHAPE_CAPTURE_TIMEOUT_SEC` | `1800`          | Timeout in seconds for the dense vLLM TunableOp recording benchmark. Block-FP8 fallback uses the standard Roofline/ProfileExecutor timeout. Values below `60` are clamped to `60`. |
 | `INFERENCE_OPTIMIZER`<br>`_KERNEL_OPT_MAX_PARTIAL` | Unset           | Cap on how many `PARTIAL` kernel-opt verdicts an action can yield before it short-circuits to `NEEDS_REVIEW`. Useful for keeping budget contained when GEAK is consistently timing out.            |
-| `KERNEL_OPT_BACKEND_BUDGET_MIN` | `60`                         | Wall-clock budget in minutes for one optimization, mirrored by the `kernel_optimization.py` wrapper. The env deliberately wins over the payload `budget_minutes`, which is LLM-authored from a prompt template, so an operator raising the budget is not silently overridden. forge-loop reserves half the window for finalize, so `60` leaves roughly 30 minutes of real iteration. |
+| `KERNEL_OPT_BACKEND_BUDGET_MIN` | `90`                         | Wall-clock budget in minutes for one optimization, mirrored by the `kernel_optimization.py` wrapper. The env deliberately wins over the payload `budget_minutes`, which is LLM-authored from a prompt template, so an operator raising the budget is not silently overridden. forge-loop reserves half the window for finalize, so `90` leaves roughly 45 minutes of real iteration. |
 | `AITER_LOG_TUNED_CONFIG`       | `1` (set for every serving run) | Makes aiter log each tuned-config lookup it *hits*, not only the ones it misses. Two checks have no input without it: the GEMM demand list, which learns the shapes the runtime actually asks for (config-derived shapes covered 0.4% of them), and the apply verdict, which cannot tell "the tuned table was never read" from "it was read and did not help". A scan of 60 production logs found it set in none of them, so it is now injected by default. An operator value wins — set `0` to turn hit logging off, at the cost of both checks going inconclusive. Every miss already prints a line regardless of this setting; hit logging adds roughly one line per lookup that succeeds. |
 | `HYPERLOOM_GEMM_PAIRED_PAIRS`  | `0` (off)                     | How many interleaved baseline/tuned pairs to re-measure before a GEMM tuning KEEP is reported as confirmed. One end-to-end measurement cannot separate a gain from drift on this fleet: three rounds of a single unchanged configuration spanned 58%, and one controlled repeat moved 16%. Each pair costs two extra benchmark rounds. When `0`, the gain is still promoted — it is the best number available — but recorded as an unpaired block comparison rather than presented as a paired one. |
 

src/hyperloom/agents/kernel/tests/test_bypass_report.py

@@ -663,10 +663,11 @@ def _boom(*a, **k):  # pragma: no cover - must not be called
     assert cand["shape_provenance"] == "torch_trace"
 
 
-def test_routable_candidate_carries_shapes_for_orchestrator_gate():
-    # The orchestrator shape gate reads candidate["shapes"] and rejects dispatch
-    # with "empty_kernel_shape" when empty, so a routable candidate with real
-    # trace-captured dims must expose a non-empty "shapes" list.
+def test_routable_candidate_carries_shapes_for_dispatch():
+    # Dispatch reads candidate["shapes"] to pin the harness to the serving dims,
+    # so a routable candidate whose trace DID record them must expose them in
+    # the downstream contract form rather than leaving the backend to recover
+    # dims that were measured all along.
     kernels = [
         {
             "name": "triton_silu",
@@ -682,7 +683,7 @@ def test_routable_candidate_carries_shapes_for_orchestrator_gate():
     cand = report.build_candidates(_analyze(kernels), framework="vllm", target_platform="MI300X")["hot_kernels"][0]
     shapes = cand.get("shapes")
     assert isinstance(shapes, list) and shapes, (
-        "routable candidate must expose a non-empty 'shapes' for the orchestrator gate"
+        "a routable candidate with trace-captured dims must expose a non-empty 'shapes'"
     )
     assert cand["shape_provenance"] in {"torch_trace", "tuning_csv"}
     # shapes mirrors input_shapes in the harness-consumable contract form.
@@ -699,15 +700,15 @@ def test_trace_shape_entries_contract_format():
     assert out == [{"call_num": 5, "shape": "(4,1024) bf16<br>(1024,) fp32"}]
     # unmapped dtype -> bare shape (no suffix).
     assert report._trace_shape_entries([[8, 8]], ["weird"], 1) == [{"call_num": 1, "shape": "(8,8)"}]
-    # no renderable operand -> empty (gate will reject as empty_kernel_shape).
+    # no renderable operand -> empty, which the backend recovers dims for.
     assert report._trace_shape_entries([[]], ["float"], 1) == []
     assert report._trace_shape_entries([], [], 1) == []
 
 
 def test_unresolved_shape_candidate_has_empty_shapes():
     # A kernel with no captured dims stays shape-less: "shapes" is an empty list
-    # (present, not absent) and the gate will correctly reject it as
-    # empty_kernel_shape.
+    # (present, not absent) and the provenance says why, so the dispatch can
+    # tell "no dims were recorded" from "these are the dims".
     kernels = [{"name": "mystery_kernel", "op_name": "aten::mystery", "gpu_time_us": 100.0, "count": 1}]
     cand = report.build_candidates(_analyze(kernels), framework="vllm", target_platform="MI300X")["hot_kernels"][0]
     assert cand.get("shapes") == []