| myst |
|
|---|
User-configurable environment variables for Hyperloom, grouped by purpose. Runtime parameters such as framework, tensor parallelism, prompt lengths, and phase toggles are configured with CLI flags; internal subprocess handoff envs are intentionally not listed as user configuration.
Variables marked Required must be set (using shell or $REPO_ROOT/.env)
or the CLI will exit fast at startup. Variables marked Optional have
sensible defaults; the default is shown in the Default column.
Precedence rule (applies everywhere): shell-exported env wins over .env.
See Hyperloom authentication and credentials.
These variables configure LLM gateway access and optional backend credentials.
| Variable | Required | Default | Description |
|---|---|---|---|
ANTHROPIC_BASE_URL |
Conditional | — | Anthropic-side endpoint. Required together with ANTHROPIC_API_KEY to enable Claude. |
ANTHROPIC_API_KEY |
Conditional | — | Anthropic-side key. Pairs with ANTHROPIC_BASE_URL. |
ANTHROPIC_AUTH_TOKEN |
No | — | Claude CLI auth token alias, accepted in place of ANTHROPIC_API_KEY. Preflight never fills it; the Ray / e2e / forge-fusion env builders default it from the Anthropic-side key when they hand credentials to a subprocess. |
ANTHROPIC_CUSTOM_HEADERS |
No | — | Extra request headers for the Anthropic side, for gateways that authenticate on a header of their own (for example Azure API Management). Newline-delimited Name: value as in the Anthropic SDK; a JSON object is accepted too. ${VAR} references are expanded from the same environment, so a gateway header can reuse ANTHROPIC_API_KEY instead of duplicating the secret. |
CLAUDE_CODE_OAUTH_TOKEN |
No | — | Claude Max/Pro subscription token from claude setup-token. Lowest-priority Anthropic credential: either API-key variable outranks it. On its own it implies https://api.anthropic.com. Passed to subprocesses verbatim and never copied into ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN, or ~/.claude/config.json, which would switch the run to API-credits billing. |
GEAK_API_KEY |
No | — | Internal alias, never derived from either side. GEAK runs on the Anthropic side (ANTHROPIC_* + GEAK_CLAUDE_MODEL); set this only to point GEAK elsewhere. |
GEAK_BASE_URL |
No | — | Internal alias, never derived from either side. Set it only to point GEAK at a different endpoint than the Anthropic side. |
GEAK_CLAUDE_MODEL |
No | Inherits CLAUDE_MODEL |
GEAKv4 Claude Code workflow model id. |
FORGE_CLAUDE_MODEL |
No | Inherits CLAUDE_MODEL |
Forge Claude backend model id (fusion, rewrite, collective). Set when Forge should use a different Claude model than orchestration. |
FORGE_CODEX_MODEL |
No | Inherits CODEX_MODEL |
Forge Codex backend model id (fusion, rewrite, collective). Set when Forge should use a different Codex model than the OpenAI-side default. |
LANGFUSE_HOST |
No (required only when HYPERLOOM_LANGFUSE_ENABLE=1) |
Unset | Base URL of your Langfuse deployment (for example, https://langfuse.<your-domain>). Used by both the live trace push and the offline backfill_langfuse CLI. |
LANGFUSE_PUBLIC_KEY |
No (required only when HYPERLOOM_LANGFUSE_ENABLE=1) |
Unset | Langfuse project public key (pk-...). |
LANGFUSE_SECRET_KEY |
No (required only when HYPERLOOM_LANGFUSE_ENABLE=1) |
Unset | Langfuse project secret key (sk-...). |
The following variables configure filesystem paths for Hyperloom's runtime dependencies and session data.
| Variable | Required | Default | Description |
|---|---|---|---|
REPO_ROOT |
No (recommended) | $(pwd) |
This Hyperloom checkout. Used to locate .env, skills, scripts. Falls back to the current working directory when unset. |
INFERENCEX_PATH |
Conditional | Auto-cloned by install.sh |
Path to the SemiAnalysisAI/InferenceX repo, used by baseline / target analysis. install.sh clones it when unset; only required if that auto-clone fails. |
TRACELENS_ROOT |
No (installer auto-clones) | ${HYPERLOOM_CACHE_DIR:-$REPO_ROOT/.cache}/TraceLens@<resolved-sha> (auto-clone of AMD-AGI/TraceLens pinned to a fixed SHA) |
src/hyperloom/agents/kernel/scripts/install.sh clones the public repo into the repo-local cache root when unset. Export it to opt into a pre-existing checkout you maintain — that is an explicit operator override and skips both the clone and the SHA pin. |
GEAK_CLAUDE_BIN |
No (installer auto-resolves) | First of $HOME/.local/bin/claude, /usr/local/bin/claude, $(command -v claude); written to kernel-agent.env.sh |
Pins the Claude Code binary the GEAK SDK path uses, so claude_agent_sdk doesn't fall back to its older bundled CLI. Export to force a specific build. |
USER_DATA_PATH |
No | /workspace/hyperloom |
Session directory root (logs, runs, mirrors, breakdown). Replaces the retired INFERENCE_OPTIMIZER_SESSION_DIR and WORKSPACE_PATH. |
HYPERLOOM_RUNTIME_DIR |
No | $USER_DATA_PATH/runtime (installer) |
Private writable runtime state. Codex SDK turns create a unique mode-0700 CODEX_HOME here and remove it after the SDK client closes. When unset, Codex uses the first safe declared output root, then a run-local working directory; it never falls back to /tmp or a source checkout. |
INFERENCE_OPTIMIZER_CURRENT_SESSION_DIR |
No (set by CLI) | Set at session boot | Absolute path to the active session directory. Written by the CLI when a session starts and inherited by every benchmark subprocess; session-path resolution prefers it over scanning USER_DATA_PATH. Do not set by hand. |
HYPERLOOM_ROOT |
No | $HYPERLOOM_RUNTIME_DIR/source-mirrors |
Legacy source-mirror root kept for compatibility. Current open-source dependency checkouts default to the repo-local cache root (${HYPERLOOM_CACHE_DIR:-$REPO_ROOT/.cache}), not this path. |
HYPERLOOM_CACHE_DIR |
No | $REPO_ROOT/.cache |
Writable, repo-local base for auto-cloned open-source deps (TraceLens, Magpie, etc.), cloned per revision as <name>@<sha>. Not under $TMPDIR so a reaper cannot wipe it mid-run. |
MAGPIE_PATH |
No | Resolved from installed Magpie package unless explicitly set |
Magpie package root for benchmark wrappers and patch inspection. |
FORGE_PATH |
Conditional | Unset | KernelForge checkout root, and the single canonical variable for it. Required whenever the forge kernel backend is enabled (KERNEL_OPT_BACKEND_ORDER=forge): forge_submit.py prepends it to sys.path to import kernel_agents, and resolves the vendor-playbook task bundles beneath it. Unset with kernel_agents already installed still imports, but the playbook bundles are then unresolvable. |
INFERENCE_OPTIMIZER_MODEL_PATH_ROOTS |
No | Built-in model roots such as /models and /shared_nfs |
os.pathsep-separated allowlist for absolute model paths restored from state.json during a resume. HuggingFace-style repo IDs remain allowed. Set this when production models live outside the built-in roots. |
SESSION_DIR |
No (robustness-agent) | Scan known paths | Path containing storage/coordinator.db; the robustness FindingSink writes under {session_dir}/agents/robustness/findings/{session_id}.jsonl. |
WORKSPACE_PATH (legacy) |
No | Unset | Legacy path variable. Still consumed in two narrow spots: the CLI setdefaults it to the repo root for the critic subprocess's static assets, and TraceLens uses it as a USER_DATA_PATH fallback. Prefer USER_DATA_PATH. See Upgrade Hyperloom version. |
INFERENCE_OPTIMIZER_SESSION_DIR (deprecated) |
No | Unset | Retired — replaced by USER_DATA_PATH. No longer read. |
Set with CLI flags, not env vars. Pre-set ISL / OSL / CONC / PRECISION /
TP / EP env vars are ignored and overwritten (GPU_TYPE is a fallback when
--gpu-type is omitted).
- Model / workload shape:
--model,--model-class,--framework,--framework-version,--precision,--tp,--ep,--isl,--osl,--conc,--max-model-len,--profile-osl. - Goal / budget:
--target-gain,--max-hours,--target-summary,--target-tput,--compare-against-gpu. - Cluster topology & multi-node backend:
--nodes,--gpus-per-node,--gpu-type,--mn-backend(rayjob/infera),--server-args(rayjob). Per-pod sizing, the pod image and pod-side env are the provisioning platform's inputs, notoptimizeflags — the cluster already exists by the time the optimizer runs. - PD disaggregation (infera):
--pd-mode disaggregated,--pd-prefill-nodes/--pd-prefill-tp/--pd-prefill-ep/--pd-prefill-extra-args,--pd-decode-nodes/--pd-decode-tp/--pd-decode-ep/--pd-decode-extra-args,--pd-transfer-backend,--pd-ib-device. - Phase toggles:
--enable-roofline/--no-enable-roofline,--enable-conc-sweep/--no-enable-conc-sweep,--conc-sweep-concs,--no-framework-agent,--no-framework-local-explore,--no-kernel,--no-explore,--no-eval. - Agent models:
--claude-model,--codex-model. - Session / resume:
--resume-from,--force-resume,--reset-state. - Quantization:
--quantize,--quantize-scheme.
Run inference_optimizer optimize --help for the exhaustive flag list.
A candidate that clears the throughput bar must also hold accuracy before it is kept. Grading runs only after the throughput bar is cleared, and reads the score back from the run's own eval output, so a gate never costs an extra eval and a regressing candidate never spends a verdict on itself.
In every lane a measured drop beyond the tolerance is a REVERT. A missing
verdict while a positive baseline accuracy is on record drops to
NEEDS_REVIEW — eval should have worked and didn't. No baseline accuracy at
all degrades to a throughput-only KEEP rather than blocking every candidate,
so eval-less environments still make progress. Pass --no-eval to turn the
eval off for the whole run: the baseline anchors on throughput instead of
halting on a missing accuracy reference, and every candidate then lands on
that degraded path.
| Variable | Default | Description |
|---|---|---|
RUN_EVAL |
true |
Whether a serving benchmark runs the GSM8K eval. Turning it off removes the per-candidate accuracy signal entirely — accuracy regressions stop being caught. Ignored by scriptable workloads, whose correctness signal is the quality_gate in benchmark_report.json. |
HYPERLOOM_QUALITY_REFHYPERLOOM_QUALITY_REF_WRITE |
Derived under the session dir | The scriptable quality gate's reference artifact: _WRITE establishes it on the baseline, the other compares against it on every later candidate. What the artifact holds is the workload's own business — xDiT stores an image, an operator-supplied custom workload stores whatever its script compares. Also emitted as XDIT_QUALITY_REF / XDIT_QUALITY_REF_WRITE for bench scripts written before the rename; either name is read, both are written. |
INFERENCE_OPTIMIZER_REQUIRE_KERNEL_ACCURACY |
On | Gates the KEEP for a kernel patch integrated by the kernel lane. Set to 0 / false / no / off to fall back to a throughput-only KEEP. Disable only when the eval lane is known-broken: this gate is what stops a faster-but-wrong kernel from being kept. |
INFERENCE_OPTIMIZER_REQUIRE_FRAMEWORK_ACCURACY |
On | Same gate for a framework source patch authored by a specialist. Same disable spellings. |
MAGPIE_EVAL_LIMIT |
Unset (full task set) | Caps the number of eval problems (lm_eval --limit). Useful for smoke runs; see the noise caveat below before using it on a run whose KEEP decisions matter. |
The tolerance is deliberately not an env knob: ACCURACY_THRESHOLD in
src/hyperloom/orchestrator/actions/executors/_accuracy_gate.py is a fixed
0.05, i.e. a candidate must stay within 5 percentage points of the recorded
baseline accuracy.
Note that the score is measured once per candidate, not averaged over repeats.
On a full GSM8K run (1319 problems) the 5-point tolerance sits several standard
errors away from the baseline, so single-run noise does not trip it. Capping the
eval with a small MAGPIE_EVAL_LIMIT shrinks that margin sharply and can make
the gate noise-sensitive — prefer the full task set whenever a gate decision
depends on the result.
InferenceX runs lm-eval with max_tokens=min(16384, ctx-4096), so a sample that
does not converge spends that entire budget, and 1319 of them can consume the
whole baseline timeout. Every generation request is therefore capped, and the
terminators the model declares are supplied with it — lm-eval carries a single
eos_string and its concurrent request path does not send even that one, so a
model like Qwen3, which declares eos_token_id [151645, 151643], would
otherwise run with no end-of-turn stop condition at all.
Both are applied inside the eval process rather than passed in, which is what keeps them equal across the baseline and candidate arms. That symmetry is the whole point: the gate compares a difference of two scores, so a bound or a terminator that reaches only one arm biases the verdict instead of merely limiting it. Prefer leaving these alone; if you do change one, change it for the whole session rather than a single round.
Each run reports what it applied, to stderr as HYPERLOOM_EVAL_BOUNDS_SUMMARY
and to hyperloom_eval_bounds.json in the result dir, including how many
generations hit the ceiling. Check truncated there before concluding a score
is low for any other reason.
| Variable | Default | Description |
|---|---|---|
HYPERLOOM_EVAL_MAX_TOKENS |
4096 |
Per-request generation ceiling. Never raises a lower ceiling a task already asked for. 0 disables the cap and restores the full upstream budget — a degenerate model then costs the whole timeout again. An unparseable value falls back to the default rather than to "unbounded". |
HYPERLOOM_EVAL_DERIVE_STOP |
On | Whether to read the model's generation_config.json / tokenizer_config.json for its terminators. Resolution is cache-only and never downloads, so an uncached repo id simply derives nothing. Set to 0 / false / no / off to reproduce an upstream number exactly, or for a server that rejects stop_token_ids (vLLM and SGLang both accept it). |
HYPERLOOM_EVAL_STOP_STRINGS |
Unset (derived) | Explicit terminators, separated by ASCII unit separator 0x1f — commas and newlines are themselves legitimate stop strings. Outranks the derived values; use it when a checkpoint's metadata is absent or wrong. |
Set explicit terminators like this, quoting so the separator is a real 0x1f
byte:
export HYPERLOOM_EVAL_STOP_STRINGS=$'<|im_end|>\x1f<|endoftext|>'Upstream keeps at most four stop strings, and the task's own until list is what
its answer extraction depends on, so that list is never displaced: an explicit
HYPERLOOM_EVAL_STOP_STRINGS goes first, the task's list next, and derived
terminators last. Derived token ids travel separately as stop_token_ids, which
has no such limit, so nothing is lost on a server that supports it.
The following variables control the kernel optimization backend ladder.
| Variable | Default | Description |
|---|---|---|
KERNEL_OPT_BACKEND_ORDER |
Unset (resolves to geak) |
Selects the kernel-opt backend. Unset resolves to geak (whole-pipeline GEAK owns the KERNEL phase); the bare-metal installer and the Slurm launchers export ${KERNEL_OPT_BACKEND_ORDER:-geak} on top of that. Only an exact, case-insensitive forge opts into the per-kernel forge backend (forge_explicitly_enabled in common/env.py). Despite the historical name, a comma list is not parsed: forge,geak silently stays on geak, as does any other value, legacy alias, or payload override. |
KERNEL_OPT_MAX_PARALLEL |
8 (GPU-adaptive cap) |
Max parallel kernel-opt attempts per request (per-kernel race fan-out). The runtime caps this by visible GPUs and per-attempt GPU reservation when it can detect them. |
HYPERLOOM_GEMM_SHAPE_CAPTURE |
1 |
Enables automatic runtime GEMM-shape capture for eligible single-node dense vLLM Forge tuning when no explicit shape input is available. Block-FP8 first reuses shapes from the TraceLens-selected steady-state trace of a successful Roofline with exactly matching model, workload, server arguments, environment, and backend controls. Missing or stale evidence triggers the same standard Roofline/ProfileExecutor/TraceLens steady-state pipeline as a fallback. Set to 0 to preserve the no-capture path. |
HYPERLOOM_GEMM_SHAPE_CAPTURE_TIMEOUT_SEC |
1800 |
Timeout in seconds for the dense vLLM TunableOp recording benchmark. Block-FP8 fallback uses the standard Roofline/ProfileExecutor timeout. Values below 60 are clamped to 60. |
INFERENCE_OPTIMIZER_KERNEL_OPT_MAX_PARTIAL |
Unset | Cap on how many PARTIAL kernel-opt verdicts an action can yield before it short-circuits to NEEDS_REVIEW. Useful for keeping budget contained when GEAK is consistently timing out. |
KERNEL_OPT_BACKEND_BUDGET_MIN |
60 |
Wall-clock budget in minutes for one optimization, mirrored by the kernel_optimization.py wrapper. The env deliberately wins over the payload budget_minutes, which is LLM-authored from a prompt template, so an operator raising the budget is not silently overridden. forge-loop reserves half the window for finalize, so 60 leaves roughly 30 minutes of real iteration. |
AITER_LOG_TUNED_CONFIG |
1 (set for every serving run) |
Makes aiter log each tuned-config lookup it hits, not only the ones it misses. Two checks have no input without it: the GEMM demand list, which learns the shapes the runtime actually asks for (config-derived shapes covered 0.4% of them), and the apply verdict, which cannot tell "the tuned table was never read" from "it was read and did not help". A scan of 60 production logs found it set in none of them, so it is now injected by default. An operator value wins — set 0 to turn hit logging off, at the cost of both checks going inconclusive. Every miss already prints a line regardless of this setting; hit logging adds roughly one line per lookup that succeeds. |
HYPERLOOM_GEMM_PAIRED_PAIRS |
0 (off) |
How many interleaved baseline/tuned pairs to re-measure before a GEMM tuning KEEP is reported as confirmed. One end-to-end measurement cannot separate a gain from drift on this fleet: three rounds of a single unchanged configuration spanned 58%, and one controlled repeat moved 16%. Each pair costs two extra benchmark rounds. When 0, the gain is still promoted — it is the best number available — but recorded as an unpaired block comparison rather than presented as a paired one. |
The fusion lane is Coordinator-owned and forge-only: it runs at KERNEL entry on
the forge branch, never as an agent request, and the default geak backend
returns before reaching it. Its gate needs a fusion-eligible framework
(sglang, vllm or vllm-aiter), a decode trace to discover from, and no
fusion that already succeeded this session.
| Variable | Default | Description |
|---|---|---|
HYPERLOOM_SKIP_FUSION |
Unset (lane enabled) | Truthy (1 / true / yes / on) disables the fusion lane outright, before any other gate is evaluated. |
FORGE_FUSION_TIMEOUT |
7200 (2h) |
Wrapper timeout in seconds for one forge-fusion run. A payload timeout / timeout_sec takes precedence over the env; an unparseable value falls back to the default. |
FORGE_FUSION_MAX_TURNS |
100 |
Agent turn cap handed to forge-fusion for one run. A payload max_turns takes precedence. |
The collective lane is Coordinator-owned: it is dispatched directly at KERNEL
entry, never as an agent request. It requires TP > 1, a latest-snapshot
Exposed Communication % of at least 1% as parsed from the TraceLens executive
summary, a trace_analyze snapshot, and a source-resolved custom collective
candidate (all_reduce, reduce_scatter or all_gather) — vendor RCCL/NCCL
symbols are opaque binaries and never qualify.
| Variable | Default | Description |
|---|---|---|
HYPERLOOM_SKIP_COLLECTIVE |
Unset (lane enabled) | Truthy (1 / true / yes / on) disables the collective lane outright, before any gate is evaluated. |
HYPERLOOM_COLLECTIVE_ONLY |
Unset | Truthy runs ONLY the collective lane at KERNEL entry — GEAK, fusion and per-kernel kernel_opt are all skipped — and hints skip_to_sweep once the lane settles. Also the way to reach the lane while KERNEL_OPT_BACKEND_ORDER selects geak, which otherwise owns the whole phase. Mirrored into the collective_only_mode SharedState field. |
HYPERLOOM_COLLECTIVE_KEEP_PCT |
1.0 |
E2E KEEP threshold in percent for the collective integrate. Must parse as a finite, non-negative float, otherwise the integrate fails loudly rather than defaulting. |
HYPERLOOM_COLLECTIVE_ALLOW_INFERRED_SHAPES |
Unset (disabled) | Truthy allows a source-resolved collective to borrow shapes from the trace's sole all-reduce workload family. The default rejects this inference because those shapes were not observed on that device symbol. |
FORGE_COLLECTIVE_TIMEOUT |
14400 (4h) |
Wrapper timeout in seconds for one forge-collective campaign; a collective iterates over N ranks per benchmark, hence the wide default. A payload timeout takes precedence over the env. |
FORGE_COLLECTIVE_AGENT_TIMEOUT |
Unset (wrapper default) | Per-agent timeout in seconds, forwarded to forge-collective as --agent-timeout-sec. A payload agent_timeout_sec takes precedence. |
A kernel candidate must resolve to a real source file before any backend can rewrite it. Resolution runs as a ladder: curated dictionary, then the trace-derived launcher frame, then a name grep. All three are deterministic and require no configuration. Agent analysis may add the model-backed tiers below.
Every run writes kernel_source_resolution.json next to the candidate report.
It answers one question per hot kernel — which file defines it, and which tier
decided that — in a versioned schema (schema_version, currently 1.0.0), so
consumers and triage read a contract rather than candidate internals.
Two model-backed tiers may sit on top of the deterministic ladder when
--analysis-route agent is used. The deterministic route never invokes either
tier. Agent-route network calls require an explicit
HYPERLOOM_LLM_SOURCE_PROVIDER; a model name alone never implies a provider or
endpoint. The tiers differ in scope, authority and data exposure; the
constraints of one do not apply to the other.
Neither can fail a run: no model configured, a gateway error, a timeout or an unparseable reply all leave the deterministic result standing.
When it runs. Only for a candidate whose source_file is still empty after
all three deterministic tiers, and whose GPU share is at least 5%.
What it sends. One chat completion per such candidate, containing the kernel
symbol and every shortlisted path. The shortlist comes from a relaxed grep over
the known framework roots. File contents are not sent unless
HYPERLOOM_LLM_SOURCE_PREVIEW authorises it (see Source egress);
with it, each path is accompanied by its first 40 lines, capped at 2000
characters.
What it costs. One call per qualifying candidate, 60-second ceiling, no
retry. HYPERLOOM_LLM_SOURCE_MODEL overrides the selected provider's model
setting. Claude uses CLAUDE_MODEL, then the project-wide
DEFAULT_CLAUDE_MODEL. Model settings are never borrowed across providers.
Authority: selection only. The model may return one of the exact shortlist
strings and nothing else. An invented path is rejected, as is any answer below
0.7 confidence. This is deliberate — an LLM-produced sentinel written into
source_file is what broke this pipeline originally.
The fallback only fires on an empty source_file, so it cannot catch the
deterministic tiers' actual failure mode: not coming up empty, but coming up
confidently wrong. Measured across historical sessions, only 59% of
verifiable resolutions mention the kernel they claim to define, and
aten::fill_ alone has been resolved to four unrelated business files — each a
real, existing, root-resident source file passing every mechanical check.
When it runs. On the whole resolution table, including entries already filled in by the deterministic tiers. Entries below 1% GPU share are skipped.
What it sends. A single chat completion carrying up to 40 entries at once.
For each entry it includes the kernel symbol, GPU share, current path and
deciding tier. File contents follow the same rule as the fallback tier: nothing
is sent unless HYPERLOOM_LLM_SOURCE_PREVIEW authorises it. When it does, one
call can ship up to 40 file heads, considerably more than the fallback tier
sends per call — which is why the switch is global rather than per-tier.
What it costs. One call per run (not per candidate), 180-second ceiling, no
retry. Same provider and model resolution as the fallback tier. The response
must include every sent kernel_id exactly once. A missing, duplicate or extra
ID rejects the whole batch so a truncated response cannot masquerade as a
complete review.
Authority: it may rewrite, and it has no confidence threshold. Unlike the
fallback tier, this one is not restricted to a shortlist — it can replace any
entry's path with any path, or drop a resolved entry back to unresolved. There
is no 0.7 confidence gate. The mechanical limit is that a rewritten path must
exist on disk and its resolved target must sit under a known framework root.
Symlinks cannot escape that boundary. TraceLens-style
path.py(247): function answers are split into a bare, openable path plus line
and function metadata. An unverifiable path is rejected and the original
stands. Curated op_to_source verdicts — including non_rewritable and
no_kernel — are authoritative and cannot be replaced by model review.
Every revision records previous_source_file and previous_method, so a bad
review is auditable and reversible, and review_notes lists every applied and
rejected change. The batch is staged before it is committed, so an exception
while validating one revision leaves every entry untouched. Failures — no model
configured, gateway error, timeout, unparseable reply — leave the deterministic
table untouched and are recorded in review_notes.
Accepted revisions are folded back into hot_kernels, all metadata derived from
the old path is cleared, and patchability is recomputed. The resolution JSON is
the audit view of the same effective candidate state, not a detached suggestion.
Both tiers call an external model provider, so what leaves the host is a deliberate boundary rather than a side effect of building a useful prompt.
Provider routing is explicit. Set HYPERLOOM_LLM_SOURCE_PROVIDER to
claude_agent_sdk. Claude requests use the native Claude
Agent SDK with all repository, shell and web tools denied.
kernel_source_resolution.json records the provider, model, source-preview
decision, outcome and endpoint hostname. It never records keys, custom
headers, URL userinfo, query parameters or the full prompt.
Repository source is not sent by default. The file heads described above are
withheld unless HYPERLOOM_LLM_SOURCE_PREVIEW is set to 1/true/yes/on.
Without it both tiers still see candidate paths, which carry most of the
selection signal; with it, a review call can ship up to 40 file heads.
The serving command line is never forwarded verbatim. The tiers need backend
flags — the same MoE operator dispatches differently under
--moe-runner-backend triton and aiter — but EXTRA_*_ARGS also carries
credentials, model paths and user data. It is therefore tokenised, and only
flags on an explicit allowlist of backend selectors survive. A denied flag
consumes its value too, so the value cannot reappear as a stray token. Every
surviving value is dropped unless it is a short selector token. URL userinfo or
queries, authorization headers, JWTs, control characters, non-finite numbers,
vendor prefixes such as sk-, and long opaque strings are rejected. An
unbalanced quote discards the whole line rather than risking a partial parse.
Environment variables follow the same discipline: an explicit allowlist of path-selecting names, with the secret-name pattern applied on top.
Model config is allowlisted too. Only fields that select architecture,
expert layout or kernel format are included. Inside quantization_config, only
explicit quantization selectors survive; arbitrary vendor fields, nested
metadata and credential-shaped values are dropped.
| Variable | Default | Description |
|---|---|---|
HYPERLOOM_LLM_SOURCE_PROVIDER |
Unset (no network call) | Required provider for source fallback/review: claude_agent_sdk (native Claude SDK, tools denied). Common provider aliases are normalized to the canonical audit value. |
HYPERLOOM_LLM_SOURCE_MODEL |
Unset | Optional source-resolution model override. Otherwise resolves only from the selected provider's own model variables; no cross-provider fallback. |
HYPERLOOM_LLM_SOURCE_PREVIEW |
Unset (off) | Authorise sending the first 40 lines of candidate source files to the model provider. Applies to both the fallback and review tiers. Leave unset unless the provider is an approved destination for repository content. |
The fallback tier is advisory and never fails a run. Every
outcome is recorded on the candidate as source_resolution_reason, so a skip
can be told apart from a genuine failure:
source_resolution_reason |
Meaning |
|---|---|
| (absent) | Resolved before fallback, so the tier was not reached |
llm_fallback_skipped: deterministic route |
Deterministic analysis explicitly prohibited model tiers |
llm_fallback_skipped: gpu_pct ... |
Candidate below the 5% GPU-share floor; no call made |
llm_fallback_skipped: no provider configured |
No HYPERLOOM_LLM_SOURCE_PROVIDER; settled before the shortlist grep, so an unconfigured tier costs nothing |
llm_fallback_no_shortlist |
Grep found nothing to choose from; no call made |
llm_fallback_declined: ... |
Model answered but the pick was rejected (invented path, low confidence, or refusal) |
llm_fallback_error: ... |
Call failed — import error, gateway rejection, or timeout |
Accepted answers are stamped source_resolution_method="llm_fallback" alongside
a source_resolution_confidence, so they can be audited separately from
deterministic resolutions. Failures in the trace-launcher tier are recorded the
same way under trace_resolver_error: ..., and both are logged at WARNING.
| Variable | Default | Description |
|---|---|---|
INFERENCE_OPTIMIZER_RAY_EXEC |
Unset (on for single-node) |
Controls whether single-node serving benchmarks and needs_gpu specialists run through Ray actors. When unset, single-node runs are routed through Ray-managed leases while multi-node stays on the multi-node backend. Set to 0 / false / no / off to force the local subprocess path, or 1 / true / yes / on to force Ray. |
Selects how a Codex agent session (TraceLens analysis and every future
Codex-based agent) is contained. The secure default is workspace-write.
Codex implements both contained presets with bubblewrap, so Hyperloom executes
a real namespace-and-mount capability probe before starting the SDK. Merely
finding a bwrap executable is insufficient: if the current kernel or
container prevents it from establishing the sandbox, workspace-write and
read-only fail closed before the app-server starts. There is no automatic
fallback to bypass.
bypass is a deliberate double opt-in. Set both
HYPERLOOM_CODEX_SANDBOX_MODE=bypass and
HYPERLOOM_CODEX_EXTERNAL_SANDBOX=1; the second variable confirms that an
external container or sandbox already enforces the required isolation. It does
not create that boundary. A confirmed bypass maps to Codex full access even
when no writable roots are declared, because the external sandbox is
authoritative. Under the contained modes, no writable roots remains
read-only. Unknown modes and incomplete bypass configuration fail
immediately.
| Variable | Default | Description |
|---|---|---|
HYPERLOOM_CODEX_SANDBOX_MODE |
workspace-write |
workspace-write restricts writes to the session directory plus declared output roots; read-only forbids writes; bypass selects Codex full access only when the external-sandbox confirmation below is also set. |
HYPERLOOM_CODEX_EXTERNAL_SANDBOX |
Unset | Set exactly to 1 only when an external isolation boundary is already active and HYPERLOOM_CODEX_SANDBOX_MODE=bypass. Setting this alone has no effect and never weakens the default sandbox. |
These variables tune the single-node Ray execution path (active when
INFERENCE_OPTIMIZER_RAY_EXEC=1 and --nodes=1). They have no effect on
multi-node runs or when the Ray backend is disabled.
| Variable | Default | Description |
|---|---|---|
INFERENCE_OPTIMIZER_RAY_GPU_PENDING_LIMIT |
4 |
Maximum number of GPU specialists that may be simultaneously in-flight (pending Ray scheduling + running) on the single-node Ray path. Ray still serialises execution on the physical GPU(s) via num_gpus; this limit caps how many actors can queue behind the current one. Floored at 1. Reduce to 1 or 2 when GPU memory or per-process overhead is a concern (each queued actor holds a Ray worker slot even while it waits). |
INFERENCE_OPTIMIZER_RAY_SERVING_PRIORITY |
On | When enabled (default), the dispatcher defers admitting new GPU research specialists while a serving benchmark holds the whole-machine serving_slot, preventing research work from starving serving. The slot is probed immediately before each specialist is admitted so a serving start that races the dispatch pass is caught. Set to 0, false, no, or off to disable. |
Use CLI flags for multi-node topology and prefill-decode configuration:
--nodes, --mn-backend, --gpus-per-node, --tp, --ep,
--pd-mode, --pd-prefill-nodes, --pd-decode-nodes, --pd-prefill-tp,
--pd-decode-tp, --pd-transfer-backend, and --pd-ib-device.
optimize never creates or releases a multi-node cluster. The provisioning
platform (e.g. Primus-Claw) creates the RayJob or InferaDeployment and hands it
over through the variables below; without a hand-off --nodes >= 2 exits 2.
HYPERLOOM_MN_EXT_SERVICE_URL is the only variable that tells the optimizer a
cluster is ready; the rest describe how to reach it.
| Variable | Backend | Required | Description |
|---|---|---|---|
HYPERLOOM_MN_EXT_SERVICE_URL |
both | yes | Benchmark frontend URL (http(s)://…; infera frontend typically :8000). Its presence triggers external mode. |
HYPERLOOM_MN_EXT_SSH_KEY |
infera | yes | Private SSH key already authorized on the pods (the platform installs the public half at create time). |
HYPERLOOM_MN_EXT_PREFILL_IPS / _DECODE_IPS |
infera | PD | Prefill / decode pod IPs (comma-separated) for PD-disaggregated runs. |
HYPERLOOM_MN_EXT_WORKER_IPS |
infera | aggregated | Worker pod IPs (comma-separated) for aggregated (non-PD) runs. At least one of _PREFILL_IPS / _DECODE_IPS / _WORKER_IPS is required. |
HYPERLOOM_MN_EXT_SSH_PORT |
infera | No (default 2233) |
SSH base port; decode role is offset +10. |
HYPERLOOM_MN_EXT_SSH_KNOWN_HOSTS |
infera | No | known_hosts path; else a relaxed host-key check is used. |
HYPERLOOM_MN_EXT_HEAD_IP |
rayjob | No (recommended) | Ray head IP (Dashboard :8265, GCS :6379). Enables per-round restarts; omit for benchmark-only. |
HYPERLOOM_MN_EXT_RAY_DASHBOARD_TOKEN |
rayjob | No | Ray Dashboard auth token, only if the dashboard is authenticated. |
Infera external mode requires HYPERLOOM_MN_EXT_SSH_KEY plus at least one
*_IPS list, or the run fails fast at startup. RayJob external mode ignores
the SSH / IP vars and uses HYPERLOOM_MN_EXT_HEAD_IP for restarts.
Multi-node SSH fanout creates session-scoped keys under the active session
directory. Treat mn_id_ed25519 and mn_id_ed25519.pub as sensitive session
artifacts: keep the session directory on an access-controlled filesystem and
do not publish it unchanged in support bundles.
| Variable | Default | Description |
|---|---|---|
HYPERLOOM_QUANTIZE_ENABLED |
Unset | Primary switch (1 to enable) for the AMD Quark PTQ quantization prelude driven by --quantize / --quantize-scheme. |
QUARK_ROOT |
Unset | AMD Quark checkout used by the quantization-agent. Set this explicitly when quantization is enabled. |
Enablement is not configured through the environment. Both self-heal lanes
are admitted by the --enablement {off,launch,eval,all} CLI flag, which defaults
to all:
launch— a baseline that cannot boot routes into patch authoring.eval— a baseline that boots and measures throughput but fails its accuracy eval (crashes, produces no result, or scores below the floor) routes into patch authoring. Single-node only; multi-node keeps the strict stop.all(default) — both lanes.off— neither lane engages, and a baseline that keeps failing terminates the run withstop_reason='baseline_failed'instead of opening an authoring loop.
The accuracy floor shared by the eval trigger and the enablement KEEP gate is the
fixed constant _accuracy_gate.DEFAULT_ENABLEMENT_ACCURACY_FLOOR (0.05). It is
a collapse guard rather than a quality bar: a score of exactly 0.0 always fails,
otherwise score >= floor passes.
The following variables configure framework source discovery and path overrides.
| Variable | Default | Description |
|---|---|---|
INFERENCE_OPTIMIZER_FRAMEWORK_SOURCE_ROOTS |
Union with /sgl-workspace/{aiter,sglang,vllm} |
Colon-separated list of source roots used by PolicyGate and flag discovery. Populated automatically by src/hyperloom/inference_optimizer/assets/install.sh's _probe_framework_source_roots step (using hyperloom.orchestrator.framework.paths.probe_framework_source_roots_for_env). |
INFERENCE_OPTIMIZER_RESCUE_PATHS |
Unset | Colon-separated list of extra directories the harvest step scans for stray result.json files written outside the session dir (InferenceX-native scripts that hardcode --result-dir). |
INFERENCE_OPTIMIZER_AITER_JIT_DIR |
Aiter default | Override the aiter just-in-time (JIT) cache root. See Targeted builds (Rung 5). |
INFERENCE_OPTIMIZER_STRICT_PATHS |
1 when CLI bootstraps |
When 1, missing path env raises instead of falling back to discovery. Set by the CLI at session start; do not override unless debugging. |
HYPERLOOM_SGLANG_PATCH_EXACT_VERSIONS |
Unset | Pin the sglang server-patch step to specific upstream versions; advanced compatibility option. |
HYPERLOOM_ENABLE_PATCH |
1 |
Set to 0 to skip the in-place server patch step (useful when the upstream is already pre-patched). |
HYPERLOOM_SKIP_FRAMEWORK_CHECK |
Unset (check enabled) | Truthy skips the optimize preflight gate that requires the selected serving framework to be importable and a ROCm build. Last resort: when the server runs elsewhere, set BENCHMARK_BASE_URL instead, which exempts the check and configures the supported path. The gate already stays out of the way for xdit/custom (server-less), external multi-node, and any framework install_baremetal.sh cannot install (atom), where it warns instead of blocking. |
AITER_REF |
Unset | Optional bare-metal AITER install pin. When unset, the installer selects the newest tag compatible with the installed torch/triton stack. |
INFERENCE_OPTIMIZER_FRAMEWORK_AUDIT_USE_LLM |
auto |
Controls the FRAMEWORK phase semantic-audit LLM deep-read. off keeps the hermetic static verdict only; on always runs the evidence-gated LLM refine; auto (default) escalates to the LLM only when the static verdict is unknown or confidence < 0.5. The refine never upgrades to an already_* status the static layer did not already back with evidence. |
These variables control the Rung-5 off-loop compiled-component acquisition step (AITER FP4/MLA/NSA kernels, sgl-kernel, and vLLM from source). All are optional; defaults are safe for standard single-node deployments.
| Variable | Default | Description |
|---|---|---|
HYPERLOOM_ENABLEMENT_DISABLE_TARGETED_BUILD |
Unset (0) |
Set to 1 to completely disable Rung-5 auto-escalation. When set, compiled-gap failures proceed to the stall gate without attempting a build. Useful when the compile toolchain is unavailable or the session budget is too tight. |
INFERENCE_OPTIMIZER_AITER_JIT_DIR |
Aiter default | Per-attempt override set automatically to <attempt_root>/aiter_jit by each targeted build. Override manually only when you need the global JIT cache to point at a pre-built location; leaving it unset lets each build use its own isolated directory. |
PYTORCH_ROCM_ARCH |
Detected | Explicit GPU target architecture (e.g. gfx942, gfx950) injected into each compile. Set automatically from the session --gpu-type; operator-override applies to bare-metal installs outside the session. Compile target only — it does not participate in architecture detection. It names the archs a wheel is built for, not the installed device, so provenance ignores it entirely and resolves gfx_arch from HYPERLOOM_GFX_ARCH, then --gpu-type, then rocminfo. |
MAX_JOBS |
8 |
Parallelism cap for cmake/hipcc compile steps inside a targeted build. Reduce on memory-constrained nodes (MAX_JOBS=4 for a 64 GB compile node). The default 8 is conservative enough for MI300X/MI355X nodes with 512 GB+. |
HYPERLOOM_FRAMEWORK_PYTHON |
Unset | Explicit interpreter that launches the server for a from-source build (the venv Python the artifact was compiled against). Set automatically from FrameworkRuntime.runtime_python_exe via apply_runtime_override into the per-variant YAML benchmark.envs. Both backends export that mapping to the server env; the bypass backend additionally uses this value as the python -m interpreter. Operators normally do not set this by hand. |
HYPERLOOM_VLLM_ROCM_INDEX_URL |
Unset | ROCm pip index URL used as the default vLLM adapter wheel index; also seeds the index allowlist. |
HYPERLOOM_ENABLEMENT_INDEX_ALLOWLIST |
Unset | Comma-separated allowlist of pip index URL prefixes; a candidate wheel index must match one of these prefixes or provisioning is refused (supply-chain safety). |
HYPERLOOM_ENABLEMENT_ORIGIN_ALLOWLIST |
Unset | Comma-separated allowlist of git origin URL prefixes; a candidate repo origin must match one of these prefixes or provisioning is refused (supply-chain safety). |
HYPERLOOM_SGLANG_REPO_URL |
Unset | Override the SGLang source repo URL for the sgl-kernel / SGLang-from-source enablement build. |
HYPERLOOM_SGLANG_REF |
Unset | Pin the SGLang source ref (tag/branch/sha) for the enablement build. |
HYPERLOOM_SGLANG_INDEX_URL |
Unset | SGLang wheel index URL for the enablement build. |
Supply-chain security:
HYPERLOOM_ENABLEMENT_INDEX_ALLOWLISTandHYPERLOOM_ENABLEMENT_ORIGIN_ALLOWLISTare security controls. When set, only pip index / git origin URLs matching one of the listed prefixes are accepted for runtime provisioning; any non-matching candidate is refused.
These switches keep production-compatible behavior by default while still allowing operators to turn off credential/env persistence in hardened deployments.
| Variable | Default | Description |
|---|---|---|
HYPERLOOM_SPECIALIST_INHERIT_SECRET_ENV |
Unset (1) |
Specialist subprocesses inherit the limited provider credential set by default: ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN, ANTHROPIC_CUSTOM_HEADERS, CLAUDE_CODE_OAUTH_TOKEN, LLM_GATEWAY_KEY, and AWS Bedrock credential/config vars. Set to 0 only when the claude CLI is authenticated through its own config and env credentials must be suppressed. Unrelated secrets such as GitHub and KB tokens remain blocked. |
HYPERLOOM_SPECIALIST_PERMISSION_MODE |
bypassPermissions |
--permission-mode passed to the claude CLI for specialist subprocesses. Controls the Claude runtime approval-prompt behaviour only. Codex containment is resolved independently through HYPERLOOM_CODEX_SANDBOX_MODE. The default bypassPermissions is required for unattended operation; change only in setups where an external interactive approval flow is intended. |
HL_ALLOW_DANGEROUS_AGENT_PERMISSIONS |
Unset (0) |
Slurm carrier only. Set to 1 only in dedicated internal containers to re-enable legacy Claude/Codex approval and sandbox bypass flags. |
The following variables configure the Critic, Robustness, and knowledge base components.
| Variable | Default | Description |
|---|---|---|
KNOWLEDGE_STORE_MODE |
local |
Exclusive Recipe backend: local or remote. Ambient KB Store or GBrain credentials do not select remote mode. |
KNOWLEDGE_LOCAL_ROOT |
$USER_DATA_PATH/knowledge, otherwise ~/.cache/hyperloom/knowledge |
Local Recipe/KG root. It is not used for Recipe data in remote mode. |
HYPERLOOM_LOCAL_KB_ROOT |
Unset | Deprecated explicit local Recipe root compatibility input, overridden by --local-kb-root; explicit use skips automatic legacy migration. |
INFERENCE_OPTIMIZER_FA_KB_PATH |
$USER_DATA_PATH/framework-kb, otherwise /workspace/hyperloom/framework-kb |
Framework-agent KB root, holding the lessons ledger the FRAMEWORK phase reads and writes. The only supported override: the fa reader and the orchestrator's writeback both resolve through it, so it moves both halves at once. The withdrawn FRAMEWORK_AGENT_KB_DIR is ignored with a warning naming the resolved root. On first start-up an existing partition under the legacy $USER_DATA_PATH/kb is copied across once; a copy that fails warns and leaves the phase to cold-start. |
KB_STORE_URL |
Unset | KB Store endpoint. Required when KNOWLEDGE_STORE_MODE=remote; remote Recipe mode selects the current Recipe View, replays its combined config, ordered Explore/Framework overlays, and Kernel section, then writes one final session at CLOSE. |
KB_STORE_TOKEN |
Unset | KB Store bearer token. Required when KNOWLEDGE_STORE_MODE=remote; transport failures during the final write are non-fatal. |
KB_DRAFT_DIR |
Runtime-generated | Internal remote-mode handoff where out-of-process agents stage their section knowledge and files. Hyperloom creates and exports it; operators must not set it. The facade is inactive when it is absent. |
KB_WARM_START_DIR |
Runtime-generated | Internal remote-mode handoff pointing agents at the downloaded recipe.json + files/ selected Recipe View. Hyperloom creates and exports it; operators must not set it. |
GBRAIN_BASE_URL |
Unset | Optional GBrain endpoint for non-Recipe KG and Framework PR capabilities. It never enables or satisfies Recipe remote mode. |
GBRAIN_TOKEN |
Unset | Optional GBrain bearer token for non-Recipe KG and Framework PR capabilities. It never enables or satisfies Recipe remote mode. |
CRITIC_AGENT_ROOT |
Derived from REPO_ROOT |
Override location of the critic-agent runtime. |
CRITIC_AGENT_MAX_COMPLETION_TOKENS |
32000 |
Output-token cap for one critic review call. A reply cut off at the cap is retried once at twice this value and then fails the turn, so the cap is a ceiling rather than a budget: unused headroom is never billed, while a truncated reply bills the whole call and yields nothing. Lower it for a model whose own output limit is smaller. A non-positive or unparseable value logs a warning and falls back to the default. |
ROBUSTNESS_AGENT_ROOT |
Derived from REPO_ROOT |
Override location of the robustness-agent runtime. |
ROBUSTNESS_LLM_RCA_DISABLED |
Unset | Set to 1 to forcibly disable the LLM root cause analysis (RCA) engine even when credentials are present. |
These are read by src/hyperloom/inference_optimizer/session/manifest.py and the src/hyperloom/inference_optimizer/breakdown/collectors/
package to populate session_breakdown.json for downstream consumers.
| Variable | Description |
|---|---|
CLAW_SESSION_ID |
Hosted SaFE / Claw session id, written to session.claw_session_id in session_breakdown.json. Set by the Primus-Claw sandbox; unset for local runs. |
SANDBOX_USER_ID |
Hosted SaFE / Claw user id, written to session.sandbox_user_id. Set by Primus-Claw; unset for local runs. |
HYPERLOOM_LANGFUSE_ENABLE |
Primary switch (default off) for live Langfuse trace push. See details below. |
HYPERLOOM_LANGFUSE_ENABLE details:
Primary switch (default off) for live Langfuse trace push.
-
SDK install: when this flag is on,
src/hyperloom/inference_optimizer/assets/install.shauto-installs the optionallangfuseSDK on demand and skips it entirely when off — no separatepip install '...[trace]'is required. -
Live push: when set to
1/true/yes/onand the threeLANGFUSE_*credentials are present, every in-process LLM call is mirrored into Langfuse while the run is live. A session-end flush backfills out-of-process children (geak, forge, robustness, specialist) and KEEP/REVERT decision Scores. -
Local ledger:
reports/trace/*.jsonlis always written regardless of this flag. If the SDK is unavailable, live push degrades to a no-op. -
Correlation: the Langfuse trace ID and
session_idgrouping are derived fromclaw_session_id(envCLAW_SESSION_ID), falling back to the internal session ID for standalone runs. Live push and the offlinebackfill_langfuseCLI collapse onto one trace per Primus-Claw session. -
Span layout:
trace → phase span (PRELUDE/FRAMEWORK_AGENT/EXPLORE/KERNEL_AGENT/SWEEP/…) → agent span (component: orchestration/kernel/specialist/critic/geak/forge/…) → Generation. Each KEEP/REVERT/gain_pctScore attaches to the agent span that produced the decision, with a trace-level fallback when no matching span exists. -
Recipe-KB spans: under the
recipe_kbagent span, local reads/writes and remote KB Store publish attempts are recorded fromruntime/recipe_snapshot/.audit.jsonl. Read spans usekb:recipe_snapshot:<method>; write spans usekb:recipe_write:<generator>, where the generator distinguishes normalclosefromt4_fallback. Remote rows reportwritten,skipped, orerrorwithout recording credentials or payload bodies. -
Receipt: every session records a
langfusesection insession_breakdown.json(andreports/trace/langfuse_receipt.json) noting:- Whether push was enabled (or the
disabled_reason) - The redacted connection config (host and key-presence booleans — never the keys themselves)
- The derived
trace_idandsession_id - How many generations, scores, and spans were sent
This lets an operator confirm post-hoc whether a run reached Langfuse.
- Whether push was enabled (or the
- Sensitive data surface: When live push is on,
conversations.jsonl(and Langfuse Generations) carry full prompt/response text.redact_secretsscrubs common token shapes (Bearer,sk-/pk-, GitHub tokens, someKEY=value) but is not a complete data loss prevention (DLP) filter — bare keys without a recognizable prefix (for example, raw AWSAKIA…) can slip through. The artifact packager also copiesreports/trace/*.jsonland, with the loose mode on by default (HYPERLOOM_SESSION_PACKAGE_LOOSE), drops them under/workspacefor the Claw sync. If a session might contain customer code or secrets, define an explicit retention + access-control policy for both the Langfuse project and the/workspacepackage destination, and consider disabling live push or loose packaging for those runs. live push+backfill_langfuseoverlap: Both derive the sametrace_idfromclaw_session_id, so running the offline backfill after a live run re-emits the out-of-process children onto the same trace and can duplicate observations. Use one path per session, or treat backfill as a recovery tool only when live push did not run.flush_sessionis idempotent: A second flush only re-writes the receipt (no re-emit), so a duplicated CLOSE step won't double-push.- Package truncation: The bundle caps at 5000 files / 256 MB. On a very
long session the cap can stop the bundle short; the
PACKAGE_MANIFESTthen setstruncated: trueand listsdropped_files, so consumers must not treat a truncated package as complete. - Generation duration is ~0: Both live and backfill stamp a single
timestamp (
end == start), so Langfuse shows no meaningful per-Generation duration — counts/usage are accurate, latency is not captured.
Every breakdown carries a top-level token_usage section: a promoted,
discoverable rollup of LLM token spend derived from the per-call ledger
(reports/trace/llm_calls.jsonl + ext/*.jsonl). It is purely derived from
decision_trace.token_rollup, so it always reconciles with that section. No
env var controls it; it is always present (zeroed on pre-trace sessions).
session_total: whole-session total across every call, with two convenience figures:total_in_out(prompt + completion only) andgrand_total(in + out + all cache-creation + cache-read tokens).by_component: per-agent breakdown (orchestration / kernel / critic / specialist / proposal_scorer / geak / forge / …), each with the same convenience totals.by_phase: per-phase breakdown (PRELUDE / FRAMEWORK_AGENT / EXPLORE / KERNEL_AGENT / SWEEP / CLOSE).attribution:attributed_to_decisionsvsunattributedsplit plusattributed_calls_pct. Only calls that carry atask_id/dyn_idjoining to a KEEP/REVERT or dynamic_action decision (for example, specialist subprocess turns) are attributed; orchestration / kernel / critic / proposal_scorer turns are LLM-internal and land inunattributed(this is expected, not a gap in the data).timeline: eachaction_timelinerow annotated with the tokens that join to it ontask_id. Rows whose action has no LLM spend showtokens: null(rather than a zero bucket) to make the sparsity explicit.
To get the single "total tokens for this run" number, read
token_usage.session_total.grand_total (all-in) or .total_in_out
(prompt+completion only).
| Variable | Required | Default | Description |
|---|---|---|---|
INFERENCE_OPTIMIZER_CYCLE_RELOOP_MIN_REMAINING_SEC |
Optional | 10800 |
Absolute minimum remaining session seconds to justify opening a new macro-cycle. For bounded sessions the effective floor is min(this, max_minutes * 60 * 0.15) so shorter sessions are not unconditionally blocked. |
These are read by os.environ somewhere in the codebase but are
internal-only — do not set them by hand:
HYPERLOOM_KERNEL_AGENT_ROOT: internal CLI-only handoff to the kernel subprocess (Python constant_KERNEL_AGENT_ROOT_ENV).- Any
_INFERENCE_OPTIMIZER_*_INTERNAL_*symbol: internal toggles for the test suite.
If you find one of these in a log message, treat it as diagnostic detail rather than something you should tune.
Use these resources for related configuration and reference information:
- Hyperloom authentication and credentials: Credential precedence and direct upstream gateway wiring.
- Troubleshooting Hyperloom: Symptom → variable reverse-lookup for common failures.