Repository navigation
test: verify installed Decisions end to end before calibration #125
Copy link
Copy link
Closed
Labels
hermes-kanban-goready-for-agentFully specified and ready for an AFK agentFully specified and ready for an AFK agent
Description
Activity
- addedready-for-agentFully specified and ready for an AFK agentFully specified and ready for an AFK agent
on Sep 28, 2026 Installed/live acceptance complete.
Implementation and repair revisions:
- Initial Decisions implementation on upstream main:
cefc7bdbbccb35500fb1707756072013f0d7e37a - Corrected upstream main after PR fix: describe Trust Gate egress metadata as selected fields, not redacted #130:
f4f7e27bbb415f628df62be2a5ccdade41b0546a - Installed non-editably into disposable virtual environments; product/server version
0.7.0, MCP SDK2.2.0, JJ0.45.1.
Secret-free configuration:
- policy:
decisions - provider:
openrouter - model alias:
~typesafe/jev-latest - endpoint:
https://openrouter.ai/api/alpha/decisions - Noul question:
Does this candidate meet the supplied Trust Gate policy for promotion as a useful institutional memory? - threshold:
0.5 - credential supplied only through
trust_gate_api_key_file; value was never printed.
Real installed MCP evidence:
- Quality candidate: probability
0.90, approved/promoted, governed recall andget_thoughtvisible, Decisions provenance persisted. - Server restart: the approved record and provenance remained readable.
- Vague candidate: probability
0.04, rejected, absent from governed recall. - Invalid credential: real HTTP 401, failed closed, no promotion, no fallback.
- Valid credential restored: real request succeeded; probability
0.30, rejected at threshold0.5. - Returned model identity:
typesafe/jev-1.13-20260917.
Confirmed defect and repair:
- The original egress disclosure incorrectly called the transmitted metadata “redacted.”
- PR fix: describe Trust Gate egress metadata as selected fields, not redacted #130 corrected code, diagnostics, docs, and tests to say the full prompt, full candidate body, and complete selected metadata fields are sent;
agent_idandmetadata.extraare excluded. - Exact-head validation for PR fix: describe Trust Gate egress metadata as selected fields, not redacted #130: Ruff passed; 108 affected trust-gate tests passed locally; both JJ CI matrices, CodeQL, and semantic-title checks passed after rerunning a GitHub-rate-limited JJ download.
- Fresh install from repaired main:
doctorreported the corrected disclosure and credential source without secrets. - Fresh real MCP/Jev request after repair: model
typesafe/jev-1.13-20260917, probability0.78, approved/promoted at0.5, governed recall visible with persisted Decisions provenance.
All acceptance data remained under disposable
/private/tmpstorage; no institutional data repository or pinned vendor checkout was modified.- Initial Decisions implementation on upstream main:
Installed runtime acceptance and the post-repair live Jev check are complete. Closing this dependency so calibration issue #126 can begin.
- added a commit that references this issue
on Sep 30, 2026
Metadata
Metadata
Assignees
Labels
hermes-kanban-goready-for-agentFully specified and ready for an AFK agentFully specified and ready for an AFK agent
02 — Install merged main locally and verify Decisions before calibration
Parent
#123
What to build
Prove the installed Decisions path works before calibration. After ticket 01 merges, install that upstream main revision locally, explicitly configure Decisions with an initial Noul question and threshold
0.5, and verify real OpenRouter review and durable recall through the installed MCP server. The initial settings are test inputs, not calibrated defaults. A suitable initial question is: Does this candidate meet the supplied Trust Gate policy for promotion as a useful institutional memory?Acceptance criteria
decisions, OpenRouter,~typesafe/jev-latest, the initial Noul question, and threshold0.5using the normal configuration and nestedtrust_gate_decisions_config. Read the authorized credential throughtrust_gate_api_key_file; never print it.propose_truth, and verify Jev approval, persisted Decisions provenance, and visibility through governedrecallandget_thought.0.5afterward and record the initial question and observed probabilities.Blocked by
Execution ownership
The Captain coordinates this local acceptance autonomously on the operator's host; Hermes crew handles implementation and repairs. This is executable agent work, not a human-only gate. The parent epic is not complete until the installed runtime passes these checks. Dispatch starts only after the operator's command to run the epic.
The operator has authorized using the local OpenRouter key file for these real tests. Its host-specific location is supplied in the private Captain handoff, not embedded as a worker-container path assumption. If it is missing, unreadable, or rejected, report the concrete result without exposing the credential. Do not replace provider state or request a new key without that evidence.