[benpankow]

Summary

From the 013 re-run after #5499 and #5503:

• Approved calls. eve only runs a tool with an approval policy once a person approves it, and sets ctx.approval on that call. The 013/014 graders ran add_note and close_ticket without it. Opus 5.5 checks ctx.approval in add_note as a second guard, in case a consuming agent re-defines the tool with a weaker policy, so runs 2 and 3 failed "add_note posts the note" and "honors a configured key and base URL". The graders now pass an approver to calls of tools that have an approval policy.
• Looking customers up before notes. The prompt asks that agents "always look the customer up before writing a note". The grader only accepted an extension instructions fragment saying so. Haiku 5.5 (runs 1, 2) and GPT-5.6 Terra (run 2) made add_note take an email and look the customer up itself, so a note can't land on the wrong record. That's stronger than instructions, but it failed the test. The test now passes when add_note, given only an email, looks the customer up by it before posting. Otherwise it still requires the instructions fragment, now in a test renamed "gives the agent both tools and has it look customers up before notes". The check binds config through the extension handle, not the agent's mount, because the mount binds once and later tests depend on when.
• OpenCode still asking. Gemini 3.8 Flash's 013 runs were dropped as infra failures after OpenCode rejected reading .env.local. OpenCode's defaults ask before reading *.env / *.env.* and after three identical calls in a row (doom_loop), as well as for external directories, which fix(benchmarks): let OpenCode reach outside the project in authoring runs #5503 covered. The global config now allows all three. Those were the only ask rules in opencode debug agent build.

Validation

Rebuilt the runs' projects from their published results over an eve init scaffold of the same canary, and ran the old and new 013 graders:

Run	main	This PR
Opus 5.5 r2 (ctx.approval guard)	2 failed	pass
Haiku 5.5 r1 (add_note looks up by email)	1 failed	pass
Sonnet 5.5 r1 (instructions fragment)	pass	pass
Haiku r1 mutated so add_note takes a customer ID with no lookup and no instructions	n/a	fails the lookup test

The 014 support-queue test still passes in the earlier scratch project. For OpenCode: against a mock model that reads the project's .env.local, OpenCode 1.19.4 rejects it with #5503's config and completes with this one. A bash read outside the project still completes.

🤖

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
eve-docs	Ready	Preview, v0	Oct 28, 2026 11:50pm UTC

1 Skipped Deployment

Project	Deployment	Actions	Updated
eve-pkg	Skipped	v0	Oct 28, 2026 11:50pm UTC