Every other provider failure in this extension fails open and says so — notifyError puts (failing open) on screen. A response whose answers map is present but empty is the one case that fails open silently, and the status line reports it as a judgement that was made.
normalizeResponse requires answers to exist but not to contain anything, so {"model":"jev-latest","answers":{}} is a valid response. evaluateGate then reads each missing noul as 0:
const destructive = response.answers.destructive?.type === "noul"
? response.answers.destructive.noul
: 0;0 is not "unknown", it is "certainly not destructive". The verdict comes back flagged: false, the status line reads jev: clear (enforce), and /jev last will show a verdict no model produced.
Reproduction
Driving askJev + evaluateGate at 80a80f3 against a local stand-in on 127.0.0.1:9187, with state = a bash call running rm -rf / --no-preserve-root:
# stand-in returns {"model":"jev-latest","answers":{}}
raw answers from provider: {}
verdict: {"flagged":false,"reasons":[],"destructive":0,"exfiltration":0,"beyondScope":0}
End to end inside pi, gate in enforce with blockWithoutUI: true — the mode where a flagged call is supposed to be stopped — the command ran:
== empty answers : jev requests = 2
-- data.txt still there? NO
For comparison, the same run against a provider returning {"model":"jev-latest"} with no answers key at all takes the intended path: normalizeResponse throws response is missing the answers map, notifyError fires, and the failure is visible.
Suggestion
Treat a response that answers none of the questions it was asked as a failed request rather than as four zeros. Something like: if no id in GATE_QUESTIONS is present in response.answers, throw a JevError from normalizeResponse (or have evaluateGate return undefined), so it reaches notifyError and the operator sees (failing open) like every other failure.
A smaller, related gap in the same place: isJevAnswer checks only the discriminant field, so a choice answer with no confidence or no probabilities passes validation and then throws downstream — describeAnswer on such an answer raises TypeError: Cannot read properties of undefined (reading 'toFixed'). In jev_ask the try/catch turns that into a tool error, so nothing crashes, but the message the model gets is a TypeError rather than "the provider sent an answer we cannot read".
Found while reviewing pi-jev for mrjev.com, an independent directory of Jev projects. No Jev API calls were made — everything above is against a local stand-in.
Every other provider failure in this extension fails open and says so —
notifyErrorputs(failing open)on screen. A response whoseanswersmap is present but empty is the one case that fails open silently, and the status line reports it as a judgement that was made.normalizeResponserequiresanswersto exist but not to contain anything, so{"model":"jev-latest","answers":{}}is a valid response.evaluateGatethen reads each missing noul as0:0is not "unknown", it is "certainly not destructive". The verdict comes backflagged: false, the status line readsjev: clear (enforce), and/jev lastwill show a verdict no model produced.Reproduction
Driving
askJev+evaluateGateat 80a80f3 against a local stand-in on127.0.0.1:9187, withstate= abashcall runningrm -rf / --no-preserve-root:End to end inside pi, gate in
enforcewithblockWithoutUI: true— the mode where a flagged call is supposed to be stopped — the command ran:For comparison, the same run against a provider returning
{"model":"jev-latest"}with noanswerskey at all takes the intended path:normalizeResponsethrowsresponse is missing the answers map,notifyErrorfires, and the failure is visible.Suggestion
Treat a response that answers none of the questions it was asked as a failed request rather than as four zeros. Something like: if no id in
GATE_QUESTIONSis present inresponse.answers, throw aJevErrorfromnormalizeResponse(or haveevaluateGatereturnundefined), so it reachesnotifyErrorand the operator sees(failing open)like every other failure.A smaller, related gap in the same place:
isJevAnswerchecks only the discriminant field, so achoiceanswer with noconfidenceor noprobabilitiespasses validation and then throws downstream —describeAnsweron such an answer raisesTypeError: Cannot read properties of undefined (reading 'toFixed'). Injev_askthetry/catchturns that into a tool error, so nothing crashes, but the message the model gets is aTypeErrorrather than "the provider sent an answer we cannot read".Found while reviewing pi-jev for mrjev.com, an independent directory of Jev projects. No Jev API calls were made — everything above is against a local stand-in.