Skip to content

fix(input-required-result): report the capability check as untestable when nothing is requested - #442

Open
marcus-kepler-92 wants to merge 1 commit into
modelcontextprotocol:mainfrom
marcus-kepler-92:fix/input-required-capability-check-empty-requests
Open

fix(input-required-result): report the capability check as untestable when nothing is requested#442
marcus-kepler-92 wants to merge 1 commit into
modelcontextprotocol:mainfrom
marcus-kepler-92:fix/input-required-capability-check-empty-requests

Conversation

@marcus-kepler-92

Copy link
Copy Markdown

Narrows #439 item 3. The round-2 consequence of the same case is #440; this PR
does not touch it.

The bug

input-required-result-capability-check guarded its elicitation scan with:

} else if (isInputRequiredResult(result) && result.inputRequests) {

With inputRequests absent, that arm is skipped and the following
isCompleteResult arm is false by construction for an input_required result, so
nothing is recorded and the check passes. inputRequests: {} passes by a different
path — the arm is taken ({} is truthy), the loop just has nothing to scan.

Either way the check reports SUCCESS without having exercised the requirement it
scores.

Why notTestable() rather than a plain assertion

inputRequests is optional — a result carrying only requestState still satisfies
"at least one of inputRequests or requestState" — so a server that names none
has not violated sep-2322-respect-client-capabilities
("Servers MUST NOT send an inputRequests that the client has not declared support
for in its capabilities"
). It has not exercised it either.

That is the #248 / #372 case, so the check now reports through notTestable() with
details.untestable, per AGENTS.md and the existing call sites in stateless.ts.
Status stays FAILURE — the severity follows the underlying MUST NOT — but the
message says the requirement could not be verified rather than claiming a violation
the server did not commit.

Does this fail a compliant implementation?

No. The scenario's description is an explicit fixture contract: implement
test_input_required_result_capabilities, read
_meta['io.modelcontextprotocol/clientCapabilities'], and return inputRequests for
the declared capabilities. The probe declares sampling: {}, so a server following
that contract returns a sampling/createMessage request and still scores SUCCESS
(verified below).

What now goes red is a server that implements the tool but returns no input
request — i.e. the fixture prerequisite is missing, which is exactly what #248
says should be red, with the expected-failures baseline as the escape hatch. It is
still a behaviour change for anyone currently green on that basis.

Fixture

Adds the case to sep-2322-mrtr-broken-server. The fixture returns a conformant
envelope — requestState present, no input requests — rather than a bare
{"resultType":"input_required"}, so it isolates this check instead of also
tripping sep-2322-request-state-incomplete.

Verified

Against --spec-version 2026-07-28:

everything-server              Passed: 2/2, 0 failed, 0 warnings

sep-2322-mrtr-broken-server    FAILURE  RespectClientCapabilities
                               Not testable: server returned no inputRequests,
                               so the capability restriction was never exercised

The negative test pins the slug, the Not testable: prefix and
details.untestable rather than just asserting a failure count.

npm run lint, npm run typecheck and the full vitest run (43 files / 505
tests) pass.

… when nothing is requested

`input-required-result-capability-check` guarded its elicitation scan with
`isInputRequiredResult(result) && result.inputRequests`. When a server answered
without `inputRequests`, that arm was skipped, and the following
`isCompleteResult` arm is false by construction for an input_required result, so
nothing was recorded and the check passed. An empty `inputRequests` object passed
too: the arm was taken, but the loop had nothing to scan.

`inputRequests` is optional — a result carrying only `requestState` still
satisfies "at least one of inputRequests or requestState" — so such a server has
not violated "servers MUST NOT send an inputRequests that the client has not
declared support for in its capabilities". It has not exercised it either.
Reporting SUCCESS makes an unverified MUST read as green, which is the failure
mode modelcontextprotocol#248 and modelcontextprotocol#372 established should be reported as a failure instead.

So report it through `notTestable()` with `details.untestable`, matching the
convention in AGENTS.md and the call sites in stateless.ts, rather than claiming a
violation the server did not commit. Servers that legitimately have nothing to ask
for now go red on this check; per modelcontextprotocol#248 the expected-failures baseline is the
escape hatch.

Adds the case to sep-2322-mrtr-broken-server so the existing MRTR negative suite
covers it. The fixture returns a conformant envelope (`requestState` present, no
input requests) rather than a bare one, so it isolates this check rather than also
tripping sep-2322-request-state-incomplete.

Verified against --spec-version 2026-07-28:

  everything-server            2/2 passed
  sep-2322-mrtr-broken-server  FAILURE  Not testable: server returned no
                               inputRequests, so the capability restriction
                               was never exercised

Refs modelcontextprotocol#439
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant