Skip to content

feat: add logging conformance scenarios (capability, invalid level, threshold filtering) - #449

Open
elang2 wants to merge 1 commit into
modelcontextprotocol:mainfrom
elang2:feat/logging-setlevel-conformance
Open

feat: add logging conformance scenarios (capability, invalid level, threshold filtering)#449
elang2 wants to merge 1 commit into
modelcontextprotocol:mainfrom
elang2:feat/logging-setlevel-conformance

Conversation

@elang2

@elang2 elang2 commented Aug 9, 2026

Copy link
Copy Markdown

Summary

Adds three new conformance scenarios for the logging/setLevel endpoint (issue #435):

  1. logging-capability-advertisement — verifies the server advertises capabilities.logging in its initialize response
  2. logging-invalid-level — verifies the server rejects unrecognized severity levels (e.g. "banana") with -32602 (Invalid Params)
  3. logging-threshold-filtering — verifies that notifications/message respects the configured severity threshold (suppresses below, allows above)

Scenarios 2 and 3 are registered in pendingClientScenariosList because the everything-server does not implement invalid-level rejection or threshold-based filtering. They can be run targeted against a compliant fixture server via npm start -- server --scenario logging-invalid-level --url <fixture>.

Scenario 1 is in the active list and passes against the everything-server.

Spec references

Test plan

  • npm run build passes
  • logging-capability-advertisement passes against everything-server
  • logging-invalid-level correctly skipped (pending) in default suite
  • logging-threshold-filtering correctly skipped (pending) in default suite

Closes #435

…, and threshold filtering

Addresses modelcontextprotocol#435 by adding three new conformance scenarios that test
logging/setLevel behavior beyond the existing basic happy-path check:

- logging-capability-advertisement: verifies server advertises logging
  in capabilities before emitting notifications
- logging-invalid-level: verifies -32602 rejection for unrecognized
  severity levels
- logging-threshold-filtering: verifies notifications respect the
  configured severity threshold (suppresses below, allows above)

All scenarios target 2025-06-18 through 2025-11-25 (removedIn draft).
@elang2

elang2 commented Aug 9, 2026

Copy link
Copy Markdown
Author

@joy7758 would appreciate your review on this one when you get a chance.

@joy7758

joy7758 commented Aug 10, 2026

Copy link
Copy Markdown

Thanks for the follow-up. I reviewed current head 79acb3987aec6ca0acf32264abbfeba250c0779f. I am not proposing a competing implementation.

I found four scope/evidence points worth resolving before these checks are treated as conformance results:

  1. Threshold filtering does not currently have RFC-keyword backing. logging.ts:L256-L262 labels suppression as MUST NOT and emission at debug as SHOULD. The cited 2025-11-25 logging page says clients MAY send logging/setLevel, but I cannot find either receiver requirement there. Unless there is another normative sentence to cite, these should be treated as a non-scoring interoperability observation or an upstream spec gap, not enforced as protocol conformance.

  2. Capability advertisement is conditional. logging.ts:L71-L95 warns whenever capabilities.logging is absent. The spec requires that capability only for a server that emits log notifications. A server that does not implement logging can therefore be compliant. This check needs either evidence that the server emits/supports logging or a non-scoring outcome when the feature is not claimed.

  3. Zero notifications should not silently become a neutral result after the fixture contract is invoked. logging.ts:L293-L305 returns SKIPPED. If test_tool_with_logging is a controlled prerequisite that promises at least an error-level sentinel, zero notifications means the prerequisite/evidence path failed, so the threshold was not tested. I would use the suite's explicit inconclusive/prerequisite-failure convention rather than SKIPPED; if the server is allowed to emit nothing, this behavior is simply unobservable and should not be scored.

  4. The invalid-value probe still needs wire evidence. Registering it as pending keeps it out of the default suite, but does not by itself show that a targeted run avoids wire-schema-harness-error. Please include the targeted checks.json/wire-schema output for the intentionally invalid request.

The repository contribution guide also asks for a passing real-SDK run and a specific failing case. The PR currently documents the in-repo everything-server path, so adding the pinned real-SDK output plus a negative fixture/result would complete the evidence loop.

The -32602 handling itself is correctly scoped as a SHOULD. No protocol change is being suggested here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add conformance scenario: logging/setLevel and log notifications

2 participants