Skip to content

feat(mcp): opt-in captureModel — agent-self-reported model id on MCP events #4633

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
DanielVisca wants to merge 3 commits into
base: main
Choose a base branch
from
+772 −62
Open

feat(mcp): opt-in captureModel — agent-self-reported model id on MCP events #4633

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
586ff7b
feat(mcp): add opt-in captureModel option for agent-self-reported mod…
DanielVisca Aug 24, 2026
a960d92
fix(mcp): inject llm_model into the virtual tool, share the parameter…
DanielVisca Aug 25, 2026
6c10c1b
fix(mcp): resolve virtual-tool parameter ownership without the tools/…
DanielVisca Aug 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/model-capture-option.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@posthog/mcp': minor
---

Add opt-in `captureModel` option: injects a required `llm_model` parameter into every tool — including the `get_more_tools` virtual tool — so the calling agent self-reports the model it runs as, captured as `$mcp_llm_model` with `$mcp_llm_model_source = "self_reported"`. Stripping the argument before the handler runs and capturing the property both require confirmed SDK ownership of the parameter: a customer-declared `llm_model` is never stolen or captured, and a low-level `Server` that builds a fresh instance per request records nothing (see the README). An honest `"unknown"` from the agent is dropped rather than recorded.
26 changes: 26 additions & 0 deletions packages/mcp/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,32 @@ instrument(server, posthog, { // keep it, dr

`intentFallback` is the third option: supply the intent yourself when the agent did not send one.

### What `$mcp_llm_model` records, and when it stays empty

`captureModel` is **off** by default. Turn it on and the SDK adds a required `llm_model` parameter to
every tool it advertises — including the `get_more_tools` virtual tool — asks the agent which model
it runs as, and records the answer as `$mcp_llm_model` with `$mcp_llm_model_source = "self_reported"`.

The value is self-reported and unverified, exactly like `clientInfo` in the MCP spec. Use it to spot
degradation across models ("does our MCP get worse on model X?"), never for billing or access
control. An agent that answers `unknown` is recorded as nothing rather than as a model called
"unknown".

Unlike `context`, this option degrades to **silence** rather than to a kept argument. Both the strip
and the capture require the SDK to have confirmed the parameter is its own:

- `instrument(server)` on a high-level `McpServer` resolves ownership per request from the live tool
registry, so it works even on a fresh instance.
- Instrumenting a low-level `Server` learns ownership while serving `tools/list`. On a server that
builds a fresh instance per HTTP request — `createMcpHandler`, or `@rekog/mcp-nest` in its
stateless mode — the instance handling a `tools/call` never served one, so it neither strips
`llm_model` nor records `$mcp_llm_model`. Nothing breaks and no wrong value is stored; the property
is simply absent while agents still pay a token for the extra field.

As with `context`, what matters is instance lifetime rather than statelessness: a transport-stateless
server (`sessionIdGenerator: undefined`) that keeps one long-lived server object learns ownership
from the first `tools/list` and keeps it.

### If you switched to `instrument(server.server)`

Before v2 support landed, the compatibility gate rejected high-level v2 servers, and the usual
Expand Down
5 changes: 4 additions & 1 deletion packages/mcp/docs/ARCHITECTURE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ All events are emitted by `buildPostHogCaptureEvents`. The main event name is co

| PostHog event | When | Notable extras |
| ------------------------- | ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `$mcp_tool_call` | Every tool invocation | `$mcp_tool_name`, `$mcp_tool_description`, `$mcp_tool_category`, `$mcp_parameters`, `$mcp_response`, `$mcp_duration_ms`, `$mcp_is_error`, optionally `$mcp_intent` / `$mcp_intent_source` |
| `$mcp_tool_call` | Every tool invocation | `$mcp_tool_name`, `$mcp_tool_description`, `$mcp_tool_category`, `$mcp_parameters`, `$mcp_response`, `$mcp_duration_ms`, `$mcp_is_error`, optionally `$mcp_intent` / `$mcp_intent_source`, `$mcp_llm_model` / `$mcp_llm_model_source` |
| `$mcp_tools_list` | Client lists tools | `$mcp_listed_tool_names` (array of tool names advertised); useful for "did this client discover us?" and "which advertised tools never get called?" |
| `$mcp_initialize` | Client/server handshake | `$mcp_client_name`, `$mcp_client_version`, `$mcp_server_name`, `$mcp_server_version`, `$mcp_protocol_version` (negotiated MCP spec version — for tracking spec-revision adoption) |
| `$mcp_missing_capability` | Agent calls the `get_more_tools` virtual tool | A capability gap, **not** a tool invocation. The `context` arg is captured as `$mcp_intent` with `$mcp_intent_source = "context_parameter"` |
Expand Down Expand Up @@ -141,6 +141,8 @@ All wire keys live in `PostHogMCPAnalyticsProperty` (`src/extensions/constants.t
| `Intent` | `$mcp_intent` | string | `context` argument when present, else `intentFallback()` return |
| `IntentSource` | `$mcp_intent_source` | `"context_parameter" \| "inferred"` | Where the intent came from |
| `ConversationId` | `$mcp_conversation_id` | string | Optional; set when `enableConversationId: true` and the SDK owns the tool's injected `conversation_id` parameter |
| `LlmModel` | `$mcp_llm_model` | string | Optional; set when `captureModel` is enabled and the SDK owns the tool's injected `llm_model` parameter. The calling agent's **self-reported, unverified** model id — right for degradation analytics ("does our MCP get worse on model X?"), never for billing or security. A blank or `"unknown"` answer is dropped rather than recorded |
| `LlmModelSource` | `$mcp_llm_model_source` | `"self_reported"` | How the model id was obtained. Always `self_reported` today — it exists so a future verified source stays distinguishable |
| `Parameters` | `$mcp_parameters` | object | Sanitized MCP request payload (see §3) |
| `Response` | `$mcp_response` | object | Sanitized tool result |

Expand Down Expand Up @@ -171,6 +173,7 @@ The `posthog-node` client is **not** an option — it is the required positional
| `enableConversationId` | `false` | Inject `conversation_id` into eligible tools that don't already declare it and stamp `$mcp_conversation_id` on their events. |
| `reportMissing` | `false` | Register the `get_more_tools` virtual tool. |
| `context` | `true` (object form: `{ description }`) | Inject required `context` arg into every tool schema. |
| `captureModel` | `false` (object form: `{ description }`) | Inject a required `llm_model` arg into every tool schema (including the `get_more_tools` virtual tool) so the agent self-reports the model it runs as → `$mcp_llm_model`. Capture and stripping both require confirmed SDK ownership, so a fresh-instance-per-request server that never served the `tools/list` records nothing (see README). |
| `intentFallback` | — | Consumer-supplied callback returning a `$mcp_intent` string when the client didn't pass a `context` argument. SDK does no inference. |
| `identify` | — | Per-request callback returning `{ distinctId, properties?, groups? } \| null` — posthog-node's `identify` shape. `properties` → `$set`, `groups` → `$groups`. |
| `beforeSend` | — | `(event) => event \| null \| undefined` (sync or async), matching posthog-node. Runs on each fully-built payload right before `posthog.capture()` — once per emitted event, including the `$exception` sibling. Return nullish (or throw) to drop that event. |
Expand Down
Loading