Skip to content

docs(slack): decide when to reply in a thread with a decision model - #5501

Draft
benpankow wants to merge 1 commit into
mainfrom
docs/slack-decide-ambient-replies
Draft

benpankow wants to merge 1 commit into
mainfrom
docs/slack-decide-ambient-replies

Conversation

@benpankow

Copy link
Copy Markdown
Collaborator

Documents how a Slack agent can listen in a thread without a mention on every message, using decide from eve/ai inside onMessage.

Why

The Slack docs show two policies for un-mentioned thread replies: answer all of them (isSubscribed()), or route by participants. Neither covers the common case: keep listening in a thread the agent has joined, but reply only when a message is actually meant for it. That works today with decide in a message hook, but nothing in the docs shows it, so people assume it isn't supported.

Approach

Adds a "Decide when to reply in a thread" section to docs/channels/slack.mdx, after the existing mention/subscription examples. The example:

  • always replies to mentions, DMs, and private channels
  • ignores top-level messages and threads without an active eve session
  • otherwise passes the last ten thread messages plus the new one to decide (defaults to typesafe-ai/jev) as one boolean question, and accepts above a 0.7 probability
  • stays quiet if the decider fails, so a mention remains the fallback

Also adds a one-line cross-link from docs/guides/decide.md, where it already says decide works outside a tool.

Validation

  • Typechecked the snippet against packages/eve/src public eve/ai and eve/channels/slack types (no errors; a deliberate bad property access in the same file errored as expected, confirming the types resolved).
  • oxfmt applied to both files.
  • node scripts/check-docs.mjs: ok. The snippet and MDX checks could not run locally because the full workspace install was incomplete; leaving those to CI.

Signed-off-by: Ben Pankow ben.pankow@vercel.com

Signed-off-by: Ben Pankow <ben.pankow@vercel.com>
@vercel

vercel Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
eve-docs Ready Ready Preview, v0 Oct 28, 2026 4:14am UTC
eve-pkg Ready Ready Preview, v0 Oct 28, 2026 4:14am UTC

Comment thread docs/channels/slack.mdx
Comment on lines +256 to +257
from: m.isMe ? "agent" : (m.user ?? "other-bot"),
text: m.markdown,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In threads longer than 50 messages, refresh() followed by slice(-10) supplies the decider with messages 41–50 instead of recent context, omitting later agent replies needed to interpret follow-ups.

Fix on Vercel

Comment thread docs/channels/slack.mdx
Comment on lines +254 to +258
await ctx.thread.refresh();
const thread = ctx.thread.recentMessages.slice(-10).map((m) => ({
from: m.isMe ? "agent" : (m.user ?? "other-bot"),
text: m.markdown,
}));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This window isn't the last ten messages before the new one, which is what the prose on L284 promises.

  • refresh() loads the thread's first page of 50 replies, oldest first (THREAD_REFRESH_LIMIT, refreshMessages). Once a thread passes 50 messages, slice(-10) returns messages 41 to 50 forever, so the decider judges the new message against stale context. Long, busy threads are exactly where ambient listening runs.
  • In shorter threads the page already includes the triggering message, so thread ends with the same message that latest repeats. loadThreadContextMessages strips it for this reason (thread.ts#L52-L58).

The minimal fix is to filter out the current message and state the 50-message cap in the prose:

const thread = ctx.thread.recentMessages
  .filter((m) => m.ts !== message.ts)
  .slice(-10)
  .map((m) => ({ from: m.isMe ? "agent" : (m.user ?? "other-bot"), text: m.markdown }));

If long threads need to work, the real fix is a public recent-window helper. fetchSlackThreadWindow already does this internally, but it isn't exported.

Comment thread docs/channels/slack.mdx
Comment on lines +260 to +279
try {
const { answers } = await decide({
state: { thread, latest: { from: message.author.userId, text: message.markdown } },
questions: {
reply: {
type: "boolean",
instructions: "Should the agent reply to the latest message in this Slack thread?",
criteria: {
true: "It asks the agent something, follows up on the agent's last reply, or asks it to do work.",
false:
"It is addressed to other people, is banter or thanks, or needs nothing from the agent.",
},
},
},
});
return answers.reply.probability >= 0.7 ? { auth } : null;
} catch {
// Stay quiet when the decider is unavailable; a mention still works.
return null;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can drop this try/catch. eve already catches a thrown onMessage error, logs channel_message handler failed, and drops the message (slackChannel.ts#L1709-L1720), so a decider failure already stays quiet. The bare catch {} does the same thing but loses that log, so an outage of the decision model would look like an agent that has stopped replying. Without it, the snippet loses a level of nesting and five lines. If you keep the sentence about falling back to a mention, move it to the prose.

Comment thread docs/channels/slack.mdx
Comment on lines +247 to +248
// Mentions and DMs always get a reply.
if (ctx.isBotMentioned() || (await ctx.isDMOrPrivateChannel())) return { auth };

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment says "Mentions and DMs", but isDMOrPrivateChannel() is also true for private channels and group DMs (privacy.ts#L9-L12). So in those conversations, every message skips decide and gets a reply, including thread replies between humans. That is the talk-over case this section exists to prevent. If only 1:1 DMs should bypass the decider, check that conversation type specifically, for example message.raw.channel_type === "im" (mentions are already covered by isBotMentioned()). If the bypass is intentional, update the comment and mention it in the prose.

Comment thread docs/channels/slack.mdx
});
```

The decider sees only the state you pass, here the last ten thread messages and the new one, so trim what you send to what the decision needs. Raise the threshold to make the agent quieter. A message the hook drops never reaches the agent, and returning `{ auth }` starts or steers a turn like any other accepted message. This needs the same `message.channels` event and `channels:history` scope as [continuing conversations without repeated mentions](#continue-conversations-without-repeated-mentions).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"A message the hook drops never reaches the agent" isn't true for this snippet. L242 sets threadContext: { since: "last-agent-reply" }, so the next accepted message injects every message since the agent's last reply as context, including the ones the hook dropped. That is probably why threadContext is in the example, but the prose never explains it, so it reads like an unrelated setting.

Suggest something like: "A dropped message starts no turn. Because threadContext loads replies since the agent's last message, the agent still sees the skipped conversation when it next replies." Also update "the last ten thread messages and the new one" to match the window fix on L254 to L258.

This branch was successfully deployed

2 active deployments
Preview – eve-docs — a2d3b577 Deployed Oct 28, 2026 by vercel[bot]
Preview – eve-pkg — a2d3b577 Deployed Oct 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant