TASIP-003: Human-in-the-loop steps

[ryw]

Status: Draft — open for input · Category: Big bet

Summary

Let an agent pause mid-run to ask a person for a decision or input through a structured form, then resume with the answer — unlocking workflows that need human judgment at a checkpoint (approval, disambiguation, sign-off) without abandoning automation.

Why

Plenty of useful work is "mostly automatable" with one human gate in the middle. Today you either fully automate (and accept the risk) or don't automate at all. A clean pause/resume with real forms is the missing middle.

Sketch

• An agent can emit a "need input" step with a schema (fields, choices, free-text).
• The run pauses; the request shows up for a human to act on.
• On submit, the answer is injected back and the run continues from where it paused.
  (Provisional — the shape changes based on the discussion.)

Open questions

• How long can a run stay paused, and what happens on timeout?
• Who can answer — anyone, the owner, a named approver?
• How does this interact with schedules and Slack-launched runs?

How to help

React 👍 if you want this, and comment with your use case, constraints, or pushback. This is a draft Tembo Agent Studio Improvement Proposal (TASIP); your input shapes it before it becomes scheduled work.

[kevine824-alt]

@ryw

Would be great addition to TAS overall. A lot of our agents are built around not having it, at Carefeed and on our own instance: the Carefeed follow-up agent drafts and stops so a rep can send, the Carefeed onboarding tracker ends with a "needs your confirmation" list, and our Avoma sync moves a deal and posts a reversible note after the fact. Each of those is a human checkpoint we fake by ending the run.

Thoughts from that experience:

• Batch the asks. A run that pauses ten times for ten drafts won't get used. One form with all ten (approve, edit, skip on each) will. Same for a weekly tracker with eight items to confirm.
• Answer where people work. For the CX team that's Slack. If the form only lives in TAS, a lot of requests will sit. A Slack form that resumes the run when answered would get used.
• Paused runs shouldn't cost anything while they wait. State saved, not running.
• Security and oversight. This is the cleanest answer we'd have to "what if an agent goes rogue." Tool limits decide what an agent can do, and a pause before anything irreversible (send, payment, delete) puts a person on the final step. For customers worried about agents acting on their own, that's the story: bounded by design, and a human signs off where it matters.
• Supervised mode for testing. Today it's dry run, then live. Dry run only stubs the declared delivery tools, so other write tools still write. A supervised mode in between, where every write pauses for approval, would let a new agent run on real data while a person checks each action before it happens. For sensitive data, a policy template could require a set number of supervised runs before an agent can go live. That ties this to the policy template and certification work.

On the open questions:

• Timeout - set per step, with a declared safe default when nobody answers, usually skip and report it. Never default to the risky action. For scheduled agents, the timeout should land before the next run fires.
• Who answers - named in the step: the run-as person by default, a named person or role when it matters. Option (ties to the policy template work) to require someone other than the requester for send/write approvals. Who answered goes in the audit log.
• Schedules - if a paused run is still waiting when the next one fires, don't stack them. Skip the new run or fold its items into the open request, and say which happened.
• Slack-launched runs - answer in the same thread.

One safety note: whatever comes back in the form should be treated as data for that step, checked against its schema, never as new instructions to the agent.

Happy to help test it on a Carefeed agent, the follow-up drafts would be a good first case.