Skip to content

Latest commit

 

History

History
216 lines (181 loc) · 91.9 KB

workflow-engine.md

File metadata and controls

216 lines (181 loc) · 91.9 KB

Workflow engine

Maintainer notes on src/workflow/. The workflow file format and the driver contract are in the Workflows and Advanced workflows guide pages.

Each rule below spans more than one file. The rule names the code that owns it, so you can read the details there.

Rules that span files

  • A run ends only in settleRun. The shared finishRun path calls it when settling reaches the top workflow's result, before it hands any lease back or out, and on a call that takes no lease too. reportAttempt first reconciles earlier reports, so an end cancels a later report before it can advance the revision. It records an accepted outcome and never ends the run. After a failed report, the next next skips the dependents and ends the run. Respond, cancel and retry in run-controls.ts, and next in engine.ts, share that path. Whole-run cancel settles directly as cancelled, without reconciling another round. To run code when a run ends, hook into finishRun. Do network I/O only after withRun releases the run lock. That path queues the end record, a local write, under the lock, and saves whether it did in the run's endRecordQueued field. When updateRun then fails to save the run file, the queue is one save ahead of the run file, and the next next settles the run again and overwrites the queued <run>.end.json. A later next reports a record state only for a run whose field is true (runRecordState in records.ts).

  • Reconcile the latest report before applying another or handing out a lease. reconcileRun, shared by leaseNext and reportAttempt, settles every driverless transition to a fixpoint at the run's current revision, through settleScope and settleItems, including workflow starts, fan-out expansion, render failures and bound exhaustion. reportAttempt advances the revision only after it, so only the latest revision can be unreconciled. A reached end applies before a loop decision. Only after reconciliation does leaseNext hand back an omitted open lease or issue the first ready leaf in declaration order that this driver can take. A running workflow occupies its parent's position in that order. Settling returns every ready leaf in that order (ready), and leaseNext walks it. A ready item does not count against max_parallel until it is leased, and one next leases at most one step, so a held-back item leaves its room to the next one without exceeding the cap. canTakeLease: false still reconciles driverless transitions. Every changed run is saved, including a wait.

  • A report is checked against the revision of its own lease. reportAttempt and retryStep increment revision; skips, if, end, and the creation of child steps inside next never do. A lease keeps the run revision it was handed out at, and reportAttempt compares a report with that, so the report of a parallel lease does not make it stale. An attempt that is not an open lease keeps the older answers: stale_revision when its revision is not the run's, otherwise wrong_attempt.

  • An end or break stops what is still open under its workflow. stopWhere cancels every leased step under the workflow that reached the end, its sub-workflows included, and keeps the attempt on the step and in cancelledAttempts. It cancels running workflow steps there and skips pending and waiting ones. The walk takes a path predicate and failure. End and break construct their own end_reached failure; manual cancel and retry supply theirs. Step descendants use the same isUnder predicate as archiveUnder in loop.ts. A cancelled delivering step keeps its agent settlement and its original settledAt for explicit recovery, records and scoring. Automatic delivery and workflow-item branch summaries exclude cancelled children. Cancel records the ids this walk changed before reconciliation can settle or skip other steps. reportAttempt answers a cancelled attempt lease_cancelled before it checks the driver; cancelledAttempts is never trimmed. Every next answer lists in cancelled the attempts the driver says it runs that are no longer its open leases, so the driver learns of a cancellation from any answer, done included.

  • The local driver overlaps steps that do not share a checkout. runLocalDriver runs up to workflows.max_parallel steps (local config, default 1) and sends the ones it has started as running. A command with no workspace, and a prompt in the current checkout, share that checkout, so only one of them runs there at a time. A prompt earlier in the queue for that checkout blocks a later command there. The driver starts the prompt when the checkout is free and fewer than max_parallel steps are in flight; commands in other worktrees keep running. A prompt with its own worktree does not take the checkout. A headless prompt takes one of the slots, and the driver keeps leasing steps for the others while it runs. An interactive prompt uses the terminal, starts only when no command is running, and no step is leased while it runs. On cancelled, it drops a held lease, and it stops a running command with SIGTERM, then SIGKILL after 5 s while any captured descendant or process in a detached group remains, even after its command exited. Attached commands snapshot their descendant tree and start times before stopping, then probe and send SIGKILL only to descendants whose start times still match. If the identity snapshot fails, only the shell is stopped; detached commands use their process group. It does not send a stopping command as running again, so next does not cancel it twice. Dying commands still take a slot, and a new attempt of their step waits until they are gone. The signal policy follows max_parallel. At 1, a command runs in teamai's process group, as env exec runs one (src/env-exec.ts): the terminal's Ctrl-C reaches it, and it can read /dev/tty. A signal env exec would pass on (SIGTERM or SIGHUP, and SIGINT or SIGQUIT when the terminal does not reach the command) makes the driver stop the command, wait for it, and exit with that signal. Above 1, each command runs detached in a process group of its own, recorded in the attempt's group-<pid>.json as setup's is, with no terminal, and SIGINT, SIGQUIT, SIGTERM or SIGHUP makes the driver stop every group, wait for them, and exit with that signal. Windows uses taskkill /T /F in place of the group kill. Staging runs inside the attempt at every max_parallel (stageThenStart), so the driver keeps polling while it stages, and a cancel aborts staging. A step whose attempt was cancelled, or whose lease expired, while it staged never starts. runPlaceSetup in place/place.ts runs setup (setup.ts) after the git phase, only in a worktree prepare just created, with setup's own keys, and removes the worktree when setup fails. It records setting-up before setup starts, and a prepare that finds that phase gives the place up as a failed setup does, so a second setup is never checked against what one cut short left there. Before it stages an attempt the engine hands back, the driver stops every group a killed driver's setup or command left there (stopRecordedGroup), each recorded in the attempt's own group-<pid>.json.

  • A workspace is a git contract on the lease. TeamAI owns the workspace contract and its git; the host owns the place. TeamAI reads the delivered commit and opens the PR outside the run lock. place/ prepares and finishes every place: the local driver prepares through place.ts in process, and next --prepare through protocol.ts for a driver that runs the protocol from the CLI; every driver's report finishes and releases its place through managedReport in protocol.ts. preparePlace records each place in attempts/<attempt>/place.json by phase (opening, preparing, staged, setting-up, ready, finished, abandoned), with the reader's snapshot in before.json and the branch's agent files in overlay/ beside it, so whichever process reports the attempt finishes it. It records opening before git touches the copy, with whether the copy existed and, once it is open, a writer's range start, taken before any merge, so a prepare killed in git is finished by the next prepare of the attempt, which still runs setup in a copy the killed one created and keeps its merges in the step's changes. It records preparing before it touches the agent files, and copies them aside to overlay.partial/, renamed to overlay/ once complete, so a prepare killed during the overlay is finished by the next prepare of the attempt, or put back by abandonPlace. next --prepare leases and prepares under the run lock, and a prepare that fails settles the attempt failed with its own code. report refuses caller changes and read for a prepared attempt, finishes its place before it reports, and leaves the place as prepare found it (abandonPlace) when the attempt is released or the run answers lease_cancelled. planWorkspace in workspace.ts chooses the mode. An explicit workspace wins. A command that omits it is read-only, even inside a copy; when it exits 0 and changed the checkout, the driver fails it read_only_dirty. That includes a commit that leaves the worktree clean, a switch to another branch or a detach at the same commit, a change that only stages an existing edit, another staged hunk that leaves the status mixed, a permission change on a file that was already dirty, an edit to a quoted non-ASCII path, a symlink whose target changes while the file it names stays the same, and an edit to the destination of a staged rename, and a change inside a submodule that was already dirty. An interactive prompt that sets workspace: read-only is checked the same way. An interactive prompt that does not may change the checkout, and those edits stay. Conflict paths, including agent files, are read without Git's quoting, so a non-ASCII name is still checked for leftover markers. A prompt that omits it continues the nearest enclosing new or continue workflow, and a for_each ancestor makes that new, so items do not share a worktree. leaseNext puts the contract on the lease for new and continue, and for a read-only step inside a copy (mode: 'read-only', merge: []). That reader's branch is the one its declared needs wrote or read (changes or read, through items and children, but not after items or inherited needs), or the enclosing copy's when they touched none; needs on several branches fail workspace_base. It needs continue in can.workspace and the copy's owner, as continue does. A read-only step outside any copy gets no contract and runs in the active checkout. preparePlace stages it there as a reader, and finishPlace checks it; a second prepare or finish returns the first one's result. place/reader.ts stages a reader in a copy without merges, takes its snapshot over the base overlay, and afterwards restores the overlay; when the step changed the copy or left its branch, it keeps the state on refs/teamai/kept/<attempt> (git add -A, then write-tree and commit-tree on HEAD; not the stash list, which every worktree shares), and pushes each submodule commit that tree records and the read commit does not into the checkout's git dir as refs/teamai/kept-modules/<attempt>/<commit> (at any depth, without the user's push hooks or push.recurseSubmodules), since a gitlink does not keep its commit reachable once the copy is removed, then runs git checkout -f <branch>, git reset --hard to the commit it read, git clean -fd, and git submodule update --checkout --recursive on the submodules it changed, so a later writer never commits a reader's edits. It resets nothing when a repository among the changed paths has uncommitted or untracked files, or a nested submodule commit its parent does not record, which the keep cannot hold. The copy counts as put back only when HEAD is refs/heads/<branch> at that commit and git status --porcelain --untracked-files=all --ignore-submodules=none prints the same lines it printed when the reader was staged; ignored files are left in place. Otherwise the reader fails workspace_base. preparePlace writes a teamai-unrestored marker, with that status, in the worktree's git dir before the reader's overlay, and the driver removes it once the copy is put back or left untouched, so a driver killed mid-reader still leaves it. openWorktree in place/created.ts refuses later leases in a marked copy until the copy is back, when it removes the marker. A status line does not show content, so a marker whose status was not empty stays until the person who puts the copy back deletes it. ensureWorktree also refuses an existing worktree whose HEAD is not on its branch or a directory that is not a worktree, and prunes and recreates one whose directory was deleted. Parallel steps run git in one repository, and git writes a new worktree's entry without a lock, so a command that reads every worktree (fetch, branch, worktree add) beside it dies on the entry half written (failed to read .git/worktrees/<id>/commondir). git in worktree.ts runs such a command again, and ensureWorktree makes the branch before worktree add: worktree add -b makes it before it reads the worktrees, and would find it made when it ran again. The reader reports read (branch, base, commit) and never changes. report returns invalid_report for read on any other lease, and for changes or a success without read on a reader's lease. run.copies[].commit and fan-out branches follow changes only. cleanup comes from the lease's contract, so a reader that was the copy's last user removes it too. leaseNext holds a step back for this driver, and tries the next ready leaf, when the driver's can.workspace lacks that mode, or when a continue copy belongs to another driver (run.copies). It answers wait, with every held step's reason joined by ; , only when every ready leaf is held back. Driver capabilities stay out of reconcileRun, which reportAttempt and the run controls also call. It fails workspace_base when continue has no enclosing new, or when the run has no base. workflow start in a git checkout records the base (commit, and the branch unless HEAD is detached); otherwise the first next that sends base sets it, and a later base never changes it. The local driver fails a lease whose workspace it cannot read with workspace_base instead of running it in the active checkout. The branch is teamai/<runId>/<path-safe step path>/p<pass> (branchName); a [n] in the path becomes -n. The first branch in needs order is where the step starts, and the rest are merged in that order. An item's after branches come first. place/ (created.ts, overlay.ts, worktree.ts) adds the worktree under .teamai/workflow-worktrees/ of the checkout the run started from (run.checkout), or of the driver's own checkout when it does not share that checkout's git directory, such as a separate clone (placeRoot), points core.hooksPath at an empty directory, and replaces AGENT_PATHS (AGENTS.md, CLAUDE.md, .agents, .githooks, and each default toolPaths entry's project directory, project MCP file and Copilot .github paths) with the run's base commit. An agent path under a symlinked parent fails base_config before anything is copied or removed, since both would follow the link. It then gives the worktree the inline pull a new worktree gets (prepareCheckout), which delivers the team's skills, rules, agents and MCP and records that checkout's team revision. The overlay is taken after that delivery, so those files are not the step's commit. A path the step edits is committed. A step that left HEAD off the leased branch fails workspace_base before anything is committed, reported as changes, or pushed, and the worktree stays as it left it. A path it leaves untouched is put back from the branch. A merge conflict stops the merge: a command fails merge_conflict, and a prompt is told the branches that remain. Snapshots and commits read submodules with --ignore-submodules=none, and after git add -A, which skips a submodule whose ignore is all, each submodule is staged by name, so submodule.<name>.ignore and diff.ignoreSubmodules hide no change from a reader's check, its keep, or a writer's commit. The driver commits as TeamAI <teamai@localhost> and reports changes (branch, base, from, to, commits). It pushes when origin exists and this step made commits, whether the step succeeded or failed, and also when a later attempt adds no commit but this branch has commits origin does not have, including a first push that never landed and an origin with no tracking refs yet. A failed push fails a step that succeeded with push_failed, and the worktree stays. report sets cleanup only when the step succeeded and no step that can still run uses that branch, judged on the run as the next next settles it: steps a loop runs again count, and a workflow or for_each step that has not started counts by the branch plannedBranch gives, the one around it unless it opens its own; only then does releasePlace run git worktree remove --force. It does not delete the branch, and it never resets, cleans, or switches the active checkout. A command that starts its own sandbox, such as docker, stays read-only. What it writes does not feed new, branches, or a later step. session: continue and session.from are step reads. The engine puts { tool, id } on the lease, with earlier, the sessions that one continued, newest first, and never copies a session. The driver resumes Claude and Codex by that id, copies another tool through AgentSession.copy, restores a copy from the team session archive when neither works, and tries each earlier session the same way before it starts fresh. OpenCode has no resume flag, so it starts fresh. A driver that continued the lease's session or an earlier one, by that id or in a copy, echoes the one it continued as resumed in its report. One that started a new session leaves resumed out. reportAttempt answers invalid_report for a resumed on a step that resumes no session, and for one that no pass of the run reported and no link continued (knowsSession). When the reported tool and session differ from resumed, reportAttempt appends { from: resumed, to } to run.sessionContinuations. sessionOf follows those links from the session a step would resume, taking the latest link from each session, so a later step resumes the copy that holds the last turns, whichever machine or tool made it. The snapshot publishes the links, so resumableSessions on this machine and the claim on the next one both see them. A step that resumed by the same id adds no link, so a run on one machine with one tool leases the same sessions as without links.

  • A driver pushes the commits it reports, and fetches the commits it is leased. Every driver keeps these two git rules, so a run another driver or another machine continues finds each commit on origin. Whenever origin exists, a driver pushes the commits it reports in changes before it reports them, failed steps included; finishWriter in place/writer.ts pushes for every outcome. A push that fails still reports the commits. A step that succeeded fails push_failed. A step that failed keeps its own failure, and the push sentence is added to its message: a retry must fix the step first, and a push_failed in its place would hide that cause from the person and from the run record. Before it starts from, merges, or applies a lease commit (start, each merge[] entry, configCommit), a driver checks it with git cat-file -e <c>^{commit}. For a missing one it fetches +refs/heads/teamai/<run>/*:refs/remotes/origin/teamai/<run>/*, and checks again, since a refspec fetch that matches no branch exits 0. A fetch by commit id is not used: a server can refuse it, and a fetched commit no ref holds can be collected later. A commit still missing fails workspace_base naming the commit and its branch, before anything merges or runs, so it is never a file-less merge_conflict or a prompt told to resolve one. openWorktree in place/created.ts does this through fetchMissingCommits, for the local driver and for next --prepare, and adoptedMismatch in place/adopted.ts for a place the host made. workflowGitConformanceScenarios in conformance/git.ts checks both rules against a bare origin, and workflow-git-conformance.test.ts runs them against the local driver.

  • next delivers pull requests, outside the run lock. A deliver: pr step whose agent succeeded becomes delivering (startDelivery), holding delivery (branch, head, and a kept reported URL). It is neither stopped() nor settled(), so dependents wait, and startedItems does not count it, since max_parallel bounds agents. Every leaseNext calls drainDeliveries after an optional claim and before its locked body, whichever driver called it. The drain takes deliveryLockPath once; while another process of this machine holds it, the call delivers nothing and next answers wait.delivering, which the local driver polls. Each round picks a job under the run lock (nextDelivery, then unresolvedCreate in delivery-ledger.ts), runs deliverPr or findDelivered in pr-delivery.ts with no lock held, and applies the outcome under the run lock with completeDelivery or resolveCreate, which read the immutable job rather than the step. deliverPr requires origin's tip of the branch (ls-remote) to be the reported head or the latest copies[branch].commit a report recorded, and a recorded later tip to contain the reported head (merge-base --is-ancestor), else branch_rewritten. A tip past the recorded commit that still contains it is branch_ahead. completeDelivery keeps the step delivering while a step that writes the branch is leased (branchWriterLeased in engine.ts), since that step may push before it reports, or when copies[branch].commit is no longer the job's recordedHead, since a report recorded a commit while the drain read the branch. The drain does not pick it again in that call. Otherwise no report of that tip can come, and the step fails at once with the branch_ahead message, which says the tip holds the checked changes and later ones no report recorded. The same two conditions hold back a pull request: the step stays delivering, and one the job found or was told of gets no receipt, while one it opened does. The branch_moved and branch_missing remedies name the recorded commit. deliverPr fetches and diffs the selected tip against the run base on the run's checkout, running every git call that reaches origin within DELIVERY_TIMEOUT_MS and with NON_INTERACTIVE_GIT_ENV, and rejects protected paths (protectedPaths in worktree.ts: AGENT_PATHS plus DELIVERY_ONLY_PATHS) before it accepts a reported URL, looks up or creates. When a recorded later tip changes protected paths, it also diffs the step's own head: a clean own head is later_protected_path, a delivery_failed that names the later commit, so a retry delivers again instead of running the agent. Its remedies are the ones the run accepts: put origin back at the step's own head, or retry the workflow step whose copy the branch is (the job's openedBy). The job of a workflow step carries childWork (childWorkOff in workspace.ts): the head of each step inside it that succeeded on another branch that no step inside it delivers. Each must be an ancestor of the tip, else child_work_left_out, since the pull request would leave that work out. A workflow step whose branch holds nothing beyond the run base opens no pull request, so finishStep fails it with the same message when such work exists (workLeftUndelivered). It looks the branch up before the create and again after it, and reads origin's tip again between that first lookup and the gate. Only a lookup's URL is ever recorded. A post-create lookup that fails, or that finds none after a create that answered, fails delivery with create_unconfirmed and keeps the unresolved entry for a later lookup. It treats a found URL on another host than the origin's pull requests as a fault. Its gate, markCreating, takes the run lock right before the create: it reconciles, refuses unless the same pass of the same commit still delivers, answers busy when copies[branch].commit is no longer the job's recordedHead or while a step that writes the branch is leased (branchWriterLeased), and lists the request in run.creating. nextDelivery passes over a step while a step that writes its branch is leased (branchWriterLeased), so no git or provider call runs for it, and no reported URL or found request is accepted, until that step's commit can be checked. The gate's check covers a lease granted while the delivery runs. Timeouts (expire in limits.ts) and budgets settle in reconcileRun, which both nextDelivery and this gate apply, so no delivery is looked up, opened or accepted once a limit stops the step or the run. A new condition that stops a run belongs there too. A run lock it cannot take is busy too, as is a tip that moved during the lookup, and neither is a refusal: nothing is listed or opened, the step stays delivering, and the drain stops for that call, so the next one checks the branch again. A new receipt on a run that already ended queues the end record under the run lock, and it publishes after the drain releases the delivery lock. In reportAttempt, the lookup that vets a reported changes.pr_url (reportedPrMismatch) is the only delivery I/O, and it runs before the run lock. On the origin's host, a lookup that fails or answers an unsafe URL drops the reported URL. No provider code merges; no-provider-merge.test.ts scans src/providers/**.

  • A pull request is reused by provider lookup, never by a stored receipt. delivery-schema.ts owns the schemas of delivery receipts and unresolved creates, and CommitIdSchema. It imports only zod and pr-url.ts, so snapshot.ts and the run store load no delivery code or provider, which no-provider-merge.test.ts checks. delivery-ledger.ts owns their pure updates. The engine passes reconcileRun into the ledger and keeps the drain and create gate. Three fields keep what delivery did. run.deliveries is an append-only receipt list, deduplicated by URL, that only recordReceipt writes, for completeDelivery and resolveCreate. deliveryTarget in workspace.ts reads which branches delivered, buildEndRecord in records.ts reads it as prs in list order, and the snapshot publishes it. copies[].from is the branch a copy grew from: rememberCopy writes it once, at the copy's first lease, from the contract planWorkspace ordered (after, then needs, then inherited needs), before the copy has a commit. deliveryTarget walks it to the nearest branch with a receipt, else run.base.ref, and the snapshot publishes it and refuses one outside teamai/<run>/. run.creating lists each request TeamAI asked a host to open and holds no receipt for. markCreating appends it, completeDelivery removes it with the receipt of a delivered outcome and keeps it on a failed one, and resolveCreate applies a lookup of it: found appends the receipt and removes it, unsafe removes it immediately, and none or failed removes it after its second none or third failed answer. Each entry keeps total, none and failed lookup counts in lookups, set to zero by the gate each time it opens, including for a request listed before, and advanced only by resolveCreate. unresolvedCreate picks each at most once per drain, never in the drain that created or failed it, after the delivering steps, in an ended run too, and findDelivered reads only the origin and findOpen. It touches no step, so a retry, cancel or archive cannot lose the request, and a lookup cannot settle or fail a step. A delivering step whose request is listed still looks up before it creates again. The snapshot publishes the list, so a claimer looks the requests up.

  • What an attempt's own commands deliver reaches its pass only through runs/<run>/outcomes/<attempt>.jsonl. appendAttemptOutcome in store.ts is its only writer, called by contribute when ATTEMPT_ENV names the attempt and the run is in its scope. reportAttempt reads it with readAttemptOutcomes once the attempt has ended, for every driver, and keeps it as the pass's outcomes. buildEndRecord publishes a pass's outcomes as its commits, its delivery receipt, then those, each once. An attempt that ends without a report, such as one whose lease expired, keeps none.

  • A run changes machines only through handoff.ts. publishRun and the claim behind claimRun and resumeRun are the only writers of workflows/runs/<run>/state.json on teamai-reports, and they write it inside updateReports. A publish writes only while origin shows this install (installOwner, from <dataHome>/workflows/owner.json) and the state revision this machine last saw (acknowledged). A publish writes repository, the business checkout's getRepoIdentity, and, for a running run with a known repository, offered (handoffOf), keeping origin's offer time; --next takes runs by repository, so it could not take an offer without one. An unchanged body, offer and repository write nothing. A publish checks the business origin without a lock, then answers state_moved when, under the locks, the run's base, a copy's commit, or the sessions it archived differ from what it checked (checkedOutsideLocks). Anything else that moved meanwhile is published as the locks find it, so a run a driver is still advancing can be published. resumeRun continues a run locally when origin has no snapshot and this machine never saw one acknowledged (stateRevision 0), since no other install can have claimed it. A claim reads origin under the reports lock and releases it, decides outside the lock (definitionsOf can fetch and sync a shared clone, and checkOrigin runs ls-remote, only for a running run: a settled one leases nothing until a retry reopens it, and the local driver's reconcileTakenCopies waits for that too), then writes only while origin still holds the write id it read (replaceSnapshot). The compare key is the write id, not the revision, because two claims of one revision both write stateRevision + 1. A moved or missing snapshot answers state_moved; a write whose push failed answers it only once a fresh read shows another write. Every claim writes the snapshot without offered, so a runner never takes a run another install drives, and claimOffered (take: 'offered', behind resume --next) answers state_moved once origin no longer offers the run. When the owner's own resume or claim finds its run offered, acquire takes the offer away in the same guarded write (withdrawOffer), at the same state revision, so a runner's claim prepared against the offer answers state_moved. PublishedRuns in src/workflow-published-runs.ts is the one reader of the snapshots for the resume listing, --next, and the dashboard; it lives outside src/workflow/ because it judges each offered run by the skill driver's preflight rules, which it reaches through skill-driver.ts, and, for the local driver, by the tool launchToolFor picks for each prompt step left, the one a step would run on. What the steps left need of a machine comes from describeWorkflow's facts.runs, and isLeft drops the steps settled for good, as stepNeeds does. A next without claim calls claimedAway, which reads the snapshot on the reports checkout's tracking ref as of its last fetch, so it adds no network call, and answers not_owner once that shows another install. Every next that answers done, and every run control, calls publishRunEnd, which runs publishRun when the run was published and has ended, so origin never hands out a run its owner finished. A publish drains the run's records first, so none lands after its end. The drain (publishPendingRecords) drops the records of a run whose snapshot on origin names another install, so an install that ends its copy of a run it lost never replaces the owner's record. Once the end is on origin, a repeat writes nothing, so an end that a process saved but stopped before publishing reaches origin on the next request. A claim changes steps only through takeOver in step-state.ts: it releases a lease, and repends a lease in a new step so its next pass gets a new branch. handoff.ts never imports the engine. The engine calls claimRun for the claim field of next, and workflow publish --sessions passes the engine's resumableSessions in. What the snapshot carries is a rule table per type in snapshot.ts (publish, drop, or rewrite), so a new run or step field does not compile until it has a rule, and parseSnapshot refuses a field it would drop.

  • The end reached first wins. A reported step's settledAt is the revision its report made. Every terminal transition, including a skip or cancellation, carries settledAt. A step the engine settles takes the run's current revision, which is that of the report that caused it, because every earlier revision was reconciled before that report was accepted. An open lease it fails takes the revision it was leased at, and a cancellation the winning end's revision unless it keeps an already completed agent settlement. reportAttempt accepts a report only after reconcileRun has applied every end reached before it, so an end a later report reaches never competes with an earlier one. Every end settleScope has yet to apply was reached at the run's current revision, and it applies the one declared first. The winner therefore does not depend on when next runs between reports.

  • A break is a nameless succeeded end. A succeeded step whose break holds competes with the steps that reached an end by the same rule. It ends its workflow, or that workflow's loop, as success. Its value cannot change after its step succeeds, because pull rejects break inside a step loop's range, where a previous-pass read could change.

  • A workflow loop is decided when a round has nothing left. Once every step of a looping workflow has settled and no end or break was reached, decideRound in loop.ts evaluates the workflow's loop.while. When it is false, the workflow ends with the round's outcome and outputs. Otherwise every step of the workflow repends for the next round, unless the workflow has run max_iterations rounds: then it fails loop_exhausted (WorkflowResult.failure, and run.failure, shown in done, at the top). A failed step alone does not stop the loop. The round is kept on the step that runs the workflow (Scope.runner), so repend resets it with that step, or on the run at the top. Pull rejects a step loop in a looping workflow.

  • A step waits for every step it reads. readsOf collects the steps that the step's expressions (stepExpressions), templates, and with read. For a step that runs an inline workflow, it also collects the outer steps that the inline steps read. skipUnreachable uses that set to decide readiness. If a new field reads steps, add it to stepExpressions or templatedTexts. Otherwise the step can read a value before it exists, or an inline workflow can wait forever. A step outside a loop that needs or reads a step inside it also waits until the loop owner is settled (waitsFor in loop.ts), and is neither skipped nor ready before.

  • A read of a later step in the same loop is a previous-pass read, and never waits. previous in loop.ts classifies it: the target is in the reader's loop range and needs the reader, directly or not. For an inline step, the reader is the step that runs its workflow. Pull allows the read, the engine leaves it out of the reads a step waits for, and expressionSteps gives the reader the target's latest archived pass (previousPass), which holds no pass fields on pass 1. Classify a new kind of read through previous too, or pull and the engine disagree on it.

  • A step loop is decided once per run of its owner. After checking for an end, settleScope calls decideStepLoop for each succeeded owner without loopFinal. A false while sets loopFinal. Otherwise the range runs again through repend, unless a while owner has run max_iterations times, which fails it loop_exhausted. A loop without while is bounded where the owner's action would start, after the skip rule and if, for a leaf and a workflow step alike (exhaustedAtStart). The owner counts its runs in iteration. Pull rejects ranges that share a step and loops that wait for each other (loopProblems), so the engine does not check them again.

  • A pass id is never reused. repend moves a step's current pass into its passes and bumps pass. archiveUnder moves what a repended workflow or for_each step ran into run.archive, each pass stamped within with the step path and pass that archived it, and firstPass starts a new instance after every archived pass of its path. The full record of a run is run.steps, with each step's passes, plus run.archive; a run record, scorer, or budget reads both. steps.<id>.passes is a step's passes plus its current pass once it settled (passesOf). The expression projection keeps results for a for_each pass, including in previousPass. Pull checks the same outputs-or-results shape in step reads and workflow output aliases.

  • An expression reads its absolute paths, wherever they are. expressionPaths returns every inputs and steps path, including those inside function arguments and predicates, and never a relative path such as .size, which reads the current element of a predicate. readsOf, skipUnreachable and the pull read checks use it, so a new kind of expression must return its paths there.

  • Expression types are checked in two places, through typeProblems. parseExpression rejects what an expression shows by itself, such as x < 'a' or any(...) < 1. readDiagnostics checks paths against the types the workflow declares. It checks each path only in the workflow that owns it, so a predicate an inline step runs over an enclosing step is reported once.

  • Pull and the engine share one list of templated fields, one list of step expressions, and one list of actions. The templated fields are in templatedTexts and the expressions, such as if and break, in stepExpressions, both in template.ts. HITL declarations normalize in parse-hitl.ts. The actions are in STEP_ACTIONS in types.ts. Add a new field or action there, not to a copy in parse.ts or engine.ts. Pull, the cache's prompt_file check, the lease render and readsOf all split a template with parseTemplate, which decides which {{ … }} is a template, and the render evaluates it with evaluateExpression, as for an if. parseReads in engine.ts and the step parse in parse.ts key a table by each expression, so a new one does not compile until both say how to read it.

  • A new step status must fit five switches and two predicates. stopped(), settled(), takeOver() and stopWhere() are in step-state.ts, and retryInstance() in run-controls.ts. stopped() decides which dependents are skipped, which steps can recover, and the run outcome. settled() decides readiness. takeOver() decides what a claim does to each step, stopWhere() what an end or a break does to each step under it, and retryInstance() what a retry does to each step in its cone. The predicates inProgress() and cancellable() in step-state.ts decide whether settleScope waits on a step and whether cancelRun may stop it. Each of the seven switches over every status with a never default, so a status added to StepStateSchema does not compile until all of them place it. StepStateSchema and the snapshot's StepCheckpointSchema are built from one variant list, SHARED_VARIANTS, so the status also reaches a handed-over run; they differ only in the leased variant.

  • Only the transitions in step-state.ts write step fields. waitForAnswer, answerHitl, resetPending, resetIteration, skip, lease, release, start, succeed, fail, cancel, repend, nextRound, endLoop and exhaust change a step in place and drop the fields its new status does not have. readRun parses every step through StepStateSchema, which gives a run file from an older teamai its defaults: settledAt 0, outcome from status, and no fields its status does not have.

  • run.steps is replaced, never changed in place. Its type is readonly, so addItems, insertChildren and archiveUnder add or remove instances by assigning a new array. An instance's id is readonly too, so an instance cannot move to another scope's group. statesIn in scope.ts keeps the instances of every scope per array, in a WeakMap, so settling a scope reads its own instances without scanning every step. A new array starts a new index, except that insertChildren, which adds the instances of a workflow a step starts, carries the old index over.

  • A pull check that changes what a file means carries a code. dropsKeptVersion in cache.ts removes the cached copy when a diagnostic has a code. An older definition then cannot keep running with the new meaning. A diagnostic without a code keeps the last valid version, but only after checkReferences checks that version again against what the workflows it runs settle to in the same sync.

  • The workflow cache holds only what one sync settled. checkReferences checks each file's version against the versions the other files settle to, and dropUnsettled removes every other cache entry. startRun relies on this: it reads the workflows a run uses from the cache and does not check them against each other again. A new way for a file to keep or lose its version goes through checkReferences.

  • A teamai default is resolved, never cached, and a run records which definition it ran. loadCachedWorkflow and listCachedWorkflows in cache.ts return the cached team file of a name or alias first, else the default the package ships in workflows/ (DefaultWorkflows in src/default-workflows.ts). A sync never writes a default, and checkReferences lets a team file run one. resolveStart picks the scope a start runs in: the project's team file, then the user scope's, then the default in the project's scope; readyStartConfig and the session-close offer go through it. startRun writes run.origin (originOf: team, or default with the teamai version); the records, the snapshot and the dashboard read it, and sourceOfRun gives describe its source. A claim of a default run (definitionsOf in handoff.ts) uses only the default this install ships, and refuses with definition_mismatch when its bytes differ from the ones the run ran.

  • The items of a for_each step settle like a workflow. When settling reaches a for_each step, advanceInstance evaluates its list and addItems puts an instance per item right after the step; the step is running until every item settled. settleRunning settles a running for_each step through settleItems, the way it settles a workflow step through settleScope. An item waits for the items its after names and is skipped when one of them stopped (afterStanding). It starts only while fewer than max_parallel items are leased or running (startedItems, counted once per scan). A failed item cancels nothing; the step fails once every item settled (itemsSettled). The rules that decide the items are in fan-out.ts, the settling in engine.ts. Pull rejects for_each with loop, end or break (loopProblems).

  • item is bound per instance, in the engine and in pull alike. contextOf gives an item its own value and any other instance its workflow's (Scope.item), which childScope passes into the inline workflow an item runs. A for_each step's own if and for_each therefore read the enclosing item, never their own. Pull checks the same binding: ReadSite.item marks the templates of a for_each step, and ReadScope.item a workflow that runs per item. A template reads item only where it is bound: parseTemplate takes the binding, which pull decides from the step and the engine from whether the context holds an item, so elsewhere {{ item }} stays text in both. A new place that reads item must decide its binding in both.

  • Step ids are paths. A step inside a sub-workflow has an id such as ship.check.classify, an item of a for_each step one such as build[2], and a step inside the workflow an item runs one such as build[2].verify. parseStepPath in step-path.ts reads them all; don't split ids by hand. A child step exists in the run only after the engine reaches it. To find a step's definition, use stepDefinitions in scope.ts, which reads the definitions alone, so it also finds the step of an archived pass. locateStep in engine.ts also gives the step's scope, and needs the instances along the path. Don't look a step up by top-level id.

  • What a step requires is decided once per lease, in negotiate. offerTerms in engine.ts builds the offer for the ready loop and the hand-back alike, and negotiate in requirements.ts answers grant, hold or fail. ENFORCEMENT classifies every requirement as strict, which holds the step for a driver whose can declares it (wait.held), or best effort, which grants it and records the gap in unenforced on the lease, the step and the record. A copy is used once an accepted report shows its step ran there: a success, an exitCode, changes or read (run.copies[].used); a staging or setup failure leaves it unused. Every lease on a copy in a workflow that declares setup carries it, so can.setup holds the lease, and the driver decides whether setup runs: when it creates the copy, or finds its setup unfinished (the marker openWorktree in place/created.ts leaves). A held hand-back cancels the attempt and releases the step; on an unused copy the driver also gives the copy up, so a capable driver can take the step, unless the driver still holds another lease in it, while a used copy keeps its owner, whose work lives there. A granted hand-back keeps its driver as the copy's owner (rememberCopy). The member's workflows.allow holds before any capability. Env values never enter the run: the lease carries key names in envScope (and setup.envScope), and the driver resolves them. Deadlines and budgets settle in reconcileRun(run, now), which runs expire (limits.ts) first; the run deadline and the run's budget count from run.limitsFrom, which a retry sets when it reopens the run or finds it ended, or else from createdAt with nothing spent; a workflow step's budget counts from the limitsFrom its start records (startLimits); a for_each step spends its budget over all its items together (itemsSpent in settleItems), and each item keeps only the budget of the workflow file it runs. An attempt the engine stops goes through stopAttempt, which records it in run.stoppedAttempts with its step, driver and stop time, whatever its step becomes later; a late report from that driver adds its tokens once (countLateTokens), and a budget whose limitsFrom started at or after the stop takes them into its baseline, so it does not pay for them. Known secret values are masked at the three exits, updateRun, enqueueRecord and writeEnvelope, each of which takes the secret set as a required argument. The set holds the value of each key env/secrets.yaml declares, wherever this machine resolves it from, and each environment value whose name looks secret; masking follows the team's declarations, not what a run requires. A command resolves it once and passes it on: startRun, withRun and the run controls require it, so what the envelope masks and what the run file masks agree; the local driver also masks an attempt's result.json before it reports, as text when it is not JSON, and every attempt's result file when it stops driving the run, so an attempt it never reported keeps no secret either.

  • A HITL request waits before any action or end. settleScope checks loop exhaustion first, renders a HITL request message, and moves an unanswered step to waiting. Waiting is neither stopped nor settled. An answer moves it to pending without a revision bump. Approve no skips, choose succeeds, approve yes continues. An end-only step with HITL reaches its end only after yes. The shared templatedTexts makes HITL reads participate in readiness. Only driver.can.hitl declares HITL. parseNextPayload reads driver.can into NextOptions.can; stored DriverIdentity and leases contain only kind, machine and workspace. Busy polls never answer waiting HITL requests through the no-HITL rules. Only a driver declaring HITL receives a waiting request; a busy poll without HITL receives wait. An interactive step is a requirement, not a gate: negotiate holds it for a driver without HITL, on new leases and hand-backs alike. The JSON payload parser reads autoApprove only from a driver declaring HITL, which sends it when the person chose auto-approve for the run; answerOptionalGates then answers optional approvals and choices with a default, recorded auto_approve, and never reads workflows.auto_approve. respond --json uses the same identity schema as next, reports unread fields through ignored, and records a driver UI's declared person answer with channel driver. The dashboard answers through the same respondToHitl with channel dashboard, always sending the pass it showed (answerFromDashboard in src/dashboard/workflow-runs.ts). respondProvenance in run-controls.ts decides agent from where the answer came in and the process that sent it: a terminal without a TTY, respond --driver skill, and any process with an AGENT_SESSION_ENV variable give agent: true, which a required approval refuses with agent_in_path. A process with TEAMAI_WORKFLOW_ATTEMPT answers nothing: respond refuses it with nested_workflow first (below). Answer driver identity stays local; records publish only choice, channel and agent.

  • An agent session starts or continues a workflow only as its invoke lets it. start in protocol.ts checks the definition's invoke, absent meaning deny, against the caller's agent evidence (agentEvidence in attempt-marker.ts), for every driver, before the run exists: TEAMAI_WORKFLOW_ATTEMPT answers nested_workflow (below), and with an AGENT_SESSION_ENV variable, or from a --driver skill call, which only a coding agent makes (skillContext), deny answers invoke_denied, and ask answers confirm_needed unless the start passed --confirmed, the agent's word that the person agreed, which the CLI cannot verify, and unattended_denied in an unattended run (InvokeCaller.mode), --confirmed or not, with describe's R17 wording (unattendedReason). A process without evidence, such as the person's terminal or a CI job, starts any workflow; CI exempts nothing. workflow resume, with either driver, next --driver skill --takeover, and a next payload's claim make the same check through resumeRefusal, of the definition the run carries (this machine's run, else the one on teamai-reports), before anything is claimed or changed; --driver skill counts there as for a start, and the session whose skill.json here names it as the run's owner is not checked again (drivesRun), as its start or takeover passed. A claim answers confirm_needed with the resume <run> --claim --confirmed literal, since a payload cannot carry the person's yes.

  • No step starts or drives a run. Every driver gives the processes it starts for an attempt TEAMAI_WORKFLOW_ATTEMPT=<run>/<attempt>: the local driver its commands, setup (runPlaceSetup) and tools, the skill driver its runners, and through them their commands and setup. workflow-commands.ts answers nested_workflow before it reads any run (attempt-marker.ts): to every run-changing subcommand (start, next, report, respond, retry, cancel, resume, publish) from a process with the marker, except report <run> --driver skill --attempt <a> whose marker names exactly <run>/<a>, the runner of that attempt; and to start from a workflow's place, a directory under .teamai/workflow-worktrees or a checkout whose HEAD is on a teamai/<run>/… branch, the only trace a place the host made keeps. start and respond in protocol.ts refuse the marker too, so a client in process answers as the CLI does. Only start reads the place, since a step's child and a --json driver report the attempt from it. A step command that reports its own attempt passes as the runner and gets held, since the runner holds the attempt's runner.lock.

  • Retry validates every container before resetting any. retryContainers in run-controls.ts climbs from the target through scopes and fan-out items until a running runner or the top. Each dependency cone includes needs, reads and loop waits, or item after dependencies. An independent standing end or break rejects the retry. The apply pass archives replaced passes and descendants, reopens settled runners with start to keep inputs and round, then bumps revision and reconciles. Only the targeted loop owner's iteration resets; the top-level round does not. An answered refused approve or succeeded choose is retryable too, so a changed answer resets its dependent branches. A HITL request in the cone that is waiting, or answered and not yet run, starts a new pass too, so its answer is asked again in the new context and an answer to the old pass is stale. Unanswered skips in every affected container reset so skips caused by cancellation can run again. Retry's prelude reconciles without finishing a provisional outcome. Reopening clears the run's end, failure and endRecordQueued flag and removes its queued end file through records.ts. These three pieces stay consistent: run state, the pending end record, and published stats. The next end upserts the same record path; stats replace the origin end's outcome and duration contribution with the new end's, without adding another run. Replaying the same endedAt contributes nothing.

  • The local driver keeps tools and questions interruptible. A single question joins command and prompt exits in the driver's race. An interactive tool owns the terminal alone: no question opens while it runs, and starting one aborts and closes an open question. A poll closes the question when its own step is leased or a different step or pass waits; an unrelated lease leaves it open. It polls next every second while asking, every two seconds while tools run, and aborts a replaced HITL request or cancelled prompt. src/tool-launch/process-stop.ts shares process stopping between commands and adapters. Headless tools detach and stop with SIGTERM; interactive tools stay attached and stop with their captured descendant tree. Foreground interactive signals reuse env-exec's terminal detection and signal policy: when the terminal reaches the child, teamai ignores SIGINT and SIGQUIT, while SIGTERM and SIGHUP are forwarded and stop the driver. Interactive adapters inherit stdio and accept an optional result file. Repository edits are kept for interactive steps; only headless prompts run the read-only check. A nonzero interactive exit without a result reports its exit status. leaseNext records the identity of the latest driver that called next in RunState.driver, without capabilities. Resume uses this machine's open local lease workspace, then run.driver.workspace for a local driver on this machine, then cwd. Controls pass run.driver to finishRun so end records retain the driver kind.

  • One local driver owns a run on this machine. takeRunLock in store.ts takes driverLockPath with acquireLock, waiting at most the caller's waitMs, and releases it in finally; the RunLock it passes is the proof a caller holds it. It answers held with the body's value or busy, so no answer of a body reads as a busy lock; withRunLock turns busy into the driver_running error for a body that answers an envelope. withDriverLock in local-driver.ts takes it without waiting. runLocalDriver drives inside it, and workflow resume also claims inside it, so a claim never lands under a driver already running the run. A live holder returns driver_running, including its pid, before anything is claimed or driven; the existing lock machinery reclaims dead owners. The driver lock spans tool execution and questions, while withRun holds a different run lock only during one engine operation. Neither lock's scope is extended by the other. Every HITL decision without a HITL driver logs once when the engine changes the step. HITL replies carry one literal respond command template, which drivers display unchanged.

  • Judge work never comes from workflow next. scoring.ts computes open judge work on every teamai score --next from facts already on disk: the run files, the scorers startRun pinned in run.scorers, the installId that reportAttempt stamps on each pass it settles, the within stamp on archived passes, and the labels, attempts and on-demand requests (teamai score --run) under <dataHome>/scores/. Nothing else is queued. The engine has no scoring code path, and scoring never writes a run file, so a run's counts, tokens, failure and endedAt cannot move and it settles without waiting. A judge lease settles only through teamai score: --lease <ref> --submit, or --lease <ref> and --run, which run the local judge and settle through the same submit path. workflow report answers judge_lease. A driver never lists a judge ref in workflow next's running, which holds step attempts only, and leaseNext answers judge_lease before it changes anything when one does: judge refs go in score --next's running, which renews them.

Tests

Test parsing through parseWorkflow and syncWorkflowDefinitions. Test the engine through startRun, leaseNext, and reportAttempt. Every driver must pass workflowConformanceScenarios in src/workflow/conformance/run.ts, which runConformanceScenario in src/workflow/conformance-interpreter.ts runs, and a driver that takes workspace leases the git and delivery scenarios in conformance/git.ts and conformance/delivery.ts too. That one list takes in the scenarios a concern keeps in its own file, such as limits-conformance.ts, so a runner cannot miss one. workflow-delivery-drivers.test.ts runs the delivery scenarios through the built CLI for the local driver, a driver that pushes from its own worktree, one that knows nothing of delivery, and one that reports its own pull request. If a driver can see a new behavior, add a scenario to the file for its concern; a new file joins the list in conformance/run.ts.

Session driver workspace preparation

workflow next --prepare prepares through place.ts, as the local driver does; it never implements graph semantics. A lease whose place another open attempt holds (placeHeldBy) answers busy and stays leased, as the local and skill drivers queue it. An attempt the run stopped holds its place too, for the driver that held it, until that driver reports it and the report abandons the place, since what it started may still run there (otherLeases lists that driver's run.stoppedAttempts). Another driver's stopped attempt holds nothing, as that driver may be gone and never report it. --attempt <a> prepares an attempt the driver already leased: prepareAttempt in protocol.ts lists every other lease of the run as running, with canTakeLease: false, so the engine hands that one back, and answers wrong_attempt when it does not, naming the release when the re-hand gave the attempt up because the driver's can no longer covers the step. --adopt <dir> makes a writer's copy a place the host made (place/adopted.ts). adoptRefusal checks it and changes nothing: an attempt already prepared elsewhere and a step in the checkout adopt none, and a reader only the place the host made for its branch, and the place must be the top of a worktree sharing the checkout's git directory, clean, on the branch or detached at its start while no such branch exists, with the start in its history and every lease commit present after fetchMissingCommits. A place that fails answers adopt_mismatch, and the attempt stays leased; so does a place another open attempt began to adopt (placeHeldBy), with the message prepareHostPlace refuses it with. takeAdopted then points its hooks at the empty hooks directory and switches a detached place onto the branch. Setup runs there on the first prepare, unless an earlier attempt of the run finished in that place. The skill driver prepares a writer on h3 through prepareHostPlace: host goes on in the place the last attempt on the branch finished in while it passes the checks, else answers needs-place; adopt answers needs-place with refused for a place that fails them, or that another open attempt began to adopt (placeHeldBy). Every other lease on a branch whose last attempt finished in a place the host made (adoptedPlaceOf) goes on there while it passes the checks, a command's and a reader's included, and TeamAI makes the copy once it does not; placeHeldBy counts that place too. Every prepare in protocol.ts takes its lease, checkout and the run's other leases from leasedPlace, and its busy rule from placeLeased. The record keeps adopted whatever the lease's role, so releasePlace leaves the place, and a failed setup puts the branch agent files back instead of removing the worktree. The core.hooksPath takeAdopted set is unset again at every end of the attempt there: a prepare or setup that failed, a finish of any outcome, and an abandon. Setup marks the place it runs in until it finishes (markSetup), and adoptedMismatch refuses a place that still has the mark: what a setup that failed or was cut short left there would pass the next setup's checks, so the host makes the place again. Attempt-local metadata contains paths, workspace contracts and fingerprints only. Repeated next calls reuse that metadata without reapplying an overlay over agent edits. Public report reconciles cancellations before git, checks driver/revision ownership, finishes through the shared local workspace helpers, and cleans up only after the engine accepts success with cleanup. Git network operations run under the run lock too, such as the push that finishes a writer, and the fetch of the run's branches (fetchMissingCommits) by a prepare that opens or adopts a copy lacking a commit the lease needs. A skill driver's foreground call waits TICK_LOCK_WAIT_MS for the lock, then answers driver_running. The existing per-run driver lock serializes local and public preparation/report. Run lookup scans configured local data homes by id and rejects duplicates, independent of cwd; the run's checkout governs git work.

The skill driver

drivers/skill/ runs a workflow from a coding session. The agent runs only the literal commands each reply prints. A detached runner per attempt, workflow report <run> --driver skill --attempt <a> with TEAMAI_WORKFLOW_ATTEMPT=<run>/<a> (attempt-marker.ts), prepares the attempt's place under the run lock and runs its setup outside it; then it runs a command step outside the lock and reports it, or writes a prompt step's brief and offers it to the agent, which hands it to a child agent; the child reports it, from whatever coding session it runs in. A runner holds the attempt's runner.lock while it lives, which is how a tick tells it is alive. Setup and a command each run in a process group of its own, recorded in the attempt's own group-<pid>.json as it starts (recordGroup in tool-launch/process-stop.ts), so neither displaces the other; a runner killed while either runs leaves that group running, so C10 and C11 stop every group the attempt recorded (stopRecordedGroup) before the step runs again or its place is given up. Windows has no process groups, and there a setup or command a killed runner left runs on. skill-state.ts is the only writer of runs/<run>/skill.json. journal.ts appends runs/<run>/journal.jsonl beside it, after each call's answer: a line for each call a session makes on the run (the call, who made it and whether from the run's checkout, the attempt or step it names, and its answer code), and an asked line for each dispatch and person action the first time a turn shows it. It holds no env value, input or session id, no turn prints it, and the driver never reads it but to journal an action once. An error thrown inside an agent's call answers skill_call_failed with its message and next: null (runSkillCall), so the agent always reads one envelope; a runner's own call lets it go on up, as nobody reads its answer. transitions.ts holds the rows below as TRANSITIONS, and one row function per input, such as replyRow or pollRow, returns the row it took with only what its caller needs: the new state and its effects, or the error the call answers. tick.ts and skill-runner.ts save that state and apply the effects through one interpreter, applyEffects. workflow-skill-transitions.test.ts compares these tables with TRANSITIONS cell by cell, both ways, and walks every row through runSkillCall, so a row lands in both places or in neither.

Run states, derived from skill.json and never stored as a field: none (no skill.json for this run, or no run yet), host (the run has prompt steps and the agent has not answered the host question), drive, and done (the engine answered done). Attempt phases: preparing (its runner prepares its place), queued (another open attempt of the run holds its place), placing (it waits for the agent to adopt a place its host made), running (its runner runs the command), offered (its dispatch is shown, not acked), dispatched (acked: a child works it), stopping (the engine cancelled it, or its expiresAt passed: its runner stops its command, or the agent stops its child and reports it), and reporting (a write-ahead report body is stored).

Order of one call. Parse options with zod; for a resume or a takeover, R22, before the lock, whose file would leave runs/<run>/ behind a refusal on a machine that lacks the run; take the run lock (D19); S rows; the row of the input; a tick; build the turn; save; release; journal the call. A start WorkflowClient.start refuses after R10 answers its error with next the describe literal only for a refusal another answer fixes (ANSWERABLE_START_REFUSALS in skill-driver.ts: invalid_inputs, confirm_needed), and null for the rest. A tick sends the reports a crash left stored (P11), gives up attempts stopping by a runner that is gone (C11), respawns the runners of queued attempts whose place is free (D9) and of attempts whose lock is free (C10), then calls the engine next (D1), takes D2 to D5 for each attempt its cancelled lists, then H5, and applies its reply, repeating while a lease comes back and a slot is free, once more when a reply to a call's next with no slot gave up an attempt that waited (D2), so the slot it freed is leased in the same call, and once more after the last lease while a gate is journalled, so that reply, made with no slot, says whether the gate still waits; then, while judging is on, it takes J5 and J1, in drive and in done. A poll (--wait) repeats lock, tick, release and a WAIT_POLL_MS sleep until one of D22 to D25 applies. Runners take C rows under the same lock and never take S rows. An agent's report (report --attempt without the marker) skips the session check when the agent reports the attempt (S6), since a child an app's orchestration tools start has a session of its own; it takes its P row, sends the report and saves under the lock, then answers as a runner does, with no tick and no turn, so it leaves the session's events for the orchestrator's next call. A runner's check: while its command runs, a runner ticks every RUNNER_POLL_MS, and once at the attempt's expiresAt, with canTakeLease false. The engine applies deadlines only when it is called, so the check at expiresAt is what turns an expired attempt into a stop (D3, C9).

Describe and start

Row State Input Guard Effects Next Action
R1 none describe invoke: deny and the caller has agent evidence none none describe ask.kind: denied with the reason, no literal
R2 none describe run mode attended, and the definition has a local route: a step this driver takes and a later one it is held from for env or tokens work in one workspace copy none none describe ask.kind: local with the reason and the --driver local start literal, with --tools default=<session tool> when localPromptTool names one
R3 none describe R27 does not apply, and something to ask: invoke: ask, a required input, width above 1, an optional gate, or judging here or by the session none none describe ask.kind: ask: one question stating every fact, and at most five start literals, the recommended first
R4 none describe R27 does not apply, and nothing to ask none none describe ask.kind: start with the one start literal, and the facts to tell the person
R16 none describe deliveryProblem answers for the checkout: a deliver: pr step the definition cannot deliver (bad_delivery), or a detached HEAD (no_base_branch) none none describe ask.kind: denied with its message, no literal
R23 none describe run mode unattended, R1 and R16 do not apply, and the definition has a local route, as R2's none none describe ask.kind: denied with the local route's reason, no literal: nobody is there to run the local driver
R17 none describe run mode unattended, R1, R16 and R23 do not apply, and attendedReason names what needs a person: invoke: ask, an interactive step, a required approval, or a choice without a default none none describe ask.kind: denied with that reason, no literal
R18 none describe run mode unattended, and R1, R16, R23, R17 and R27 do not apply none none describe ask.kind: start with the start of R3's recommended option under --auto-approve; {inputs} kept and inputs listed when the workflow has inputs
R27 none describe the session names no tool, the definition has a step whose model or judging needs one (needsTool), and R1, R16, R2, R23 and R17 do not apply none none describe ask.kind: tool with the reason, the describe literal with --tools default='{tool}' for the agent to fill, and the ids of the tools teamai knows
R8 none start --driver skill, or R19's input the definition has a local route; for R19's input, the run's own definition, once R20 and R21 do not apply, or, in run mode unattended, as the run stands before the claim none none error needs_local_driver with the reason, next null; for R19's input it names the person's teamai workflow resume <run> literal, and the run stays readied here, or, unattended, as it was: nothing is claimed
R25 none start --driver skill run mode unattended, R8 does not apply, and attendedReason names what needs a person, as R17's none none error unattended_denied with that reason, next null: no run is created
R9 none start --driver skill checkStartAnswers answers confirm-needed, and R8 and R25 do not apply none none error confirm_needed, next: the describe literal
R10 none start --driver skill checkStartAnswers answers ok WorkflowClient.start; write skill.json: identity {skill, machine, run checkout}, invocation, options (--parallel, the HITL mode, the caller's run mode), host pending when the definition has a prompt step or the session judges the run, with whether it has a prompt step, owner := the caller's session, working in the tool its --tools default=<tool> names, else the one its host names; judging on with the session tool when describe judges here, or sessionJudge with it when describe judges by the session, and the start has no --no-judge host when it has a prompt step or the session judges it, else drive the turn of a tick
R11 none next, report --driver skill this machine has no skill.json for the run none none error not_skill_run, next: the resume <run> --driver skill literal, with --tools default='{tool}' for a session with no tool to fill
R28 any R19's input R19 would take the run, R8 and R24 do not apply, and the session names no tool while the run has a step whose model or judging needs one (needsTool) remove the skill.json kept here, which the published run the resume installed made stale unchanged error tool_needed, next: the resume <run> --driver skill --tools default='{tool}' literal with the options and --confirmed of the call, whose slot the agent fills with its tool
R19 any resume --driver skill; next --takeover with no skill.json here WorkflowClient.resume readied the run: it took a run this machine lacks, or with --claim one another install owns, or found the run here and this install its owner, and released each lease this machine's local driver left on it, once the process group that lease's setup or command left is stopped, and stopped the group each attempt of that driver the run already stopped left; no skill.json is kept here, or the claim installed the published state; R8, R24 and R28 do not apply write skill.json as R10 does, with the caller's run mode and the options of --parallel, --hitl or --auto-approve and --no-judge, else a start's: one step at a time, HITL mode person, or auto-approve in an unattended run host when the run has a prompt step or the session judges it, else drive the turn of a tick
R20 any R19's input WorkflowClient.resume answers that another install owns the run (a resume without --claim, or a takeover), or that this machine lacks it (a takeover) none unchanged error not_owner or run_not_found, next: the resume <run> --driver skill --claim literal, with the --tools default=<tool> the resume declared, run only when the person agrees
R21 any R19's input WorkflowClient.resume refuses the run otherwise: teamai-reports does not hold it or moved while it was claimed, its definition cannot be read here, or its copies cannot be brought to the commits it recorded none unchanged error with the refusal's code and message, next null
R22 any resume --driver skill, next --takeover the caller is not the session skill.json here names as the run's owner, and WorkflowClient.resumeRefusal refuses it by the invoke of the run's own definition, as a start is refused: agent evidence (a session variable, or --driver skill itself), and invoke: deny, or invoke: ask without --confirmed or in an unattended run none unchanged error invoke_denied, unattended_denied or confirm_needed; next: for confirm_needed the same call with --confirmed, run only when the person agrees, else null
R24 any R19's input, or next --takeover with skill.json here run mode unattended, R8 does not apply, and attendedReason names what in the run's own definition needs a person, as R17's, counting only the steps the run has not settled for good (succeeded or skipped, a step in for_each items once every item settled it or a step that holds it, and every step in a workflow or for_each step that settled whole; none while a workflow or step loop of the run could run them again), and its invoke: ask, for a run still running (one that ended needs nobody to ready it as it ended); read before the claim or the takeover, from the published run a --claim would install, else this machine's copy, so R20 and R21 come after it none unchanged error unattended_denied with that reason, next null; the run and its skill state stay as they were: nothing is claimed or taken over
R26 any R24's input, with skill.json here that R19 does not replace run mode unattended, R8 and R24 do not apply, and the skill.json's run mode is attended its run mode becomes unattended, and its HITL mode that of --hitl or --auto-approve, else auto-approve unchanged the S row of the call
R12 host tick none host choose about:host decide:agent
R13 host next --choice h host := HOST_PROFILES[h], or none for a run without a prompt step, which asked it only for its judge; on h4, which starts no child agent, drop sessionJudge drive the turn of a tick, with event judging-off no_child_agent when it dropped sessionJudge
R14 host report, next --ack or --adopt none host error choice_pending, next: the next literal
R15 drive, done next --choice h none unchanged error unknown_choice, next: the next literal

Sessions

Row State Input Guard Effects Next Action
S1 host, drive, done an agent call from session s the owner is s, or either session is unknown lastCallAt := now; the tool := the one the call declares with --tools; else owner := s, working in s's tool, when unknown; else the tool := s's when the run has none and s is its owner or neither is known unchanged the input's row
S2 host, drive, done an agent call from session s without --takeover owner o is not s, and both are known, however long o has been idle none unchanged error orchestrator_active with idleSeconds, the seconds since o's last call, next: the takeover literal
S4 host, drive, done next --takeover from session s owner := s, working in the tool its host names; lastCallAt := now; seen[s] := seen[o] when that is later; every offered or dispatched attempt becomes offered with reissued unchanged event takeover; the turn of a tick
S5 any a runner call (report --attempt a with the marker <run>/a) no session check, no lastCallAt unchanged C rows
S6 host, drive, done an agent report --attempt a, from any session a is settled, or open offered, dispatched, reporting, or stopping by the agent no session check, no lastCallAt unchanged the input's row

Ticks, polls and the lock

Row State Input Guard Effects Next Action
D1 drive tick engine next {driver: identity + canFor(host), the can of the host's profile, and h1's for a run without prompt steps, running: every open attempt, canTakeLease: fewer open than options.parallel, and than one under a serial host; false in a runner's check, autoApprove: true under HITL mode auto-approve, runMode: options.mode} drive
D2 drive reply cancelled holds a a queued or placing drop a drive event stopped
D3 drive reply cancelled holds a a preparing or running phase stopping by its runner, why timeout once its expiresAt passed, else cancelled drive
D4 drive reply cancelled holds a a offered or dispatched phase stopping by the agent, why as D3 drive stop {why, report} in each foreground turn until reported
D5 drive reply cancelled holds a a stopping or reporting none drive
D6 drive reply lease executor command phase preparing; spawn runner job command drive event started
D7 drive reply lease executor prompt, and D8 does not apply phase preparing; spawn runner job prompt drive event started
D8 drive reply lease executor prompt, workspace new or continue, and a host that makes places (h3) phase preparing with request host; spawn runner job prompt drive event started
D9 drive tick a queued attempt whose behind is no longer open phase preparing; spawn its job again drive
D10 drive reply hitl a required approval journal the gate: seq := seq + 1, unless it is the gate journalled already drive person why:approval with the person's literals; waiting <step> waits for the person's answer
D11 drive reply hitl an optional gate; HITL mode person, or auto-approve, under which only a choice without a default comes back as D10 drive choose about:gate decide:person
D12 drive reply hitl an optional gate; HITL mode recommend as D10 drive choose about:gate decide:agent
D13 drive reply wait it lists delivering none drive waiting delivering in each turn that ticked, polls' included; then: poll
D14 drive reply wait it lists held, and no attempt is open none drive person why:handoff with the person's teamai workflow resume <run> literal, with --tools default=<session tool> when localPromptTool names one; waiting held; no then
D15 drive reply wait none drive waiting
D27 drive reply wait, after its row it lists held, and no reply of this tick came to a next that took no lease engine next as D1, with canTakeLease: false, since the engine answers a held step before a gate; H5, then D10 to D12 for a gate, take its reply drive
D16 drive reply done seq := seq + 1, so every session has the end to see; keep with the end the run file's version, read after, when the end is out and its status and receipts agree with the end done done {outcome, failure, prs, judging, score}
D17 drive reply error none drive error with the engine's code, next: the next literal
D18 drive a foreground call it spawned runners release the lock; wait up to WAIT_GRACE_MS for them, re-taking the lock per check drive
D19 any a foreground call the lock is not free within TICK_LOCK_WAIT_MS (report: WAIT_BOUND_MS) none unchanged error driver_running, next: the same literal
D20 host, drive, done a foreground call, after its tick seen[s] := seq unchanged the turn: the host question, or every stop, then every unacked dispatch, then every place, then the gate's choose or person; events after the old seen[s]; then: poll until the run is done and no attempt or judge is open
D21 host, drive, done a poll tick the lock was busy, or nothing below applies, and the bound is not reached release; sleep WAIT_POLL_MS; tick again unchanged
D22 host, drive, done a poll tick the host question is unanswered, a dispatch is unacked, a place is not adopted, a stop is not reported, or the gate was journalled after seen[s] none unchanged then: wake with the next literal, no actions
D23 done a poll tick seen[s] is below seq, and no attempt is open seen[s] := seq done done with its events; then: poll while a judge is open
D24 host, drive, done a poll tick the bound is reached, and D25 does not apply none unchanged then: poll, no actions
D25 drive, done a poll tick a settled or stopped event came after seen[s], and neither D22 nor D23 applies seen[s] := seq unchanged the events after the old seen[s], no actions; then: poll
D26 drive a poll tick the tick took D14, and D22 does not apply none drive then: wake with the next literal, no actions

Runner outcomes

Row State Input Guard Effects Next Action
C1 preparing runner: prepared job command run the command outside the lock with {...stepEnvironment(env, envScope).env, ...lease.env, TEAMAI_WORKFLOW_ATTEMPT} (process.env for a lease without envScope), output masked into output.log running
C2 preparing runner: prepared job prompt pickChildModel; writeChildBrief offered dispatch in each foreground turn until acked
C3 preparing runner: prepared job prompt; pickChildModel fails model_unavailable write-ahead failed with that failure reporting
C4 preparing runner: place busy, behind b behind := b queued
C5 preparing runner: needs a place request host, and no place an earlier attempt adopted for the branch passes the adopt checks the place to ask for: name, branch, start placing place in each foreground turn until adopted
C6 preparing runner: adopt refused request adopt dir refused := why; the engine still leases a placing place with refused
C7 preparing runner: prepare or setup failed write-ahead failed with its failure reporting
C8 running runner: the command exited write-ahead: success on exit 0, else failed; outputs, exit code, masked tail reporting
C9 stopping by its runner runner: its check every RUNNER_POLL_MS and at expiresAt, its setup ending, or its command exiting stop the setup or command and every process it started (tool-launch/process-stop.ts); write-ahead released reporting
C10 preparing, running tick: its runner.lock is free it was spawned more than WAIT_GRACE_MS ago stop the process group its setup or command left; spawn the same job again (prepare is idempotent; a command restarts in its prepared place) unchanged event restarted
C11 stopping by its runner tick: its runner.lock is free it was spawned more than WAIT_GRACE_MS ago stop the process group its setup or command left; write-ahead released reporting

Acks

Row State Input Guard Effects Next Action
K1 drive next --ack dispatch.a a offered phase dispatched drive the turn of a tick
K2 drive next --ack dispatch.a a dispatched none drive the turn of a tick
K3 drive, done next --ack dispatch.a a neither offered nor dispatched none unchanged error unknown_offer, next: the next literal
K4 drive next --attempt a --adopt dir a placing phase preparing with request adopt dir; spawn its job again drive the turn of a tick
K5 drive, done next --attempt a --adopt dir a not placing none unchanged error unknown_attempt, next: the next literal

Reports

Row State Input Guard Effects Next Action
P1 drive, done report a a is in the settled list none unchanged runner answer repeated
P2 drive report a a is unknown none drive error unknown_attempt, next: the next literal
P3 drive report a a preparing, queued, placing, running, or stopping by its runner none drive error not_dispatched, next: the next literal
P4 drive report a --model-unavailable a offered or dispatched, its dispatch printed unavailable write-ahead failed model_unavailable naming the model reporting P7 to P10; after P7 to P9, runner answer reported
P5 drive report a a offered or dispatched, otherwise read result.json: an output gives success with it as outputs, a failure gives failed with it, none or an unreadable one gives failed tool_failed; write-ahead reporting P7 to P10; after P7 to P9, runner answer reported
P6 drive, done report a a stopping by the agent write-ahead released reporting P7 to P10; after P7 to P9, runner answer reported
P7 reporting engine: reported, any status drop a; add to settled drive event settled, with the failure the engine recorded, else the body's
P8 reporting engine: repeated_report as P7 drive event settled
P9 reporting engine: lease_cancelled drop a (the protocol abandoned its place); add to settled, so a late report of its child is P1 unchanged event stopped, why as its stop said, else as D3
P10 reporting engine: another error keep the body reporting error, next: the same report literal (a runner retries at its next check)
P11 reporting tick, or report a a body is stored (a crash or a refusal came between write and send) send it again reporting P7 to P10

Answers

The engine is the one judge of who may answer a gate (respondProvenance in run-controls.ts): respond --driver skill always carries an agent's answer, so a required approval refuses it, and the person answers it in their own terminal or teamai dashboard.

Row State Input Guard Effects Next Action
H1 drive respond --driver skill the engine answers repeated_response: the gate was answered already none drive the turn of a tick
H2 drive respond --driver skill the engine recorded the answer, as an agent's clear the gate when it is the one answered drive event answered; the turn of a tick
H3 drive respond --driver skill the engine answers agent_in_path: a required approval none drive error agent_in_path, next: the poll literal
H4 drive respond --driver skill the engine answers another error none drive error with the engine's code, next: the next literal
H5 drive reply, before its row a gate is journalled, and the reply is done, a wait that holds nothing, or another gate: the person answered it in their terminal or the dashboard clear the gate drive

Judges

While judging is on, a tick asks score --next for judge work of its own run only (run), never a session's, and lists the open judge's lease ref in running to renew it (J1). At most one judge is open. Its runner is the same workflow report --attempt call, keyed by the attempt in the lease ref: it runs the confined judge outside the lock, with the judge tool's process group in that attempt's own group-<pid>.json, and renews the lease through score --next with canTakeLease: false every 10 minutes until the judge answers, then takes J3 or J4. Judge refs never enter workflow next running. The first failure turns judging off, so a judge that keeps failing does not run again on every tick, and its lease lapses back to the queue for another driver.

When describe's judging is by the session (Judging session: no verified judge of the session's tool here, and a scorer that accepts best-effort), skill.json keeps sessionJudge instead of judging, so no tick takes judge work. done then carries score: the score --next literal of its run, with the skill identity and can.judge {best-effort, read, [the run's tool]}, the tool of the session that drives the run now, which a takeover changes. The session runs it once the run is done and judges each lease in a fresh subagent, as skill-data/workflow/references/judging.md says. A start or resume with --no-judge keeps neither.

Row State Input Guard Effects Next Action
J1 drive, done tick judging is on score --next {driver: identity + can.judge of the session tool, run, running: the open judge ref, canTakeLease: no judge open; false in a runner's check} unchanged
J2 drive, done score reply: a lease judging is on, and no judge is open the judge is open; spawn its runner, keyed by the attempt in its ref unchanged event judging
J3 drive, done judge runner: labelled its judge is open drop the judge unchanged event judged per scorer
J4 drive, done judge runner: failed its judge is open judging := off; its lease lapses back to the queue unchanged event judging-off with the failure
J5 drive, done tick: the open judge's runner.lock is free it was spawned more than WAIT_GRACE_MS ago stop the process group its judge left; judging := off; its lease lapses back to the queue unchanged event judging-off judge_runner_gone
J6 drive, done score reply: an error the error is not busy judging := off unchanged event judging-off with the error

Done

Row State Input Guard Effects Next Action
X1 done next no attempt is open, and a judge is none done done with judging; then: poll
X2 done next no attempt or judge is open none done done, no then
X4 done next an attempt is still open: the engine cancelled it with the run none done every stop, then done; then: poll
X5 done tick the run file shows a version other than the one kept with the end, or a create still unresolved, or none was kept keep the run file's version with the end; engine next as D1, with canTakeLease: false done
X6 done reply, before its row the reply is not done, or is a done whose outcome, failure or PRs differ from the kept end (canonicalJson), or that is not out: the engine reopened the run, as workflow retry does, another driver ended it again, or the end still has to reach teamai-reports drop done drive the reply's row; the tick goes on as in drive
X7 done tick the run file shows the version kept with the end (status, revision, pull request receipts), and no create unresolved none: no engine next, so no tick publishes an end already out again done