Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
11aaee0
feat: add report-driven adapter workflows
esokullu Aug 27, 2026
f3f7077
fix: address adapter workflow review findings
esokullu Aug 28, 2026
bcc5cdf
fix: complete adapter form inventories
esokullu Aug 28, 2026
cade8a1
fix: stabilize workflow reconciliation
esokullu Aug 28, 2026
8605831
fix: bind terminal workflow evidence
esokullu Aug 28, 2026
f0f24a2
fix: handle asynchronous workflow evidence
esokullu Aug 28, 2026
a3c571b
fix: enforce reachable workflow evidence
esokullu Aug 28, 2026
2ab7897
fix: defer scoped workflow evidence
esokullu Aug 28, 2026
160859d
fix: bind exact messaging and publish targets
esokullu Aug 28, 2026
7c6e586
fix: preserve workflow inventory coverage
esokullu Aug 28, 2026
8290f65
fix: bind workflow targets to exact evidence
esokullu Aug 28, 2026
827143a
fix: enforce complete workflow reconciliation
esokullu Aug 28, 2026
3a8d354
fix: bind workflow completion to exact outcomes
esokullu Aug 28, 2026
a632183
fix: close iframe inventory gaps
esokullu Aug 28, 2026
7d080a4
fix: bind workflow evidence to live controls
esokullu Aug 29, 2026
0992a66
fix: retain post-action workflow evidence
esokullu Aug 29, 2026
fd65117
fix: verify consumed uploads after observation
esokullu Aug 29, 2026
2dad833
fix: bind fulfillment to dispatched records
esokullu Aug 29, 2026
667da8f
fix: require exhaustive workflow evidence
esokullu Aug 29, 2026
6575662
Bound form-workflow inventory to a v1 evidence kernel.
esokullu Aug 29, 2026
a8a9c15
Shrink Compact workflow prompts and stop evidence false positives.
cursoragent Aug 29, 2026
cfa2179
Tighten Compact workflow contract under the brief-length budget.
cursoragent Aug 29, 2026
96a50e5
Tighten metadata readback and iframe completeness after review.
cursoragent Aug 29, 2026
7e41a6c
Merge origin/main into report-driven adapter workflows
esokullu Aug 29, 2026
991eb02
Stop inferring optional fields and keep lone failed frames incomplete.
cursoragent Aug 29, 2026
204a063
Merge pull request #326 from esokullu/cursor/compact-workflow-prompt-…
esokullu Aug 29, 2026
414ca27
Escape AX inventory values before quoting them.
cursoragent Aug 29, 2026
03b4422
Merge pull request #327 from esokullu/cursor/codeql-ax-value-escape-8d73
esokullu Aug 29, 2026
9a59eff
Keep live workflow contracts after plan wording edits.
cursoragent Aug 29, 2026
e744f01
Merge pull request #328 from esokullu/cursor/plan-edit-searchbox-inve…
esokullu Aug 29, 2026
b551930
Bind publish success to requested payload and allow empty thread inve…
cursoragent Aug 29, 2026
7a13199
Bind simulated workflow evidence to the current task key in tests.
cursoragent Aug 29, 2026
bf1ab22
Resolve Gmail inline-reply recipients from the enclosing reply contai…
cursoragent Aug 29, 2026
e592953
Stale form inventories after successful value-driven mutations.
cursoragent Aug 29, 2026
5bb5a2a
Refresh form inventories after value mutations in workflow tests.
cursoragent Aug 29, 2026
219bd7e
Preserve form inventory on submit, exact publish payload, and 16 Gmai…
esokullu Aug 29, 2026
3810761
Re-route the site workflow after substantive plan steps edits
esokullu Aug 29, 2026
fd93980
Verify Gmail drafts and subjects, and page large iframe inventories
esokullu Aug 30, 2026
282767a
Keep short-frame iframe rows and bind drafts to authorized field values
esokullu Aug 30, 2026
6a65702
Bind draft addressees and let an empty thread inventory finish
esokullu Aug 30, 2026
12c0526
Hand draft addressees to the guard and rebuild branched form inventories
esokullu Aug 30, 2026
bc8500d
Match requested send bodies and bind release assets to their release
esokullu Aug 30, 2026
6d4d96d
Rebuild paginated inventories on page one and keep body line structure
esokullu Aug 30, 2026
4f42337
Bind form confirmations to their form and keep requested optional rows
esokullu Aug 30, 2026
80840a8
Pin open-thread drafts, read localized resolve controls, drop hidden …
esokullu Aug 30, 2026
7e1395f
Keep finished wizard steps, thread replies, and hidden iframe fields …
esokullu Aug 30, 2026
2914ec9
Keep page-text lines and let iframe-only inventories verify their submit
esokullu Aug 30, 2026
3f1a346
Keep hidden file inputs and match requested fields on content words
esokullu Aug 30, 2026
2b43005
Fail prepare-only jobs that submit, and hold read jobs to their contract
esokullu Aug 30, 2026
19efb9c
Invalidate the mutated iframe, and give every non-submit job a contract
esokullu Aug 30, 2026
890953d
Read the count tool's real fields, check transcript coverage, allow e…
esokullu Aug 30, 2026
684b8c3
Bind each job's evidence to its own resource and stop trusting bare rows
esokullu Aug 30, 2026
c63fc5b
Chain transcript windows, prove empty comments, bound collections
esokullu Aug 30, 2026
c581c80
Bind deferred replies to their thread, tighten label matching, keep i…
esokullu Aug 30, 2026
7bcf0a2
Keep visually replaced native controls in the form inventory
esokullu Aug 30, 2026
724a1ef
Scope thread coverage and drafts, state native optionality, archive i…
esokullu Aug 30, 2026
6e78ff1
Carry transcript and release-asset evidence across Continue
esokullu Aug 30, 2026
cd9f4d7
Treat an unanswered field classifier as inconclusive, and read the wh…
esokullu Aug 30, 2026
cf0ff7f
Close the diff only from an exhaustive reader, in its own coordinates
esokullu Aug 30, 2026
056b171
Close the diff from a root read only, and stop counting disabled cont…
esokullu Aug 30, 2026
96ffebd
Let the last diff window close coverage, and keep in-place wizard rows
esokullu Aug 31, 2026
1f51690
Bind every transcript window to its video, and resolve requested labe…
esokullu Aug 31, 2026
b6e5783
Reread a form after an upload, and name the video on every YouTube route
esokullu Aug 31, 2026
baeb015
Carry a consumed form upload across Continue, and require a real acti…
esokullu Aug 31, 2026
90ac48c
Settle window-based evidence after the window is recorded
esokullu Aug 31, 2026
ee897ba
Group custom ARIA radios, and stop counting readonly controls
esokullu Aug 31, 2026
4bb765c
Verify the booking that was paid for, and the values a form was given
esokullu Aug 31, 2026
1314914
Merge pull request #320 from esokullu/codex/report-driven-adapter-wor…
esokullu Sep 1, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,31 @@ This changelog was generated from the repository Git history and release tags. V

## [33.5.0] - 2026-08-27

### Added
- Added versioned site-workflow contracts for high-evidence GitHub, Product Hunt, Microsoft Forms, Gmail, LinkedIn, YouTube, 12306, Douyin, NaukriGulf, Greenhouse, and Workday tasks (Chrome + Firefox)
- Added semantic planner routing through app-owned `site_job` IDs and content-free adapter/job/revision trace metadata
- Added content-free `adapter_match` trace metadata for notes-only and structured adapters, plus live-UI-verified AdSense and SofaScore guidance without promoting either to a workflow contract

### Changed
- Redesigned the sidebar loading and thinking UI with clearer live activity updates and a toggleable compact activity history (Chrome + Firefox)
- Localized the new activity statuses and improved screen-reader announcements across all supported languages
- Tightened selected site-workflow completion: live-URL binding survives trusted continuation only on the same adapter/job, the executor receives the app-owned stages/evidence contract, submission success needs job-bound terminal evidence (including paid/ticket-issued transaction state and recipient-bound sent confirmation), and ledger-backed workflows need exact reconciliation against an app-owned inventory rather than model-created rows
- Bounded form-workflow inventory v1: exhaustive root reads must not be depth-truncated on includable descendants, skipped required rows cannot prove success (optional `required: false` rows may skip), and checkbox/Next actions stale completeness until a fresh root read

### Fixed
- Compact selected-workflow prompts now inject a brief execution contract and a shorter `progress_update.workflowReconciliation` schema; Mid/Full keep the full contract
- YouTube `update-metadata` verification matches AX-truncated values via prefix plus `value_len`/`value_fp` after the same NFKC normalization used by verification, and values that equal the accessible name
- AX inventory `value=` tokens escape backslash then quote, and inventory readback restores the app-owned string
- Unknown metadata field names no longer discard the rest of the requirement list, but discarded classifier fields keep saved-state verification incomplete; playlist plural aliases are recognized
- Form inventory emits `required=` only for explicit native/`aria-required` state, ignores decorative DOM depth, and omits erroring/empty third-party frames only when another frame already inventoried form controls; a lone failed cross-origin application frame stays incomplete
- Reviewed plan wording edits re-resolve the live site-workflow contract instead of dropping it, and ARIA `searchbox` controls enter the form inventory
- GitHub, LinkedIn, and Douyin publish success now requires the classifier-bound tag, title, notes, body, or visibility on the published resource, not only a re-observed URL
- A complete empty GitHub `resolve-review-threads` inventory can reconcile as a no-op when no unresolved threads exist
- Gmail inline thread replies resolve To/Cc/Bcc chips from the enclosing reply container when the composer is not inside a dialog or form
- Successful `type_ax`, `set_field`, and `iframe_type` mutations stale a complete form inventory so value-driven branching cannot reconcile against the prior snapshot
- Form workflow reconciliation completeness is preserved across final submit actions so post-submit confirmation navigation can complete successfully
- Publication workflow field verification checks exact resource lines and blocks instead of unanchored substrings
- The Gmail recipient probe returns up to 16 candidates to match the schema and guard capacity

## [33.4.1] - 2026-08-27

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,7 @@ Chrome side panel shortcuts work when the WebBrain side panel has focus.
| [Prompt-injection defense](docs/prompt-injection-defense.md) | Defense layers and known gaps |
| [Privacy and data flow](docs/privacy-and-data-flow.md) | What leaves the browser, and what doesn't |
| [Accessibility tree and refs](docs/accessibility-tree-and-refs.md) | How pages are read and targeted |
| [Site adapters](docs/site-adapters.md) | Per-site guidance |
| [Site adapters](docs/site-adapters.md) | Per-site guidance and versioned workflow contracts |
| [Export and workflow formats](docs/export-and-workflow-formats.md) | `webbrain-config/1`, `webbrain-workflow/1` |
| [Adding a tool](docs/adding-a-tool.md) · [Localization](docs/localization.md) · [Test scenarios](docs/test-scenarios.md) | Contributor guides |
| [Community](docs/community.md) | Discord server guide: channels, roles, rules, escalation |
Expand Down
16 changes: 15 additions & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,20 @@ Raw page-source access through `read_page_source` is available only in Dev mode.

Manual action-mode runs (Act or Dev) call the active provider once before the tool loop with `planner.js`'s structured JSON prompt. Off uses the compact intent schema; Try and Strict use the full plan schema. Unset storage defaults to Try, while explicit Off remains Off. The planner sees the user task, sanitized URL/title, and a short recent-history digest; page context is wrapped as untrusted data and image blocks are dropped.

When the active site adapter has a validated `webbrain-adapter-workflow/2`
profile, both planner variants also receive its bounded app-owned job IDs and
descriptions. The planner returns a nullable `site_job`; the runtime resolves
that ID again against the exact active adapter instead of trusting page text or
matching user-language keywords. A selected job can tighten state-change and
require job-bound terminal evidence after submission (for example paid/ticket-
issued transaction state or recipient-bound sent-message state). Jobs that
require a ledger must exactly reconcile terminal ledger IDs against a complete
app-owned accessibility-tree or seeded inventory; model-created rows cannot prove
coverage. Every matched adapter records adapter/revision/notes-injected as
content-free trace metadata, while a selected workflow additionally records its
adapter/revision/job/template. Any user edit to reviewed plan text discards the
hidden job binding.

If the planner returns valid JSON, the side panel receives `agent_update: plan_review` and renders an editable review card. Approval pins the approved plan into the scratchpad so it survives context compaction. Rejection, timeout, or user abort stops the run before any browser tools execute. In Try mode, invalid JSON after one repair degrades only that turn to the Ask prompt and read-only tool catalog; Strict mode still stops before tools. Scheduled runs can set `autoApprovePlanReview` and pin the plan without showing the card.

### Step 5: Main Agent Loop
Expand Down Expand Up @@ -766,7 +780,7 @@ Jobs are stored in `chrome.storage.local` under the key `wb_scheduled_jobs` as a

### Site Adapters (`adapters.js`)

58+ adapters inject site-specific guidance into the first user message (and re-inject on navigation to a different matched site). Only ONE adapter fires at a time (`getActiveAdapter(url)` returns the first match). See `docs/site-adapters.md` for how to write one.
110+ adapters inject site-specific guidance into the first user message (and re-inject on navigation to a different matched site). Only ONE adapter fires at a time (`getActiveAdapter(url)` returns the first match). See `docs/site-adapters.md` for how to write one. Each matched adapter emits a content-free `adapter_match` trace note with its identity, revision, and whether notes were injected. High-evidence repeated tasks may additionally expose validated V2 workflow jobs; the planner selects an app-owned ID semantically, the binding is revalidated on the live pre-execution URL, and the executor receives its trusted stages/evidence contract. Required submissions need dispatch plus post-submit observation, while jobs with a trusted complete inventory may require an explicit job-bound complete-coverage marker whose count matches terminal current-task ledger rows.

Adapters may also expose narrowly scoped runtime policy. Douyin `/chat` is the first `messaging.verifyActiveRecipient` route. The structured planner resolves an anaphoric recipient to `named` only when authentic prior-user context identifies exactly one target; unresolved pronouns clarify, while `active_conversation` is reserved for an explicit reference to the currently open thread itself. An `active_conversation` planner target must first be pinned to exactly one strong visible header identity before any page tool runs; ambiguous or missing evidence stops for clarification. Immediately before a send-like click, submitted field, or Enter press, the content script resolves the exact target and a lower-page layout composer, then collects only unique header evidence from the narrow, non-scrollable region above it. Enter or submitted field input in a different editable is conclusively non-message only when structural semantics positively identify a search or navigation field; an alternate reply/forward/split-pane composer remains inconclusive and therefore cannot bypass recipient verification. A distant general control likewise remains inconclusive rather than being declared safe. A semantic conversation row in a separate left rail is conclusively navigation-only, allowing recovery to the requested thread whether or not the short rail currently overflows; nested row buttons, links, and their leaf descendants plus controls outside that structure remain inconclusive. The agent requires exact normalized identity equality and returns a no-dispatch blocker on missing, inconclusive, ambiguous, or mismatched evidence, and protected Enter dispatch permits exactly one keypress per verification. Every authorized `click`, `click_ax`, `set_field({submit:true})`, and composer Enter receives a one-use binding to the exact action target, composer, URL, and identity set. Direct content dispatches and Chrome's trusted CDP mouse/key paths consume and revalidate it immediately before `el.click()`, `mousePressed`, or Enter, after any field reconciliation and combobox delays; protected `click_ax` never issues a second no-progress fallback click. Search-result text, message content, input values, generic page text, failed probes, and edited plans with stale hidden metadata cannot authorize a send. Dispatch-capable tools whose effects cannot be bound to the probed recipient (`iframe_click`, `execute_js`, WebMCP execution, and `upload_file`, whose change event may auto-send) are unavailable on this protected route. Deterministic saved-workflow replay has no planner-owned recipient target, so a workflow with any potentially dispatching step scoped to a protected messaging route stops before page actions and directs the user to run a normal Act task with a named recipient; a per-step check also covers legacy workflows whose scope metadata is incomplete.

Expand Down
87 changes: 87 additions & 0 deletions docs/site-adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,91 @@ injecting Mastodon guidance more broadly.
- **First turn**: the adapter's `notes` are appended to the first user message in `_enrichUserMessageWithCurrentPage()`.
- **Mid-conversation navigation**: if the user navigates to a URL matching a different adapter, the agent injects a `[Site context changed → now on <name>]` message. Controlled by `_maybeReinjectAdapter()`.

Every matched adapter also emits one content-free `adapter_match` trace note per
adapter revision in a run. It contains only `adapter`, `revision`, and
`notesInjected`; it never copies the notes, page URL, title, or page content.
This measures notes-only adapters as well as structured workflows. When a job
is selected, the existing `adapter_context` note separately records its bounded
workflow identity. Legacy notes-only adapters use implicit revision 1; adapters
with a structured contract use their declared workflow revision.

### Universal Preamble

`UNIVERSAL_PREAMBLE` is injected alongside every system prompt when `useSiteAdapters` is enabled. It covers cookie/consent banners and paywalls — two patterns that appear across the public web and cause LLMs to make bad assumptions.

### Structured Workflow Contracts

Adapters with repeated, evidence-backed tasks may also declare a versioned
`webbrain-adapter-workflow/2` contract. This is runtime policy, not more page
prose. Before either the compact intent planner or the full planner runs,
WebBrain gives it only the active adapter name and a bounded list of app-owned
job IDs plus short descriptions. The planner selects `site_job` semantically,
so routing does not depend on the language of the user's request. Page content
cannot add or select a job, and an ID is accepted only if it still belongs to
the same adapter on the live page immediately before execution. Navigation
while planning or reviewing drops the binding unless the adapter, revision,
schema, and job still match. A trusted Continue turn may retain prior evidence
only after the same live revalidation.

The selected job can only tighten execution. It can require a consequential
tool result, job-bound terminal evidence after its submit/commit dispatch, or
item-level progress-ledger reconciliation. A generic success toast or another
site's submit cannot satisfy the selected job: transaction workflows require a
paid/ticket-issued state, protected messaging requires the recipient-bound
dispatch, an empty composer in that conversation, and a positive sent-status
confirmation, while form/publish/update jobs require their own confirmation
state. A workflow that requires a ledger
must reconcile exact app-owned inventory IDs from a complete accessibility-tree
read or app-seeded expected/classifier targets. Model-created rows—even one
terminal row—cannot prove complete coverage. Form inventory v1 is the last
exhaustive document-root snapshot (`filter: all`, not depth-truncated).
Skipped required rows cannot prove success; optional (`required: false`)
inventory rows may be skipped, and that flag is emitted only when optionality
is explicit (`aria-required="false"`). Missing `required` stays unknown.
Depth truncation is form-relevant (an omitted includable descendant). Empty
or erroring third-party frames are omitted only when another frame already
inventoried form controls; a lone failed cross-origin application frame stays
incomplete. Checkbox/radio/Next actions stale completeness until a
fresh root read. The executor receives the
app-owned stages plus success and partial evidence contract. Edited plan-review
text clears hidden workflow routing instead of retaining stale authorization.
Metadata-only traces retain only the adapter match/injection flag and, for a
selected job, its revision, schema, job, and template—never notes, page, form,
or message content.

```js
{
name: 'example-forms',
category: 'general',
revision: 1,
regions: ['global'],
jobs: ['submit-form'],
workflow: {
schema: 'webbrain-adapter-workflow/2',
jobs: {
'submit-form': {
description: 'Fill, review, submit, and verify the form.',
template: 'form',
stateChange: true,
requiresSubmission: true,
requiresLedger: true,
stages: ['inventory', 'fill', 'review', 'reconcile', 'commit', 'verify'],
successEvidence: ['A post-submit confirmation is visible.'],
partialEvidence: ['Completed and unresolved questions plus the blocker are reported.'],
},
},
},
matches: (url) => /^https:\/\/forms\.example\.com\//.test(url),
notes: `...`,
}
```

`workflow.jobs` must exactly match `jobs`. Job IDs and stages are stable
identifiers; evidence strings are bounded behavioral requirements, not CSS
selectors. `requiresSubmission` implies `stateChange`, every job includes
`verify`, and submission jobs also include `commit`. Bump `revision` whenever
the behavioral contract changes.

---

## Adapter Format
Expand All @@ -69,6 +150,10 @@ injecting Mastodon guidance more broadly.
| `category` | `'general'` or `'finance'` | `'finance'` adds a `[FINANCE / HIGH-STAKES]` banner to the heading and triggers extra safety guidance in the system prompt. |
| `matches` | `(url) => boolean` | Returns `true` when the adapter should fire for this URL. Regex is preferred — keep it specific enough to avoid false matches. |
| `notes` | string | Bulleted guidance injected into the first user message. **Keep 4–8 lines max.** See style guidance below. |
| `revision` | positive integer | Optional workflow-contract revision. Required when any structured workflow field is present. |
| `regions` | string[] | Stable regions where the structured job contract applies, such as `global`, `CN`, or `MENA`. |
| `jobs` | string[] | Stable planner-routing IDs. Must exactly match `workflow.jobs`. |
| `workflow` | object | Optional validated `webbrain-adapter-workflow/2` job contract used to tighten runtime completion. |

### Ordering

Expand Down Expand Up @@ -147,6 +232,7 @@ Adapters are ordered by category/site in the `ADAPTERS` array. **Finance adapter
3. **Verify the notes appear**: in Ask, Act, or Dev mode, type a simple instruction (e.g., "what's on this page?"). Open the side panel's verbose mode and confirm the first user message contains `[Site guidance for <name>]` with your notes.
4. **Verify only ONE adapter fires**: navigate to a URL that could match multiple matchers. Check that the first match wins and no others leak through.
5. **Test navigation re-injection**: start a conversation on a non-adapted site, then navigate to your adapted site. Confirm a `[Site context changed]` message appears.
6. **If a workflow contract is present**: validate `listAdapterWorkflowProfiles()`, verify planner routing for positive and negative URLs, and test state-change, submission, ledger, trace-privacy, and Chrome/Firefox parity behavior.

### Manual test URLs

Expand All @@ -166,4 +252,5 @@ Open each adapted site and verify:
- [ ] Verify the notes are 4–8 concise bullets
- [ ] Test matching with `getActiveAdapter(url)`
- [ ] Test end-to-end with the extension loaded
- [ ] For a structured profile, prove the job from repeated traces or current UI evidence; validate exact URL routing and completion requirements
- [ ] If the adapter targets a non-English market, add localized label hints (see the WordPress adapter for an example of how to annotate non-English UI labels)
Loading
Loading