ci(storybook): post an agent-written performance comment on PRs - #5232
ci(storybook): post an agent-written performance comment on PRs#5232sauldom102 wants to merge 3 commits into
Conversation
Expose the Storybook performance panel's numbers as JSON, so agents and humans can check a component while building or changing it: pnpm perf-metrics F0Button --snapshot perf-changed.ts measures the stories a PR adds or changes and reduces them to the few facts worth attention, for a future PR comment. Highlight thresholds are set from the measured distribution across the library, not by feel: 102 of 102 snapshot stories record at least one render cascade, so "any cascade" would highlight nothing. Layout shift is highlighted by CLS score rather than shift count because the count is not reproducible between runs, while the score is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Wires up the performance report added in #5229: measure the stories a PR changes, have an agent turn the measurements into a short comment, and post it. Informational only — there is no gate job and no required check, because a performance observation is not a reason to block a merge. Split into two jobs on a trust boundary. Measuring builds and drives the PR's own Storybook, so that job executes PR-authored code and holds no secrets. Narrating holds the Azure key, and reads the measurement as JSON from an artifact plus its prompt and script from the base branch, so no PR-authored code runs alongside the key. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
🔍 Review policy: Code changeDefault rule: any other change needs one approval from f0-devs (rule 4). Required approvals
How this was decided
Policy source: |
✅ No New Circular DependenciesNo new circular dependencies detected. Current count: 0 |
📦 Alpha Package Version PublishedUse Use |
🔍 Visual review for your branch is published 🔍Here are the links to: |
✅ No breaking public API changesNo public exports were removed, renamed, or had existing props/types changed in a breaking way compared to Comparing
|
♿ Accessibility (axe) — components changed in this PR✅ No a11y issues in the stories this PR changed. Scope: only stories in the files/component folders this PR changed. It can't yet flag downstream ripple from shared-code/token changes, or diff against |
Coverage Report for packages/react
File CoverageNo changed files found. |
Follows perf-changed.ts now mapping any changed source file to the stories that render it. The paths filter only matched *.stories.tsx, so a component-only change never started the workflow in the first place. The negations mirror the script's own exclusions: test, snapshot and docs files cannot change what a story renders, so without them the workflow would build Storybook only for the measurement to find nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
✅ No untranslated copy addedEvery user-visible string in this PR comes from the i18n layer. Codebase total unchanged at 133. |
✅ Storybook docs — no pages lostEvery page reachable on Links point at this PR's Storybook build — browse the full Storybook. Snapshot of the Storybook index (docs pages + stories) compared against |
Description
Wires up the reporting scripts from #5229 so a PR that changes stories gets a short, agent-written comment describing anything notable about their performance. Informational only: there is no gate job and no required check, because a performance observation is not a reason to block a merge and these numbers are too environment-sensitive to be a merge condition.
Depends on #5229 — this workflow runs
perf-changed.tsand uses theperformance.mdprompt, both of which land there. Merge that one first.Implementation details
🚀 Performance Reportworkflow — measure changed stories, narrate, commentadd-or-update-pr-commentaction under comment typeperformance_report, so re-runs update the comment instead of stacking new onesNotes for reviewers
Why this is two jobs rather than a new entry in agentic-checks.yaml
A trust boundary runs between them.
Measuring has to build and drive the PR's Storybook, which executes PR-authored code — its Vite config, its components, its scripts. Narrating needs
DX_AI_WORKFLOWS_API_KEY. Doing both in one job would run the PR author's code in a process holding that key.So
measurehascontents: readand nosecrets.reference anywhere in the job, and hands its result on as a JSON artifact.narrateholds the key and never executes anything from the head: it reads the measurement as data, and takes its prompt andagentic-check.shfrom the base branch using the same stash-and-restore dance asagentic-checks.yaml. The head checkout in that job exists only so the agent can read source while explaining a number.This is also why it is a separate workflow rather than a matrix entry: adding it to the
agentic-checks.yamlmatrix would put every other check (code review, a11y, storybook, test coverage) behind a Storybook build via a sharedneeds:.It cannot turn a PR red
Deliberately, at four points:
pass: true|| truerun-agentand the comment steps arecontinue-on-error✅ …gate job for branch protection to requireA failed measurement or a model refusal means no comment, never a failed check.
Cost, and when it runs at all
pathsrestricts it to PRs touching*.stories.tsx(or this workflow's own inputs), andnarrateis skipped entirely unlessstoriesMeasured > 0. So the common PR pays nothing.When it does run,
measurecosts one Storybook build (~170s) plus a few seconds per changed story; it runs in parallel with everything else, so wall-clock impact is roughly nil. It reuses the same Playwright cache key asstorybook-tests.yamlrather than populating its own.Gate logic was checked against all three states — a populated report, an empty one, and a missing file:
What triggers a report
perf-changed.tsmaps any changed source file underpackages/react/srcto the stories that render it, walking up to the owning component directory. So a component-only change is measured — verified againstbf41fa6e4(fix(F0Chat)), which reports 39 stories affected where the story-file-only version reported none.Test, snapshot and docs files are excluded in both the script and this
pathsfilter, so they cannot start the workflow or pull a component in. That mattered: on that same commit the two__tests__/files alone attributed 38 stories, meaning a PR that only adjusted assertions would have triggered a full performance comment. A docs-only commit correctly reports nothing.Each measured story records
measuredBecause:"story"when its own file changed,"source"when only its component did. The prompt uses it so the comment attributes a finding to the component rather than implying the author edited a story they never opened.