Skip to content

ci(storybook): post an agent-written performance comment on PRs #5232

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sauldom102 wants to merge 3 commits into
base: main
Choose a base branch
from
Open

ci(storybook): post an agent-written performance comment on PRs #5232

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
978114a
feat(storybook): add on-demand performance metrics scripts
sauldom102 Aug 25, 2026
a14bd9f
ci(storybook): post an agent-written performance comment on PRs
sauldom102 Aug 25, 2026
bf3d33a
ci(storybook): trigger the performance report on component changes
sauldom102 Aug 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions .github/agent-prompts/performance.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Performance Report — PR #{{PR_NUMBER}}

You are a frontend performance engineer reporting on pull request #{{PR_NUMBER}} on `{{REPO}}` (branch `{{HEAD_BRANCH}}` → `{{BASE_BRANCH}}`). This is an authorized internal report on Factorial's F0 design system, run automatically in CI with the repository owner's consent.

## Goal

Turn a machine-generated performance measurement into a short, human-readable PR comment that tells the author what — if anything — is worth their attention.

**This check never blocks a merge.** It is informational. Your verdict is always `pass: true` (see Verdict below). You are writing a note to a colleague, not gating their work.

## Inputs

- `/tmp/perf-report.json` — measurements for every story this PR adds or changes. **This is your primary input; read it first.**
- `/tmp/pr.diff` — the PR diff, so you can connect a measurement to the code that caused it.

### Reading the report

Each entry in `stories[]` has:

- `id`, `title`, `name` — which story was measured
- `storyFile` — the file it came from
- `isNew` — whether this PR adds the story
- `deterministic` — counts of work done: `mounts`, `renders`, `updates`, `cascades`, `slowUpdates`, `domElements`, `styleWrites`, `forcedReflows`, `layoutShifts`
- `timing` — wall-clock samples
- `highlights` — pre-computed strings for the things that crossed an attention threshold

**Only `deterministic` numbers and `highlights` are trustworthy.** The `timing` numbers vary run to run on CI hardware and include page-wide work that is not the component (`longTasks` is frequently axe-core, not the story). Do not report a timing number as though it means something, and never describe a component as "slow" or "fast" based on one.

An empty `highlights` array means nothing crossed a threshold. That is the normal, healthy case.

## Instructions

1. Read `/tmp/perf-report.json`.
2. If `stories` is empty, or every story has an empty `highlights` array, say so in one line. Do not manufacture concerns. Do not pad the comment with a table of unremarkable numbers.
3. For each story that does have highlights, look at the diff (and the source file if needed) and try to explain **why** — a `setState` in an effect that could be derived during render, a value or callback rebuilt every render and passed to a memoized child, a layout read after a style write, an unkeyed list. If you cannot find a plausible cause from the diff, say the measurement stands but the cause is not obvious from this change; do not invent one.
4. Group by component rather than listing every story separately. Ten stories from one component with the same highlight is one finding, not ten.
5. Prefer silence over noise. A short comment that names two real things beats a long one that names twelve maybes.

## Important context before you conclude anything

- **A cascade count above zero is normal here.** Every story in this library records at least one render cascade (median 3) because Storybook's own decorators and providers render around the story. The report only raises a cascade highlight well above that norm. Never tell an author to "eliminate render cascades" on the basis of a number the report did not highlight.
- **A new story has no "before" to compare against.** The report measures this PR only; there is no baseline from `{{BASE_BRANCH}}`. So do not claim a change made something "worse", "slower", or "a regression" — you cannot know that. Describe what the numbers are, not how they moved.
- **Do not comment on unchanged components.** Only stories this PR adds or changes are measured, and only those are in scope.
- `truncated: true` means more stories were affected than were measured — mention that the report is partial.

## Writing the comment

Write the comment to `/tmp/perf-comment.md` as GitHub-flavoured markdown. It is posted verbatim on the PR, so it must stand alone.

Structure it as:

- A one-line summary — e.g. `Measured 6 stories across 2 components. Two things worth a look.` or `Measured 4 stories. Nothing stood out.`
- Then, only if there are highlights, a short section per affected component: what was measured, what the likely cause is, and a concrete suggestion.
- Close with a one-line note that this check is informational and never blocks a merge.

Keep it under roughly 250 words unless there are genuinely several distinct findings. Use a table only when comparing three or more stories on the same metric; prose is better for one or two.

Do not include the raw JSON. Do not restate every metric for every story.

## Verdict

After writing `/tmp/perf-comment.md`, output a verdict line in exactly this format. `pass` is **always** `true` — this check reports and never fails a PR, even when it finds something notable:

<!-- VERDICT: {"pass": true, "summary": "Measured N stories across M components; brief note of what was highlighted, or 'nothing notable'."} -->
232 changes: 232 additions & 0 deletions .github/workflows/performance-report.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,232 @@
name: 🚀 Performance Report
on:
pull_request:
branches: [main]
# Kept in step with perf-changed.ts's own filtering: it maps any changed
# source file to the stories that render it, so a component-only change
# counts, but test/snapshot/docs files cannot change what a story renders
# and are excluded there too. Without these negations the workflow would
# build Storybook only for perf-changed.ts to find nothing to measure.
paths:
- "packages/react/src/**"
- "!packages/react/src/**/__tests__/**"
- "!packages/react/src/**/__snapshots__/**"
- "!packages/react/src/**/*.test.ts"
- "!packages/react/src/**/*.test.tsx"
- "!packages/react/src/**/*.spec.ts"
- "!packages/react/src/**/*.spec.tsx"
- "!packages/react/src/**/*.md"
- "!packages/react/src/**/*.mdx"
- ".github/workflows/performance-report.yaml"
- ".github/agent-prompts/performance.md"
- "packages/react/.scripts/perf-changed.ts"
- "packages/react/.scripts/perf-metrics.ts"
types:
- opened
- synchronize
- reopened
- ready_for_review

concurrency:
group: 🚀-performance-report-${{ github.event.pull_request.number }}
cancel-in-progress: true

# This workflow only ever posts a comment. It has no gate job and no required
# check: a performance observation is not a reason to block a merge, and the
# numbers behind it are too environment-sensitive to be a merge condition.

jobs:
# ── Job 1: measure ────────────────────────────────────────────────────────
# Runs the PR's own code (its Vite config, its components, its scripts) to
# build and drive Storybook. It therefore holds NO secrets — see the comment
# on `narrate` for why the two are separate jobs.
measure:
name: "[⚛️ REACT] Measure changed stories"
if: github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.draft == false
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
outputs:
has-report: ${{ steps.measure.outputs.has-report }}
steps:
- uses: actions/checkout@v4
with:
# Full history: perf-changed.ts diffs the PR's story files against
# origin/main to decide what to measure.
fetch-depth: 0
- uses: ./.github/actions/setup-node-pnpm
- name: Get Playwright version
id: playwright-version
run: |
PLAYWRIGHT_VERSION=$(pnpm why @playwright/test -r --json | jq -r '.[] | select(.devDependencies["@playwright/test"]) | .devDependencies["@playwright/test"].version')
echo "version=$PLAYWRIGHT_VERSION" >> $GITHUB_OUTPUT
# Same cache key shape as storybook-tests.yaml so the two jobs share the
# already-warm chromium entry rather than each populating their own.
- name: Cache Playwright browsers
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-chromium-${{ steps.playwright-version.outputs.version }}-${{ runner.os }}
restore-keys: |
playwright-chromium-${{ steps.playwright-version.outputs.version }}-
playwright-chromium-
- name: Install Playwright
timeout-minutes: 5
run: pnpx playwright@${{ steps.playwright-version.outputs.version }} install chromium
# checkout leaves the PR merge ref checked out; make sure the base ref
# perf-changed.ts diffs against actually exists locally.
- name: Ensure base ref is available
run: git fetch origin main --quiet || true
- name: Build Storybook
run: |
pnpm --filter @factorialco/f0-core build
pnpm --filter @factorialco/f0-react run build-storybook --quiet
# Serve the static build and measure. `-s first` tears the server down as
# soon as the measurement exits; `|| true` keeps a measurement failure from
# failing the job, because this workflow must never turn a PR red.
- name: Measure changed stories
id: measure
timeout-minutes: 15
run: |
pnpx concurrently -k -s first -n "SB,PERF" -c "magenta,blue" \
"pnpx http-server packages/react/storybook-static --port 6006 --silent" \
"pnpx wait-on http://localhost:6006 --timeout 60000 && \
pnpm --filter @factorialco/f0-react exec tsx .scripts/perf-changed.ts \
--compare-commit origin/main \
--url http://localhost:6006 \
--out $GITHUB_WORKSPACE/perf-report.json" || true

# Report "nothing to narrate" for a missing report and for one with no
# stories in it, so the agent job is skipped rather than asked to
# describe an empty measurement.
if [[ -f perf-report.json ]] && [[ "$(jq -r '.storiesMeasured // 0' perf-report.json)" -gt 0 ]]; then
echo "has-report=true" >> "$GITHUB_OUTPUT"
jq -r '"Measured \(.storiesMeasured) stories, \(.storiesWithHighlights) with highlights."' perf-report.json
else
echo "has-report=false" >> "$GITHUB_OUTPUT"
echo "No stories measured — skipping the comment."
fi
- name: Upload performance report
if: steps.measure.outputs.has-report == 'true'
uses: actions/upload-artifact@v4
with:
name: perf-report
path: perf-report.json
if-no-files-found: ignore
retention-days: 1

# ── Job 2: narrate + comment ──────────────────────────────────────────────
# Holds the Azure key, so it must not execute PR-authored code. It reads the
# measurement as data (JSON from an artifact) and the prompt/script from the
# BASE branch, mirroring the trust model in agentic-checks.yaml. The head
# checkout is present only so the agent can *read* source while explaining a
# number; nothing from it is executed.
narrate:
name: "[⚛️ REACT] Performance PR comment"
needs: measure
if: needs.measure.outputs.has-report == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 20
permissions:
contents: read
pull-requests: write
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PROMPT_FILE: .github/agent-prompts/performance.md
CHECK_NAME: Performance Report
CHECK_EMOJI: 🚀
MODEL: azure-cognitive-services/gpt-5.3-codex
PR_NUMBER: ${{ github.event.pull_request.number }}
steps:
- name: Checkout base branch (trusted revision)
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.base.sha }}
fetch-depth: 1
- id: copy-trusted
name: Stash trusted script and prompt
run: |-
if [[ -f .github/scripts/agentic-check.sh ]] && [[ -f "$PROMPT_FILE" ]]; then
cp .github/scripts/agentic-check.sh "$RUNNER_TEMP/agentic-check.sh"
cp "$PROMPT_FILE" "$RUNNER_TEMP/prompt.md"
echo "trusted=true" >> "$GITHUB_OUTPUT"
else
echo "::warning::Script or prompt not found on base branch — skipping (running the PR's own copy is not allowed)"
echo "trusted=false" >> "$GITHUB_OUTPUT"
fi
- name: Checkout PR head
uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 1
- name: Restore trusted script and prompt
if: steps.copy-trusted.outputs.trusted == 'true'
run: |-
mkdir -p .github/scripts "$(dirname "$PROMPT_FILE")"
cp "$RUNNER_TEMP/agentic-check.sh" .github/scripts/agentic-check.sh
cp "$RUNNER_TEMP/prompt.md" "$PROMPT_FILE"
# The prompt reads /tmp/perf-report.json; the OpenCode config below is what
# grants access to /tmp.
- name: Download performance report
if: steps.copy-trusted.outputs.trusted == 'true'
uses: actions/download-artifact@v4
with:
name: perf-report
path: /tmp/perf-report-artifact
- name: Stage report for the agent
if: steps.copy-trusted.outputs.trusted == 'true'
run: cp /tmp/perf-report-artifact/perf-report.json /tmp/perf-report.json
- name: Install OpenCode CLI
if: steps.copy-trusted.outputs.trusted == 'true'
run: npm install -g opencode-ai@1.15.5
# Mirrors agentic-checks.yaml: OpenCode probes its own skills directory
# under $HOME at startup, and with only /tmp/* allowed that read is
# auto-rejected and the agent abandons the run without a VERDICT.
- name: Compose OpenCode config
if: steps.copy-trusted.outputs.trusted == 'true'
run: |-
CONFIG=$(jq -nc --arg own_config "${HOME}/.config/opencode/*" '{
snapshot: false,
small_model: "azure-cognitive-services/gpt-5.3-codex",
permission: { external_directory: { "/tmp/*": "allow", ($own_config): "allow" } },
provider: { "azure-cognitive-services": { models: { "gpt-5.3-codex": { name: "GPT 5.3 Codex" } } } }
}')
echo "OPENCODE_CONFIG_CONTENT=${CONFIG}" >> "$GITHUB_ENV"
# `continue-on-error`: the prompt always returns pass=true, so a non-zero
# exit here means the agent itself failed (refusal, crash, no VERDICT). That
# is a reason to post no comment, never a reason to fail the PR.
- id: run-agent
name: Narrate the report
if: steps.copy-trusted.outputs.trusted == 'true'
continue-on-error: true
env:
AZURE_API_KEY: ${{ secrets.DX_AI_WORKFLOWS_API_KEY }}
AZURE_RESOURCE_NAME: platform-dx-ai
run: bash .github/scripts/agentic-check.sh
# The agent writes its comment to /tmp/perf-comment.md (see the prompt).
# Missing or empty file → no comment; add-or-update-pr-comment also skips
# on an empty body, so this is belt and braces.
- id: comment-body
name: Read the agent's comment
if: always() && steps.copy-trusted.outputs.trusted == 'true'
continue-on-error: true
run: |-
if [[ ! -s /tmp/perf-comment.md ]]; then
echo "No comment produced by the agent."
exit 0
fi
{
echo "body<<PERF_COMMENT_EOF"
cat /tmp/perf-comment.md
echo ""
echo "PERF_COMMENT_EOF"
} >> "$GITHUB_OUTPUT"
- name: Post performance PR comment
if: always() && steps.comment-body.outputs.body != ''
continue-on-error: true
uses: ./.github/actions/add-or-update-pr-comment
with:
comment-type: performance_report
github-token: ${{ secrets.GITHUB_TOKEN }}
comment-body: ${{ steps.comment-body.outputs.body }}
Loading
Loading