test: add dedicated VRT stories for components and patterns - #6511
Draft
rubencarvalho wants to merge 64 commits into
Draft
test: add dedicated VRT stories for components and patterns#6511rubencarvalho wants to merge 64 commits into
rubencarvalho wants to merge 64 commits into
Conversation
Chromatic was snapping every docs story automatically, and reviewing VRT-only coverage in local Storybook was hard since none of it had its own sidenav entry. This adds hand-authored `*.vrt.ts` story files (a new Storybook indexer/glob, discovered like `.stories.ts` files) that combine permutations into as few snapshots as possible, nested under `<Component>/VRT` in the sidenav. Chromatic's snapshotting now defaults to disabled globally and is re-enabled only on VRT files. A new `vrt` Storybook mode (env-gated, CI-only) builds just the `*.vrt.ts` files so the Chromatic-hosted catalog only lists what it actually snapshots; local dev keeps the full story set as before. Prototyped on Button; a shared `row()` helper and the existing `staticColorsDemo` decorator (already used by docs stories) are exported for reuse by future component VRT files.
…nd text behaviors Combine light/ltr and dark/rtl coverage, icon-slot anatomy (label-only, icon+label, icon-only), and text wrapping/truncation into the single Permutations story so it still costs one Chromatic snapshot. Adds shared `theme()` and `staticColorBackground()` helpers to .storybook/helpers/vrt.ts for reuse across future component VRT files. Drops getStorybookHelpers' template()/args/argTypes in favor of plain lit `html` bindings: more legible for a Controls-free story, and it avoids a real bug in observe-slot-text.ts, where a Lit child-position marker comment left by template()'s icon-slot serialization gets misread as label text, permanently defeating Button's `iconOnly` detection. Uses the same static-markup workaround button.stories.ts's own Anatomy story already relies on.
PR preview deploys were building the production-trimmed Storybook, which excludes internal stories, .test.ts fixtures, and (previously) VRT stories. Add an explicit SWC_STORYBOOK_MODE=dev override in main.ts, since `storybook build` forces NODE_ENV=production itself before the config runs, and wire it into preview-docs.yml. Also fixes an unescaped `|` inside a GFM table code span in the sidenav accessibility migration analysis doc, which broke MDX parsing once contributor-docs was included in a static build for the first time (previously only ever served live via `storybook dev`).
:hover and :active can't be triggered by synthetic events, and static VRT captures have no real pointer; :focus-visible has a similar heuristic gotcha. Adds a forcePseudoState() helper that mirrors a component's own :hover/:focus-visible/:active rules (in its shadow root's adopted stylesheets) into equivalent class selectors, then applies the matching class to its internal element. Takes the core trick from aziz/storybook-test-grid's pseudo-states tooling, but drops the generic data-vrt-host/data-vrt-control indirection, the toolbar toggle, and the recursive whole-document tree walk — none needed since we hand-write each VRT file and already know exactly which internal part needs the class. Wired into button.vrt.ts's Permutations story via a `play` function, covering hover/focus-visible/active for every variant.
Button's pseudo-states are styled via an internal shadow part (.swc-Button:hover); Tabs styles them directly on :host() (see tab.css). forcePseudoState()'s internalSelector param is now optional — omit it to force the class onto the host element itself instead of querying inside its shadow root. Reordered the signature to (host, state, internalSelector?) so the common case doesn't need a placeholder argument. Adds tab.vrt.ts as the second VRT file, covering both Tab densities (regular/compact) with default/selected/disabled tabs, plus forced hover/focus-visible/active. Confirmed visually (not just by class presence) that the mirrored :host(:hover)/:host(:active)/ :host(:focus-visible) rules actually render — text-color differences for hover/active, and the focus ring outline for focus-visible.
SWC intentionally never sets a page-level default text color (left to the consuming app, same policy as background), so theme()'s dark wrapper only setting background-color left plain slotted content (e.g. TabPanel's text) with nothing to inherit — it stayed illegible dark-on-dark. Add a matching color alongside background-color. Found via tab.vrt.ts's dark/rtl block.
static-color collapses semantic variants into two treatments: "solid" (primary/accent/negative, sharing one set of tokens) and "secondary" (its own, more subtle transparent-*-200/900 tokens) -- confirmed in button.css's :host([static-color=...][variant="secondary"]) overrides. The static-color rows only covered the solid treatment. Adds secondary alongside solid for both the base fill/outline permutations and the forced hover/focus-visible/active states, for both static-color white and black.
tab.vrt.ts was added to validate forcePseudoState()'s :host()-based pseudo-state support against a second component. That validation is done; keeping this PR scoped to Button, its dedicated VRT prototype. The :host() support itself stays in pseudo-state.ts for reuse whenever Tabs (or another :host()-styled component) gets its own VRT file.
Replace the composed-boolean-flags version with a flat STORIES_BY_MODE record: each mode's story list is spelled out in full, so "what does build actually include?" is answered by reading one array instead of tracing conditionals scattered through the file. Shared chunks (GUIDES, CORE_AND_CONTRIBUTOR_DOCS, TEST_FIXTURES) are only pulled out where the exact same array is reused verbatim across modes. Also drops `as const` from the extracted story-root/list constants — StorybookConfig['stories'] entries just want plain `string` fields, so it wasn't doing anything. Verified via yarn analyze + a build per mode (typescript.check: true catches type errors) that story counts are unchanged: dev 871, build 286, ci-a11y 252, vrt 1.
Chromatic supports `forced-colors` natively via parameters.chromatic.forcedColors (see tooltip.test.ts's ForcedColorsOpenTest for the existing pattern) -- a real browser media-feature emulation, not a hack like the :hover/:active/ :focus-visible forcing in pseudo-state.ts. Because it flips the whole page's palette rather than being scoped to a subtree, it can't fold into the existing Permutations snapshot the way light/dark/rtl does, so it gets its own story/snapshot. Verified via Playwright's forced-colors media emulation (the same mechanism Chromatic uses) that Button's @media (forced-colors: active) overrides actually render correctly.
Pulling the .vrt.ts glob out of the shared stories array (so it wouldn't leak into the production build) also removed it from dev's array, since both used to share that base list. dev should still show VRT stories locally -- only build/ci-a11y need to exclude them. Adds a shared VRT_STORIES chunk, included in dev's array and reused as-is for vrt mode's own array.
SWC_STORYBOOK_MODE=vrt set as a workflow-step env var only reached CI's chromaui/action run, not the local Chromatic "Visual Tests" addon panel -- that panel runs its own storybook build in the background, inheriting whatever environment the dev server itself was started with, which defaults to `build` (production) mode. buildScriptName is a real, validated Chromatic config option (not just CLI/addon UI metadata), read by the CLI, chromaui/action, and the local addon panel alike. Point it at a new storybook:build:vrt script that bakes SWC_STORYBOOK_MODE=vrt into the script itself, so every trigger path resolves the same way without depending on ambient shell state. Drops the now-redundant env override from chromatic-vrt.yml.
Registering an indexer alone does nothing -- it only runs against files the `stories` glob array actually surfaces. Only dev's array (via TEST_FIXTURES) includes a **/*.test.ts pattern at all, so gating testStoryIndexer on storybookMode === 'dev' was redundant; the stories config was already the real gate. Verified builds for build/ci-a11y/vrt are unaffected (286/252/2 entries, same as before).
forcePseudoState() previously only mirrored a shadow root's *adopted* stylesheets, so it silently did nothing for native <a>/<button> elements styled via global-button.css's classes -- that stylesheet applies at the document level, wrapped in `@layer swc-global-elements`, not a shadow root. Adds a document-level mirroring path: walks document.styleSheets and injects a <style> element with the mirrored rules, applied when a target has no shadowRoot. Also generalizes the rule collector to recurse into grouping rules (@media/@layer/@supports/@container) -- needed here since global-button.css wraps everything in one @layer, unlike Button/Tab's shadow-scoped rules which were never nested.
Full audit of Button's public API against button.vrt.ts's existing coverage turned up two gaps: - justified was untested. Added at a comfortable width and a very narrow one (a "min width" check -- confirms the button wraps text gracefully instead of collapsing or overflowing, since there's no enforced minimum). - pendingLabel is deliberately NOT added: it only changes the accessible name while pending, no visual effect, so it doesn't belong in a VRT. Adds two new stories: - GlobalStyles: native <a>/<button> elements using global-button.css's BEM classes (variant/fill-style, sizes, static colors, icon anatomy, truncate, justified, and forced hover/focus-visible/active), to confirm the shared stylesheet produces identical results to the swc-button component it's generated from, regardless of element type. `disabled` is button-only, matching the documented limitation that native links can't support a real disabled state. - CustomProperties: every `--swc-button-*` custom property is a public contract (see the Global Element Styling guide) -- a future CSS refactor that quietly drops one would be a breaking change. One row per property: a reference button next to the same button with that one property overridden to an obviously different value, forcing the relevant pseudo-state/disabled state for state-dependent properties so the difference is actually visible. ForcedColors also gained the same forced hover/focus-visible/active play function as Permutations, since forced-colors mode has its own UA-mandated focus-ring behavior worth confirming alongside the system-color palette. The repeated play-function logic across all three shadow-DOM-aware stories is now a shared applyForcedStates() helper.
…leSnapshot dance Dedupes the static-color gradient constant and ForcedPseudoState type, caches pseudo-state CSS mirrors by stylesheet instead of by shadow root, collapses three identical play functions into one, and removes the now-vestigial chromatic.disableSnapshot opt-in/opt-out now that the vrt build mode's stories glob already limits Chromatic's catalog to *.vrt.ts files only.
Renders icon+label and icon-only permutations at every size instead of just the default, and drops the now-redundant standalone label-only row since the sized icon+label rows already cover it.
That change is now its own dedicated PR (#6473). Removes the unused SWC_STORYBOOK_MODE=dev branch from main.ts along with the preview-docs.yml wiring, since nothing else in this branch sets that env var.
- split button vrt coverage into focused stories so each snapshot has a clear purpose - move common vrt rendering, pseudo-state, and parameter setup into shared helpers - keep button permutations data-driven to make future component vrt stories easier to scan Co-authored-by: Cursor <cursoragent@cursor.com>
- group global stylesheet coverage so the snapshot is easier to scan - reuse shared custom property vrt helpers to keep component stories smaller - verify documented custom properties remain covered by the vrt cases Co-authored-by: Cursor <cursoragent@cursor.com>
- add a short vrt-authoring skill for dedicated visual regression stories - point migration testing guidance at the dedicated vrt file pattern - update contributor testing docs to describe the shared helper approach Co-authored-by: Cursor <cursoragent@cursor.com>
- cover CJK button text, truncation, and icon spacing in the visual matrix - clarify VRT authoring guidance to prefer maximum meaningful coverage Co-authored-by: Cursor <cursoragent@cursor.com>
Broaden VRT authoring guidance from components-only to components and patterns: pattern test/vrt paths, deterministic realistic content for composed patterns, the global-<component>.css coverage check, and a Storybook story-sort order for VRT subsections under Components and Patterns.
Add dedicated .vrt.ts visual regression coverage across 20 components and 11 conversational-ai patterns, including permutation matrices, forced-colors, global-styles, and custom-property stories where applicable.
|
VRT stories live under components/ and match the coverage include glob, so they were being counted in code-coverage metrics like source. They are visual-test scaffolding, not product code; exclude *.vrt.ts alongside the existing *.test.ts and *.stories.ts exclusions.
…st-add-vrts-to-components
Accordion, color-handle, color-loupe, and avatar bundled several states under one generic 'States' row whose items render near-identical visuals with no visible per-item label, so a reviewer could not tell which state was which. Give each state its own row labeled with the state name, the same way button's pseudo-state rows read.
Document that a multi-item row only works when each item carries visible text that names it; small controls and fixed-content widgets whose states look near-identical should get one labeled row per state instead of a generic 'States' row.
…st-add-vrts-to-components
Storybook derives each story's display name from its export name using Title Case (ForcedColors -> 'Forced Colors'), which doesn't match this project's sentence-case convention. Rather than adding a storyName override to every multi-word export across every .vrt.ts file, lowercase every word but the first (preserving all-caps acronyms) once in the vrtStoryIndexer, so every VRT story picks up the fix automatically.
…st-add-vrts-to-components
Tooltip defaults to popover="auto", which shares one page-wide top-layer dismissal group: opening one instance light-dismisses any other open auto popover, and Tooltip's own toggle listener syncs that dismissal back into its open property. tooltip.vrt.ts rendered many open tooltips side by side (every variant, every placement), so only the last one connected was ever actually open by the time Chromatic snapshotted it — every earlier instance silently closed itself during initial render. Add a shared forceManualPopover play-function helper that switches each instance to popover="manual" (no cross-instance dismissal) and re-toggles its open property so the component's real open lifecycle runs again under the new mode. Shaped to be reusable for Popover, whose default-mode popover lives on an internal shadow element rather than the host.
PlacementController positions a tooltip relative to a trigger resolved via for="id"; with no trigger to measure against, every tooltip rendered at the popover's unpositioned default (the viewport corner), regardless of its declared placement. Pair each tooltip with a small swc-button trigger and link them via for, mirroring the pattern the docs stories already use. tooltipContent() runs twice per story (once per theme() block sharing one document), so ids now take an idPrefix argument to stay globally unique across both copies.
Six placement tooltips forced open simultaneously overlapped badly in a plain row(): a left/right/start/end tooltip's bubble is top-layer content positioned by translate, not clipped by its trigger's local box, so it spilled into the neighboring cell's trigger and tooltip. Arrange the six placements on the same compass-rose grid the docs stories use (top/right/end/bottom/left/start around an empty center), so every direction has a blank cell to pop into. The docs story only opens one tooltip at a time and never hits this; VRT forces all six open at once, so it needs the room the grid provides.
Show each Placements grid cell's placement value as its tooltip content (instead of the generic 'Tooltip content' default) so the snapshot itself identifies which direction each tooltip demonstrates.
vrtParameters' outer flex-column gap (16px / --swc-spacing-300) packed each story's rows tightly enough that row labels and content crowded each other. Every .vrt.ts file shares this parameter (vrtParameters or forcedColorsVrtParameters), so bumping it to --swc-spacing-500 (32px) gives every VRT story more breathing room between rows in one change.
row()'s label wrapper used a hardcoded 4px gap between the row label and its content, tighter than any other spacing in the shared VRT helpers. Use --swc-spacing-100 (8px) instead, applying to every row() call across all .vrt.ts files.
Bump vrtParameters' outer gap from --swc-spacing-500 (32px) to --swc-spacing-700 (48px) for more breathing room between rows across all .vrt.ts stories.
--swc-spacing-500 (32px) was the right amount; --swc-spacing-700 (48px) was too much.
row()'s inner content wrapper (both the labeled and unlabeled branches) used a hardcoded 16px gap between items within a row, not a design token. Use --swc-spacing-400 (24px) instead, applying to every row() call across all .vrt.ts files.
…orBackground() theme()'s and staticColorBackground()'s outer wrappers used a hardcoded 16px gap between stacked rows, not a design token. Use --swc-spacing-400 (24px), matching row()'s item gap.
Match the spacing pass done on the VRT rollout branch, since button.vrt.ts shares these same helpers: - vrtParameters' outer gap: 16px -> --swc-spacing-500 (32px), more room between rows. - row()'s label-to-content gap: 4px -> --swc-spacing-100 (8px). - row()'s item gap (both branches): 16px -> --swc-spacing-400 (24px), and tokenized. - theme() and staticColorBackground()'s outer gaps: 16px -> --swc-spacing-400, and tokenized.
Popover's VRT coverage previously lived as 10 individually-tagged stories
inline in popover.stories.ts (VrtPlacementTop, VrtSizeSmall, VrtModal,
VrtNested, ...), each opening exactly one popover per Chromatic snapshot to
avoid the popover="auto" light-dismiss collision. Consolidate onto the same
dedicated test/vrt/*.vrt.ts pattern used by every other component:
- One row per placement (12 total; a compass grid, as used for Tooltip's 6
placements, does not generalize to Popover's 4-direction x 3-alignment
matrix), plus one row each for sizes and hide-arrow.
- forceManualPopover keeps every non-modal instance open simultaneously,
scoped to :not([modal]) and resolving the popover-bearing element from
Popover's internal shadow child (.swc-Popover) rather than the host,
since Popover's default mode attaches popover="auto" there instead of
on the host the way Tooltip does.
- Modal mode keeps its own separate story: showModal()'s ::backdrop
covers the full viewport, which would dim every other row's snapshot if
folded into the same story.
Removed the now-redundant inline Vrt* stories and the stale meta-level
chromatic: { disableSnapshot } override from popover.stories.ts (dead
weight either way: Chromatic's CI build uses chromatic.config.json's
buildScriptName, which only scans *.vrt.ts files via SWC_STORYBOOK_MODE=vrt,
so .stories.ts files were never part of that catalog).
…ok build button-custom-properties.vrt.ts statically imports the build-generated, gitignored .storybook/custom-elements.json. In CI, Chromatic's build has failed repeatedly with an unresolved-import error immediately after yarn analyze reports success, while the same command succeeds reliably locally; cem's own write (fs.writeFileSync) is synchronous, so the failure is not inside that tool. Add a `test -s` gate between analyze and the storybook build so a missing or empty manifest fails fast with a clear signal, instead of surfacing deep inside Rollup as a confusing "could not resolve import" error.
…st-add-vrts-to-components
main moved the generated custom-elements.json manifest from .storybook/custom-elements.json to dist/custom-elements.json (cem.config.js outdir, preview.ts, blocks/*.tsx, package.json's customElements field). This branch was 58 commits behind main and still pointed at the old path in button-custom-properties.vrt.ts and the new verify-cem-manifest.mjs script, which is what Chromatic's CI build was actually failing to resolve. Verified end-to-end: a clean `yarn storybook:build:vrt` (no prior dist/ or .storybook/custom-elements.json) now builds successfully.
…st-add-vrts-to-components
… components main moved the generated custom-elements.json manifest from .storybook/custom-elements.json to dist/custom-elements.json. The merge from feat-dedicated-vrt-stories brought that change plus the button fix; update the remaining 11 *-custom-properties.vrt.ts files (accordion, action-button, avatar, badge, button-group, divider, icon, meter, progress-circle, status-light, tooltip) to match. Verified end-to-end: a clean `yarn storybook:build:vrt` (no prior dist/ or .storybook/custom-elements.json) builds successfully across all 45 VRT files plus Popover.
The actual bug was a stale custom-elements.json path (now fixed), not a race in cem's file write (which is synchronous). The verify script doesn't prevent anything the build's own unresolved-import error wouldn't already catch; keeping it around is unnecessary ceremony for a problem that's already root-caused.
…ts-to-components # Conflicts: # 2nd-gen/packages/swc/.storybook/helpers/index.ts # 2nd-gen/packages/swc/.storybook/helpers/vrt.ts # 2nd-gen/packages/swc/.storybook/main.ts
Contributor
|
This is a reference PR we would like to separate this PR into multiple batches. This is the plan
Each unit's VRT file only touches its own test/vrt/ folder, so these batches are independent and can be merged in any order |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds dedicated
.vrt.tsvisual regression stories across 20 components and 11 conversational-AI patterns, following the dedicated-VRT-stories pattern established in this base branch.Each unit gets a permutation matrix covering its meaningful visual axes (sizes, variants, states, anatomy, themes, static-color), plus separate stories where applicable for:
stylesheets/global/)Composed patterns use deterministic realistic content (fixed prompts, sources, attachments, feedback states, response text) rather than behavior demos, so snapshots stay stable.
Motivation and context
Demonstrates how the dedicated VRT authoring approach scales across the full component and pattern set, so the team can review the story shape, coverage density, and Storybook sidebar ordering before we standardize on it.
Related issue(s)
Notes for reviewers
This is a demo/preview PR targeting the feature branch, not
main. Open Storybook and look under each component/pattern's VRT group (Permutations → Forced Colors → Global Styles → Custom Properties).