Skip to content

fix(frontend): keep the test set synced across playground commits#4653

Merged
mmabrouk merged 1 commit into
wip/playground-testcase-server-columnsfrom
fix/playground-testset-sync-on-commit
Jun 12, 2026
Merged

fix(frontend): keep the test set synced across playground commits#4653
mmabrouk merged 1 commit into
wip/playground-testcase-server-columnsfrom
fix/playground-testset-sync-on-commit

Conversation

@mmabrouk

@mmabrouk mmabrouk commented Jun 11, 2026

Copy link
Copy Markdown
Member

Context

Committing a prompt in the playground silently disconnected the synced test set. The header dropdown flipped back to "Test set", the "N unused testcase columns hidden" footer disappeared from every row, and the URL snapshot then saved the rows as a local test set instead of a connection. Creating an app from a draft prompt hits the same path (it is the first commit), which is how this was reported: sync a test set, create the app, the hidden columns vanish.

Root cause: the connection lives in loadable state keyed by testset:workflow:<revisionId>. A commit produces a new revision id, so the key changes. The relink step that already carries chat history and execution results across that rename (#4457) never carried the connection, so it stayed stranded on the old key.

This also undermined #4647/#4649: once the snapshot downgrades the rows to a local test set, they lose their server snapshot, and the next Run deletes the unused columns again.

Changes

relinkLoadableSessions now migrates connectedSourceId, connectedSourceName, connectedSourceType, hiddenTestcaseIds, and activeRowId to the new loadable id in the same pass that already moves executionResults, and clears them on the old key.

Before: commit v1 to v2, the dropdown shows "Test set" and the unused-column footers are gone.
After: commit v1 to v2, the dropdown still shows "completion_testset (v2)", footers and values stay, and a reload keeps the connection (the snapshot encodes it again).

Tests

  • 3 new unit tests (anchorSwapConnectionMigration.test.ts) drive the real controller through a commit-style anchor swap with a real Jotai store. Package suite passes 89/89; tsc --noEmit and pnpm lint-fix clean.
  • QA on the dev deployment passed all four flows: create-from-draft keeps the sync, plain commit keeps it, reload keeps it, and Run still preserves the unused column values (the [fix] Keep unused test set columns on playground rows through Run #4649 regression check).
  • Adversarial subagent review found no blockers. Two pre-existing positional-swap flaws now also affect the connection and are left as follow-ups: opening the revision drawer over a stale playground selection mis-moves loadable state (it already mis-moved chat and results before this PR), and in compare mode the index-0 anchor detection can disagree with the real loadable anchor after a column reorder. Fixing both means reworking the Playground resets conversation history after committing revision #4457 anchor detection, out of scope here.

What to QA

  • Prompts > Create new > New prompt, connect a test set that has columns the prompt does not use, click Create. On the new app's playground the test set still shows as synced and every row keeps its "unused testcase column hidden" footer.
  • Edit the prompt and Commit as a new version. The sync, footers, and values survive.
  • Reload the page. Still synced.
  • Run a row. The unused column and its value are still there after the run.
  • Regression: disconnect the test set from the dropdown after a commit. It should disconnect cleanly and fall back to a local test set.

Stacked on #4649 (wip/playground-testcase-server-columns).

Closes #4655.

Committing a draft replaces the anchor revision id, which renames the
derived loadable id (testset:workflow:<revisionId>). The session relink
(AGE-3785) already moved chat history and execution results to the new
key but left the test set connection behind, so every commit silently
unsynced the test set: the dropdown showed unsynced, the unused-columns
footer (connection-gated) vanished, and the URL snapshot downgraded the
rows to a local test set, which also disabled the server-snapshot
protection added for #4647.

Migrate connectedSourceId/Name/Type, hiddenTestcaseIds and activeRowId
to the new loadable id in the same pass that moves executionResults.
@vercel

vercel Bot commented Jun 11, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agenta-documentation Ready Ready Preview, Comment Jun 11, 2026 5:24pm

Request Review

@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. bug Something isn't working Frontend labels Jun 11, 2026
@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 448da183-46ad-4df1-ac90-ca6968a8d2ee

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/playground-testset-sync-on-commit

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Railway Preview Environment

Status Destroyed (PR closed)

Updated at 2026-06-12T12:35:52.251Z

@linear-code

linear-code Bot commented Jun 11, 2026

Copy link
Copy Markdown

AGE-3827

@mmabrouk mmabrouk merged commit aaf18df into wip/playground-testcase-server-columns Jun 12, 2026
49 of 51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working Frontend size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants