Skip to content

Unify cards across all four text subtasks - #61

Merged
SkalskiP merged 8 commits into
mainfrom
feat/structured-json-cards
Oct 12, 2026
Merged

SkalskiP merged 8 commits into
mainfrom
feat/structured-json-cards

Conversation

@SkalskiP

@SkalskiP SkalskiP commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Text result cards use inconsistent comparisons, and long JSON responses spill over multiple cards. This change gives all four text subtasks a consistent native frame, purple differences, muted struck reference text, and a bottom metric drawn from the saved result.

  • Structured JSON: one card, conditional full-width grouped field map with total/difference counts, selected values with a Reference/Model key, and FIELD F1. Invalid/empty responses show a minimal centered message.

  • Transcription: 18 px text-only inline comparison with original line breaks, no map, and an ellipsis on overflow. The full-response saved Character Similarity (or Exact Match) score remains visible.

  • Single-value extraction: the question above a centered 24 px comparison. Correct answers appear once in neutral text; mismatches show a muted struck reference above the purple prediction. EXACT MATCH remains at the bottom. Long questions and answers wrap and truncate within the panel.

  • Localization + recognition: preserve the original free-standing crop layout, use muted struck references and purple model text, and show saved BOX + TEXT F1. Only one-to-one spatial matches (IoU > 0.5) enter the crop list; missing and extra boxes still affect the unchanged score and remain visible in the source overlay. Empty, malformed and unmatched outputs use the same centered purple message rendering as JSON.

Source-image layout, identity header and brand footer are preserved. No inference, scoring or raw-results changes.

Validation: 336 tests passed; Ruff check/format and git diff checks passed. Validate passes with existing benchmark warnings; summary --check confirms the payload matches saved results. Rendered 18 native single-value and 15 transcription examples across Astra, Qwen and DeepSeek; structured rendering previously covered all 690 cases across those models, followed by 14 final-design spot checks. Rendered 21 localization examples across Astra, Qwen and DeepSeek, including the zero-match comic example and invalid responses. Regression tests cover the strict IoU threshold, text mismatches within matched boxes, excluded missing/extra boxes, shared failure-state pixels, bounded JSON and text, Unicode/font handling, line breaks, and long single-value questions/responses.

Implemented in an isolated worktree to preserve local changes.

Review fixes: single-value newline/tab markers are measured before wrapping, eliminating overlap and clipping across all 209 saved responses containing those characters. Transcription now marks changed whitespace while preserving unchanged line breaks; a saved newline-only mismatch is no longer pixel-identical to a correct comparison. Added regressions for both cases.

@SkalskiP SkalskiP changed the title Render structured JSON results on a single card Update structured JSON and transcription comparison cards Oct 12, 2026
@SkalskiP SkalskiP changed the title Update structured JSON and transcription comparison cards Unify structured JSON, transcription and single-value cards Oct 12, 2026
@SkalskiP SkalskiP changed the title Unify structured JSON, transcription and single-value cards Unify cards across all four text subtasks Oct 12, 2026
@SkalskiP
SkalskiP merged commit fdf7caf into main Oct 12, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant