From 3a9113688c3b63017a71e18135c502363de14bda Mon Sep 17 00:00:00 2001 From: shivanshu11092003 Date: Fri, 31 Jul 2026 15:34:50 +0530 Subject: [PATCH 1/3] refactor(clips): render clean video filmstrip --- .../.agents/skills/video-editing/SKILL.md | 31 ++ templates/clips/AGENTS.md | 1 + templates/clips/actions/generate-filmstrip.ts | 154 +++++++++ .../actions/get-recording-player-data.ts | 6 + .../actions/lib/ensure-recording-filmstrip.ts | 242 ++++++++++++++ .../actions/lib/ensure-seekable-video.ts | 14 +- .../app/components/editor/editor-layout.tsx | 164 ++++++++-- .../clips/app/components/editor/waveform.tsx | 299 +++++++++++++----- .../clips/app/lib/video-filmstrip.test.ts | 31 ++ templates/clips/app/lib/video-filmstrip.ts | 262 +++++++++++++++ ...hows-video-frames-behind-the-waveform-g.md | 6 + templates/clips/server/db/schema.ts | 10 + .../server/lib/video-filmstrip-sprite.test.ts | 82 +++++ .../server/lib/video-filmstrip-sprite.ts | 207 ++++++++++++ templates/clips/server/lib/video-remux.ts | 17 +- templates/clips/server/plugins/db.ts | 6 + 16 files changed, 1428 insertions(+), 104 deletions(-) create mode 100644 templates/clips/actions/generate-filmstrip.ts create mode 100644 templates/clips/actions/lib/ensure-recording-filmstrip.ts create mode 100644 templates/clips/app/lib/video-filmstrip.test.ts create mode 100644 templates/clips/app/lib/video-filmstrip.ts create mode 100644 templates/clips/changelog/2026-07-31-editor-timeline-now-shows-video-frames-behind-the-waveform-g.md create mode 100644 templates/clips/server/lib/video-filmstrip-sprite.test.ts create mode 100644 templates/clips/server/lib/video-filmstrip-sprite.ts diff --git a/templates/clips/.agents/skills/video-editing/SKILL.md b/templates/clips/.agents/skills/video-editing/SKILL.md index 460fe6d786..5d9394c28f 100644 --- a/templates/clips/.agents/skills/video-editing/SKILL.md +++ b/templates/clips/.agents/skills/video-editing/SKILL.md @@ -229,6 +229,37 @@ downsamples to a 2000-point peaks array. Peaks are cached in recompute. The editor reads the cache first and falls back to computing only when the key is missing or corrupted. +### Timeline filmstrip + +The trim track shows video frames behind the waveform. Two paths, and the order +matters: + +1. **Sprite (preferred).** `generate-filmstrip` runs one ffmpeg pass + (`fps` → `scale` → `pad` → `tile`) into a single JPEG grid, uploads it, and + stores the URL plus grid geometry on the recording. The editor renders cells + with CSS `background-position` — one cached image request, no video decoding + in the browser. +2. **Browser extraction (fallback).** `extractFilmstripThumbnails()` seeks a + detached `