Repository navigation
fix(video_compose): resolve and stage edit_decisions.audio.music for Remotion renders - #938
gauravchahal20 wants to merge 2 commits into
Conversation
…Remotion renders Resolves edit_decisions.audio.music and legacy edit_decisions.music against asset_manifest before rendering. Adapts snake_case fade fields to the camelCase properties expected by Remotion compositions (CinematicRenderer and Explainer). Ensures explicit but missing asset_ids fail loudly rather than silently rendering silence. Fixes OpenMontageInc#937
ppramanik62
left a comment
There was a problem hiding this comment.
Thanks for tracking this down and filing #937. Resolving audio.music against asset_manifest is the right fix, and the loud error for an unknown asset_id is welcome. I reviewed it against docs/PR_REVIEW_GUIDE.md. There is 1 P1 and 2 P2 to address before merge; details are in the inline threads.
What works. I confirmed the Remotion fix end to end with a real render (Docker: Ubuntu 24.04, Python 3.11, Node 20, npm ci). A 4 s clip with 6 s of music: main's output is silent (mean -91 dB); the PR's is audible (-30.2 dB at volume: 0.5), and volume scales linearly (1.0 -> -24.2 dB, 0.1 -> -44.2 dB).
Blocking
- [P1] The
_composefallback breaks the ffmpeg runtime (inline on_compose). - [P2] The new integration test fails in CI (inline on the test).
- [P2] "Fails loudly" only covers an unknown
asset_id(inline on_resolve_music).
Tests run (same container, 14 related suites: adapter, cache identity, chart palette, profile aspect, vertical, media staging, staging lifecycle, audio mux, hyperframes, caption props, phase1 and phase3 contracts, qa 05, documentary governance): with node_modules, 295 passed and 3 failed on the PR versus 291 passed and 3 failed on main. The 3 failures are hyperframes runtime-check tests that fail identically on main (they need the hyperframes CLI), so they aren't from this PR. Without node_modules the PR adds one failure (the test below).
Not verified: Windows (the platform in #937), hyperframes end to end, and CI itself (workflows need approval for first-time contributors).
Nits (not blocking): the volume now always defaults to 0.15 where Explainer's own default is 0.1, and both snake_case and camelCase fade keys are emitted.
| if not audio_path and edit_decisions: | ||
| music_cfg = ( | ||
| (edit_decisions.get("audio") or {}).get("music") | ||
| or edit_decisions.get("music") | ||
| ) | ||
| if isinstance(music_cfg, dict): | ||
| cand = music_cfg.get("src") or music_cfg.get("source") or music_cfg.get("path") | ||
| if cand and Path(cand).exists(): | ||
| audio_path = str(Path(cand).resolve()) |
There was a problem hiding this comment.
[P1] This breaks the ffmpeg runtime. Setting audio_path from music.src feeds the existing -map 0:v -map 1:a -c:a aac -shortest branch below, which treats audio_path as the complete replacement track. volume, fades, loop and ducking are ignored, the clip's own audio is dropped, and -shortest cuts the video to the music length.
Repro (render_runtime: "ffmpeg", 6 s clip that has its own audio, 2 s music asset, volume: 0.1):
- PR: 1.9 s video, 2.0 s audio, music at full level (mean -21.1 dB, so
volumeis ignored), clip audio gone - main, same inputs: 6.0 s, clip audio kept
_render_via_ffmpeg now passes the resolved music (edit_decisions=resolved_edit_decisions), so this is reachable from the normal render path. Please drop this block. If ffmpeg-runtime music is wanted, it needs a real mix (volume, fades, loop, ducking, as audio_mixer does), as a separate change.
There was a problem hiding this comment.
Addressed in 5d34656 — dropped this block from _compose so clip audio is kept and video duration is not truncated.
| elif Path(music_source).exists(): | ||
| resolved_path = str(Path(music_source).resolve()) | ||
| elif music_asset_id: | ||
| return None, f"Music asset_id {music_asset_id!r} not found in asset_manifest." | ||
| else: | ||
| resolved_path = music_source |
There was a problem hiding this comment.
[P2] "Fails loudly" only covers an unknown asset_id. When the asset is in the manifest but its file does not exist (or an explicit src points at a missing file), this returns success. Staging then leaves the dead path in the props, so the failure shows up late (mid-render) or as silence. Please check Path(resolved_path).is_file() and return the same kind of error.
Related: the Remotion-unavailable fallback in _render still builds compose_inputs from the unresolved edit_decisions (compose_inputs["edit_decisions"] = dict(edit_decisions, cuts=resolved_cuts)), so music is silently dropped there, which is the #937 symptom. Either pass the resolved decisions or call it out as out of scope.
There was a problem hiding this comment.
Addressed in 5d34656 — added Path(resolved_path).is_file() validation (including file:// URI support) so missing music files fail loudly with a clear error. Also forwarded resolved_edit_decisions in the Remotion-unavailable fallback in _render.
| assert err == "Music asset_id 'asset_nonexistent_bed' not found in asset_manifest." | ||
|
|
||
|
|
||
| def test_render_resolves_and_stages_music_into_remotion_props(monkeypatch, tmp_path) -> None: |
There was a problem hiding this comment.
[P2] This test fails in CI. It goes through _render -> _needs_remotion -> _remotion_available(), which requires remotion-composer/node_modules. CI installs Python deps only, so the render falls back to ffmpeg and the fake run_command hits StopIteration (empty error string). On a Python-only container: 12 passed, 1 failed in this file; with node_modules installed: 13 passed.
Fix: add monkeypatch.setattr(VideoCompose, "_remotion_available", lambda self: True) at the top of the test (the other render tests in this file call _remotion_render directly, which is why they are unaffected).
There was a problem hiding this comment.
Addressed in 5d34656 — added monkeypatch for _remotion_available to return True so the test passes cleanly in Python-only CI environments without node_modules.
- Drop ffmpeg audio_path override in _compose to prevent clip audio regression [P1] - Validate resolved music file exists on disk to fail loudly on missing assets [P2] - Forward resolved_edit_decisions to compose_inputs in remotion-unavailable fallback - Mock _remotion_available in render adapter test so CI passes without node_modules [P2] - Emit only camelCase fade properties and preserve composition volume defaults
|
Thanks @ppramanik62 for the thorough review! I've addressed the feedback in commit 5d34656:
|
Summary
Resolves
edit_decisions.audio.musicand legacyedit_decisions.musicagainstasset_manifestbefore rendering inVideoCompose. Adapts snake_case fade fields (fade_in_seconds,fade_out_seconds) into the camelCase properties (fadeInSeconds,fadeOutSeconds) expected by Remotion compositions (CinematicRendererandExplainer), and ensures audio tracks are properly staged into Remotion's public directory. Also fails loudly with a clear error if an explicitasset_idcannot be resolved againstasset_manifestrather than silently rendering silence and failing downstream review.Related issue
Closes #937
Changes
VideoCompose._resolve_music: resolvesedit_decisions.audio.musicor top-leveledit_decisions.musicasset_idto file paths fromasset_manifest.fadeInSeconds,fadeOutSeconds) to satisfy both Remotion'sCinematicRendererProps/Explainerand the underlying artifact schema.VideoCompose._renderto pass resolved music intoremotion_inputs,_render_via_hyperframes,_render_via_ffmpeg, and_pre_compose_validation.VideoCompose._remotion_renderto synchronizeprops.music(forCinematicRenderer) andprops.audio.music(forExplainer) so both composition shapes receive the resolved and staged music configuration.VideoCompose._composeFFmpeg fallback to pick up music ifaudio_pathwas not explicitly provided.tests/tools/test_cinematic_remotion_adapter.pyverifying canonical & legacy music resolution, missing asset ID handling, and end-to-end Remotion staging into props.Testing
Ran the full compose test suite:
pytest tests/tools/test_cinematic_remotion_adapter.py(13 passed)pytest tests/tools/test_video_compose_cache_identity.py(35 passed)pytest tests/tools/test_video_compose_chart_palette.py(4 passed)pytest tests/tools/test_video_compose_profile_aspect.py(2 passed)pytest tests/tools/test_video_compose_vertical.py(3 passed)All 57 tests passed.
Checklist
make test-contracts/make test) where applicable.