Skip to content

fix(video_compose): resolve and stage edit_decisions.audio.music for Remotion renders - #938

Open
gauravchahal20 wants to merge 2 commits into
OpenMontageInc:mainfrom
gauravchahal20:fix/issue-937-edit-decisions-music
Open

gauravchahal20 wants to merge 2 commits into
OpenMontageInc:mainfrom
gauravchahal20:fix/issue-937-edit-decisions-music

Conversation

@gauravchahal20

@gauravchahal20 gauravchahal20 commented Nov 10, 2026 •

Copy link
Copy Markdown

Summary

Resolves edit_decisions.audio.music and legacy edit_decisions.music against asset_manifest before rendering in VideoCompose. Adapts snake_case fade fields (fade_in_seconds, fade_out_seconds) into the camelCase properties (fadeInSeconds, fadeOutSeconds) expected by Remotion compositions (CinematicRenderer and Explainer), and ensures audio tracks are properly staged into Remotion's public directory. Also fails loudly with a clear error if an explicit asset_id cannot be resolved against asset_manifest rather than silently rendering silence and failing downstream review.

Related issue

Closes #937

Changes

  • Added VideoCompose._resolve_music: resolves edit_decisions.audio.music or top-level edit_decisions.music asset_id to file paths from asset_manifest.
  • Adapted fade fields to both snake_case and camelCase (fadeInSeconds, fadeOutSeconds) to satisfy both Remotion's CinematicRendererProps / Explainer and the underlying artifact schema.
  • Updated VideoCompose._render to pass resolved music into remotion_inputs, _render_via_hyperframes, _render_via_ffmpeg, and _pre_compose_validation.
  • Updated VideoCompose._remotion_render to synchronize props.music (for CinematicRenderer) and props.audio.music (for Explainer) so both composition shapes receive the resolved and staged music configuration.
  • Updated VideoCompose._compose FFmpeg fallback to pick up music if audio_path was not explicitly provided.
  • Added 4 unit tests in tests/tools/test_cinematic_remotion_adapter.py verifying canonical & legacy music resolution, missing asset ID handling, and end-to-end Remotion staging into props.

Testing

Ran the full compose test suite:

  • pytest tests/tools/test_cinematic_remotion_adapter.py (13 passed)
  • pytest tests/tools/test_video_compose_cache_identity.py (35 passed)
  • pytest tests/tools/test_video_compose_chart_palette.py (4 passed)
  • pytest tests/tools/test_video_compose_profile_aspect.py (2 passed)
  • pytest tests/tools/test_video_compose_vertical.py (3 passed)

All 57 tests passed.

Checklist

  • The change is focused on a single logical concern.
  • I ran the relevant tests locally (make test-contracts / make test) where applicable.
  • I updated docs/README if behavior or usage changed.
  • No unrelated files (build artifacts, local config) are included in the diff.

…Remotion renders

Resolves edit_decisions.audio.music and legacy edit_decisions.music against
asset_manifest before rendering. Adapts snake_case fade fields to the
camelCase properties expected by Remotion compositions (CinematicRenderer
and Explainer). Ensures explicit but missing asset_ids fail loudly rather
than silently rendering silence.

Fixes OpenMontageInc#937

@ppramanik62 ppramanik62 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracking this down and filing #937. Resolving audio.music against asset_manifest is the right fix, and the loud error for an unknown asset_id is welcome. I reviewed it against docs/PR_REVIEW_GUIDE.md. There is 1 P1 and 2 P2 to address before merge; details are in the inline threads.

What works. I confirmed the Remotion fix end to end with a real render (Docker: Ubuntu 24.04, Python 3.11, Node 20, npm ci). A 4 s clip with 6 s of music: main's output is silent (mean -91 dB); the PR's is audible (-30.2 dB at volume: 0.5), and volume scales linearly (1.0 -> -24.2 dB, 0.1 -> -44.2 dB).

Blocking

  • [P1] The _compose fallback breaks the ffmpeg runtime (inline on _compose).
  • [P2] The new integration test fails in CI (inline on the test).
  • [P2] "Fails loudly" only covers an unknown asset_id (inline on _resolve_music).

Tests run (same container, 14 related suites: adapter, cache identity, chart palette, profile aspect, vertical, media staging, staging lifecycle, audio mux, hyperframes, caption props, phase1 and phase3 contracts, qa 05, documentary governance): with node_modules, 295 passed and 3 failed on the PR versus 291 passed and 3 failed on main. The 3 failures are hyperframes runtime-check tests that fail identically on main (they need the hyperframes CLI), so they aren't from this PR. Without node_modules the PR adds one failure (the test below).

Not verified: Windows (the platform in #937), hyperframes end to end, and CI itself (workflows need approval for first-time contributors).

Nits (not blocking): the volume now always defaults to 0.15 where Explainer's own default is 0.1, and both snake_case and camelCase fade keys are emitted.

Comment thread tools/video/video_compose.py Outdated
Comment on lines +482 to +490
if not audio_path and edit_decisions:
music_cfg = (
(edit_decisions.get("audio") or {}).get("music")
or edit_decisions.get("music")
)
if isinstance(music_cfg, dict):
cand = music_cfg.get("src") or music_cfg.get("source") or music_cfg.get("path")
if cand and Path(cand).exists():
audio_path = str(Path(cand).resolve())

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] This breaks the ffmpeg runtime. Setting audio_path from music.src feeds the existing -map 0:v -map 1:a -c:a aac -shortest branch below, which treats audio_path as the complete replacement track. volume, fades, loop and ducking are ignored, the clip's own audio is dropped, and -shortest cuts the video to the music length.

Repro (render_runtime: "ffmpeg", 6 s clip that has its own audio, 2 s music asset, volume: 0.1):

  • PR: 1.9 s video, 2.0 s audio, music at full level (mean -21.1 dB, so volume is ignored), clip audio gone
  • main, same inputs: 6.0 s, clip audio kept

_render_via_ffmpeg now passes the resolved music (edit_decisions=resolved_edit_decisions), so this is reachable from the normal render path. Please drop this block. If ffmpeg-runtime music is wanted, it needs a real mix (volume, fades, loop, ducking, as audio_mixer does), as a separate change.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 5d34656 — dropped this block from _compose so clip audio is kept and video duration is not truncated.

Comment thread tools/video/video_compose.py Outdated
Comment on lines +903 to +908
elif Path(music_source).exists():
resolved_path = str(Path(music_source).resolve())
elif music_asset_id:
return None, f"Music asset_id {music_asset_id!r} not found in asset_manifest."
else:
resolved_path = music_source

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] "Fails loudly" only covers an unknown asset_id. When the asset is in the manifest but its file does not exist (or an explicit src points at a missing file), this returns success. Staging then leaves the dead path in the props, so the failure shows up late (mid-render) or as silence. Please check Path(resolved_path).is_file() and return the same kind of error.

Related: the Remotion-unavailable fallback in _render still builds compose_inputs from the unresolved edit_decisions (compose_inputs["edit_decisions"] = dict(edit_decisions, cuts=resolved_cuts)), so music is silently dropped there, which is the #937 symptom. Either pass the resolved decisions or call it out as out of scope.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 5d34656 — added Path(resolved_path).is_file() validation (including file:// URI support) so missing music files fail loudly with a clear error. Also forwarded resolved_edit_decisions in the Remotion-unavailable fallback in _render.

assert err == "Music asset_id 'asset_nonexistent_bed' not found in asset_manifest."


def test_render_resolves_and_stages_music_into_remotion_props(monkeypatch, tmp_path) -> None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] This test fails in CI. It goes through _render -> _needs_remotion -> _remotion_available(), which requires remotion-composer/node_modules. CI installs Python deps only, so the render falls back to ffmpeg and the fake run_command hits StopIteration (empty error string). On a Python-only container: 12 passed, 1 failed in this file; with node_modules installed: 13 passed.

Fix: add monkeypatch.setattr(VideoCompose, "_remotion_available", lambda self: True) at the top of the test (the other render tests in this file call _remotion_render directly, which is why they are unaffected).

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 5d34656 — added monkeypatch for _remotion_available to return True so the test passes cleanly in Python-only CI environments without node_modules.

- Drop ffmpeg audio_path override in _compose to prevent clip audio regression [P1]
- Validate resolved music file exists on disk to fail loudly on missing assets [P2]
- Forward resolved_edit_decisions to compose_inputs in remotion-unavailable fallback
- Mock _remotion_available in render adapter test so CI passes without node_modules [P2]
- Emit only camelCase fade properties and preserve composition volume defaults
@gauravchahal20

Copy link
Copy Markdown
Author

Thanks @ppramanik62 for the thorough review! I've addressed the feedback in commit 5d34656:

  • [P1] FFmpeg runtime preserved: Dropped the music �udio_path assignment block from _compose so clip audio is preserved and video duration is not truncated.
  • [P2] Fail loudly on missing music files: Added is_file() validation in _resolve_music (with ile:// URI decoding) to return a clear error if the music asset or explicit path does not exist on disk. Also forwarded
    esolved_edit_decisions to compose_inputs in the Remotion-unavailable fallback in _render.
  • [P2] CI test failure: Mocked VideoCompose._remotion_available to return True in est_render_resolves_and_stages_music_into_remotion_props so the test passes in Python-only environments without
    ode_modules. Added unit tests for missing file errors.
  • Nits: Only emit �olume when explicitly provided in music_cfg so each renderer uses its natural default (CinematicRenderer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

edit_decisions.audio.music is never read by the render path — documentary-montage renders silent

2 participants