Skip to content

perf: optimize ffmpeg video saving #1198

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
starrkk wants to merge 3 commits into
base: main
Choose a base branch
from
Open

perf: optimize ffmpeg video saving #1198

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
166438c
perf: optimize ffmpeg video saving
Jun 30, 2026
3e020da
perf: avoid extra copies when streaming frames to ffmpeg
Jun 30, 2026
498f09f
docs: clarify ffmpeg preset handling
Jul 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 16 additions & 12 deletions lightx2v/utils/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,7 @@ def save_to_video(
# Get ffmpeg executable from imageio_ffmpeg
ffmpeg_exe = ffmpeg.get_ffmpeg_exe()
out_pix = output_pix_fmt or "yuv420p"
ffmpeg_preset = os.environ.get("LIGHTX2V_FFMPEG_PRESET", "").strip()

if lossless:
command = [
Expand All @@ -283,9 +284,10 @@ def save_to_video(
"libx264rgb",
"-crf",
"0",
"-an", # No audio
output_path,
]
if ffmpeg_preset:
command.extend(["-preset", ffmpeg_preset])
command.extend(["-an", output_path]) # No audio
else:
command = [
ffmpeg_exe,
Expand All @@ -308,9 +310,10 @@ def save_to_video(
"libx264",
"-pix_fmt",
out_pix,
"-an", # No audio
output_path,
]
if ffmpeg_preset:
command.extend(["-preset", ffmpeg_preset])
command.extend(["-an", output_path]) # No audio

# Run FFmpeg (stderr to DEVNULL: avoids pipe buffer deadlock; no need to capture for errors)
process = subprocess.Popen(
Expand All @@ -322,14 +325,15 @@ def save_to_video(
if process.stdin is None:
raise BrokenPipeError("No stdin buffer received.")

# Write frames to FFmpeg
for frame in frames:
# Pad frame if needed
if frame.shape[0] < height or frame.shape[1] < width:
padded = np.zeros((height, width, 3), dtype=np.uint8)
padded[: frame.shape[0], : frame.shape[1]] = frame
frame = padded
process.stdin.write(frame.tobytes())
if frames.shape[1] == height and frames.shape[2] == width:
process.stdin.write(np.ascontiguousarray(frames).tobytes())

@gemini-code-assist gemini-code-assist Bot Jun 30, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Calling .tobytes() on a numpy array creates a full copy of the array data as a Python bytes object in memory. For large video tensors, this can cause a significant memory spike and unnecessary CPU overhead.

Since frames is already a contiguous numpy array (due to the .copy() on line 252), and process.stdin.write() accepts any object implementing the buffer protocol (including contiguous numpy arrays), you can write frames directly to the pipe without any copying.

Suggested change
if frames.shape[1] == height and frames.shape[2] == width:
process.stdin.write(np.ascontiguousarray(frames).tobytes())
if frames.shape[1] == height and frames.shape[2] == width:
process.stdin.write(frames)

else:
for frame in frames:
if frame.shape[0] < height or frame.shape[1] < width:
padded = np.zeros((height, width, 3), dtype=np.uint8)
padded[: frame.shape[0], : frame.shape[1]] = frame
frame = padded
process.stdin.write(frame.tobytes())

@gemini-code-assist gemini-code-assist Bot Jun 30, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similarly, calling frame.tobytes() here creates a temporary bytes object copy for every single frame. Since frame is already contiguous (either as a slice of the contiguous frames array or as a newly allocated padded array), you can write frame directly to process.stdin to avoid these per-frame allocations and copies.

Suggested change
for frame in frames:
if frame.shape[0] < height or frame.shape[1] < width:
padded = np.zeros((height, width, 3), dtype=np.uint8)
padded[: frame.shape[0], : frame.shape[1]] = frame
frame = padded
process.stdin.write(frame.tobytes())
for frame in frames:
if frame.shape[0] < height or frame.shape[1] < width:
padded = np.zeros((height, width, 3), dtype=np.uint8)
padded[: frame.shape[0], : frame.shape[1]] = frame
frame = padded
process.stdin.write(frame)


process.stdin.close()
process.wait()
Expand Down