Skip to content

[v2] fix len when x.shape[1] != x_len.max - #44

Merged
xingchensong merged 3 commits into
mainfrom
Mddct-patch-1
Dec 9, 2025
Merged

xingchensong merged 3 commits into
mainfrom
Mddct-patch-1

Conversation

@Mddct

@Mddct Mddct commented Dec 9, 2025 •

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

Release Notes

  • Bug Fixes
    • Improved audio encoding sequence length computation and masking to correctly process padded sequences through convolutional stages.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai

coderabbitai Bot commented Dec 9, 2025 •

Copy link
Copy Markdown

Walkthrough

The AudioEncoderV2 forward path refactors its masking strategy to explicitly compute sequence length T from input, create stage-aware non-pad masks, and recompute masks iteratively after each convolutional stage using intermediate length tracking without altering the final x_len formulas.

Changes

Cohort / File(s) Summary
AudioEncoderV2 masking refactoring
s3tokenizer/model_v2.py
Forward path now computes explicit sequence length T, creates stage-aware non-pad masks using make_non_pad_mask(x_len, T).unsqueeze(1), introduces intermediate length tracking (x_slen), and recalculates masks after each Conv1d layer using updated lengths. No public API changes or modifications to final x_len update formulas.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • Areas of focus:
    • Verify mask computation logic is correct and aligns with intended stage-aware behavior
    • Confirm intermediate length tracking (x_slen) properly updates across convolutional stages
    • Validate that mask application timing and x * mask operations produce correct output shapes and semantics
    • Check for any downstream impacts on model inference or loss computation

Poem

🐰 Masks refined with care so true,
Stage by stage, the lengths renew,
No API breaks, just flow refined,
AudioEncoder's mind redesigned! ✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly addresses the specific bug fix: handling length computation when the actual tensor shape doesn't match the maximum length value.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch Mddct-patch-1

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
s3tokenizer/model_v2.py (1)

332-337: Tie x_slen updates to actual conv strides for future‑proofing

x_slen for the second stage currently uses self.stride in the length formula, while conv2 has a hard‑coded stride=2. This is correct for the current S3TokenizerV2 configuration (where stride is passed as 2), but it bakes in an implicit assumption that self.stride == self.conv2.stride[0]. For robustness if AudioEncoderV2 is ever reused with a different initial stride, consider deriving these lengths from the conv modules themselves, e.g.:

- x_slen = (T + 2 - 1 * (3 - 1) - 1) // self.stride + 1
+ x_slen = (T + 2 * self.conv1.padding[0]
+             - self.conv1.dilation[0] * (self.conv1.kernel_size[0] - 1)
+             - 1) // self.conv1.stride[0] + 1

- x_slen = (x_slen + 2 - 1 * (3 - 1) - 1) // self.stride + 1
+ x_slen = (x_slen + 2 * self.conv2.padding[0]
+             - self.conv2.dilation[0] * (self.conv2.kernel_size[0] - 1)
+             - 1) // self.conv2.stride[0] + 1

This keeps x_slen mechanically consistent with whatever conv parameters are configured.

Please double‑check that these formulas still match the intended conv shapes if you adjust kernel/stride/padding in future.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 268a02d and f3b1f9d.

📒 Files selected for processing (1)
  • s3tokenizer/model_v2.py (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
s3tokenizer/model_v2.py (1)
s3tokenizer/utils.py (1)
  • make_non_pad_mask (270-307)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: unit-test (ubuntu-22.04, 3.10.16)
🔇 Additional comments (1)
s3tokenizer/model_v2.py (1)

328-337: Masking and length tracking now correctly align with padded time dimensions

Using T = x.shape[-1] and passing it as max_len into make_non_pad_mask(x_len, T) ensures the initial mask always has time length T, even when x_len.max() is smaller due to padding. The introduction of x_slen to mirror conv output lengths from the padded T and reusing it as max_len in subsequent make_non_pad_mask(x_len, x_slen) calls keeps mask shapes in lockstep with both Conv1d stages. This directly resolves the shape mismatch when x.shape[-1] != x_len.max() while preserving the existing x_len formulas used downstream.

@xingchensong
xingchensong merged commit 2abe1c5 into main Dec 9, 2025
2 checks passed
@xingchensong
xingchensong deleted the Mddct-patch-1 branch December 9, 2025 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants