Skip to content

Fix FSQ GPU code packing precision bug - #52

Open
zzuzxy0430 wants to merge 1 commit into
xingchensong:mainfrom
zzuzxy0430:fix/fsq-gpu-code-packing
Open

zzuzxy0430 wants to merge 1 commit into
xingchensong:mainfrom
zzuzxy0430:fix/fsq-gpu-code-packing

Conversation

@zzuzxy0430

@zzuzxy0430 zzuzxy0430 commented Apr 7, 2026 •

Copy link
Copy Markdown

Summary

Fixes #49
Fix a CUDA-specific precision issue in FSQCodebook.encode().

On some CUDA stacks, floating-point torch.pow(3, x) does not always return
exact integers. For example, 3**5 can become 242.99998, and the subsequent
.int() cast truncates the packed token id by 1.

This caused systematic token mismatches such as:

  • 243 -> 242
  • 488 -> 487
  • 245 -> 244

Root Cause

FSQCodebook.encode() used floating-point arithmetic for code packing:

  • float torch.pow(...)
  • float accumulation
  • final .int() truncation

On affected GPUs this produced incorrect packed token ids.

Fix

  • convert rounded FSQ digits to int64
  • build integer powers of self.level
  • perform packing in integer space
  • return int32

Validation

  • reproduced CUDA torch.pow(3, x) precision issue locally
  • added regression test for exact FSQ code packing
  • verified the environment2 GPU mismatch count dropped from 19 to 0

Summary by CodeRabbit

  • Bug Fixes

    • Improved numerical precision in quantization code packing through optimized integer-based computation for more accurate and consistent token ID generation.
  • Tests

    • Added comprehensive unit tests to validate quantization accuracy and verify numerical stability across hardware platforms, including CUDA compatibility checks.

@coderabbitai

coderabbitai Bot commented Apr 7, 2026 •

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

The pull request updates FSQCodebook's quantization code packing to use integer-based computation instead of float-based power operations, replacing torch.pow with an explicit integer tensor construction and casting intermediate values to torch.int64. Additionally, a new test file is introduced with two unit tests to verify token ID packing behavior and CUDA float exponentiation characteristics.

Changes

Cohort / File(s) Summary
Quantization Logic Update
s3tokenizer/model_v2.py
Modified FSQCodebook.encode to replace float-based torch.pow(self.level, torch.arange(...)) with explicit torch.tensor([self.level**i ...], dtype=torch.int64), casting intermediate rounded values to torch.int64, and changing final packed index tensor cast from .int() to .to(torch.int32).
Test Coverage Addition
test/test_fsq_codebook.py
Added new test module with two unit tests: test_fsq_code_packing_returns_exact_token_ids validates packed token IDs against controlled input, and test_cuda_float_pow_can_be_non_integer documents CUDA float exponentiation precision characteristics. Includes FakeProjectDown helper module.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

The quantization logic change is precision-sensitive and requires careful verification of numerical correctness despite its small line count. The test additions are straightforward but the overall diff involves both core numerical logic and test infrastructure.

Poem

🐰 Integers dance where floats once swayed,
Precision carved in code, no fade!
With tests to guard each packed ID,
The quantizer's path runs deep and wide. ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Fix FSQ GPU code packing precision bug' directly and clearly describes the main change: fixing a floating-point precision issue in GPU-based FSQ codebook token packing.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@test/test_fsq_codebook.py`:
- Around line 13-16: The fake forward currently assumes x is 3D and unflattens
it, but FSQCodebook.project_down() receives a flattened 2D tensor
((batch*steps), dim); update the fake forward to accept x as 2D: read
batch_steps = x.shape[0] and dim = x.shape[1], assert self.output.shape[0] ==
batch_steps (and optionally self.output.shape[1] == dim) and return
self.output.to(x.device). Change the method referenced in the test (forward) so
its shape contract matches FSQCodebook.project_down().
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d004861f-2fab-4290-8fcd-e50d7e662e49

📥 Commits

Reviewing files that changed from the base of the PR and between 9bf5d84 and b723c06.

📒 Files selected for processing (2)
  • s3tokenizer/model_v2.py
  • test/test_fsq_codebook.py

Comment thread test/test_fsq_codebook.py
Comment on lines +13 to +16
def forward(self, x: torch.Tensor) -> torch.Tensor:
batch, steps = x.shape[:2]
assert self.output.shape[:2] == (batch, steps)
return self.output.to(x.device)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Make the fake match FSQCodebook.project_down()'s actual shape contract.

FSQCodebook.encode() flattens (batch, steps, dim) to ((batch * steps), dim) in preprocess() before it calls project_down(). With dummy_hidden.shape == (1, 2, 8), x.shape here is (2, 8), so this assertion fails immediately and the regression test never reaches the packing logic.

🧪 Proposed fix
 class FakeProjectDown(nn.Module):
@@
     def forward(self, x: torch.Tensor) -> torch.Tensor:
-        batch, steps = x.shape[:2]
-        assert self.output.shape[:2] == (batch, steps)
-        return self.output.to(x.device)
+        flat_output = self.output.reshape(-1, self.output.shape[-1])
+        assert x.shape == flat_output.shape
+        return flat_output.to(x.device)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
def forward(self, x: torch.Tensor) -> torch.Tensor:
batch, steps = x.shape[:2]
assert self.output.shape[:2] == (batch, steps)
return self.output.to(x.device)
def forward(self, x: torch.Tensor) -> torch.Tensor:
flat_output = self.output.reshape(-1, self.output.shape[-1])
assert x.shape == flat_output.shape
return flat_output.to(x.device)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/test_fsq_codebook.py` around lines 13 - 16, The fake forward currently
assumes x is 3D and unflattens it, but FSQCodebook.project_down() receives a
flattened 2D tensor ((batch*steps), dim); update the fake forward to accept x as
2D: read batch_steps = x.shape[0] and dim = x.shape[1], assert
self.output.shape[0] == batch_steps (and optionally self.output.shape[1] == dim)
and return self.output.to(x.device). Change the method referenced in the test
(forward) so its shape contract matches FSQCodebook.project_down().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

speech_tokenizer_v3_25hz tokens produce very different reconstruction vs CosyVoice tokens (same shape/length, very different codes)

1 participant