Repository navigation
Fix FSQ GPU code packing precision bug - #52
zzuzxy0430 wants to merge 1 commit into
Conversation
📝 WalkthroughWalkthroughThe pull request updates FSQCodebook's quantization code packing to use integer-based computation instead of float-based power operations, replacing Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes The quantization logic change is precision-sensitive and requires careful verification of numerical correctness despite its small line count. The test additions are straightforward but the overall diff involves both core numerical logic and test infrastructure. Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@test/test_fsq_codebook.py`:
- Around line 13-16: The fake forward currently assumes x is 3D and unflattens
it, but FSQCodebook.project_down() receives a flattened 2D tensor
((batch*steps), dim); update the fake forward to accept x as 2D: read
batch_steps = x.shape[0] and dim = x.shape[1], assert self.output.shape[0] ==
batch_steps (and optionally self.output.shape[1] == dim) and return
self.output.to(x.device). Change the method referenced in the test (forward) so
its shape contract matches FSQCodebook.project_down().
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: d004861f-2fab-4290-8fcd-e50d7e662e49
📒 Files selected for processing (2)
s3tokenizer/model_v2.pytest/test_fsq_codebook.py
| def forward(self, x: torch.Tensor) -> torch.Tensor: | ||
| batch, steps = x.shape[:2] | ||
| assert self.output.shape[:2] == (batch, steps) | ||
| return self.output.to(x.device) |
There was a problem hiding this comment.
Make the fake match FSQCodebook.project_down()'s actual shape contract.
FSQCodebook.encode() flattens (batch, steps, dim) to ((batch * steps), dim) in preprocess() before it calls project_down(). With dummy_hidden.shape == (1, 2, 8), x.shape here is (2, 8), so this assertion fails immediately and the regression test never reaches the packing logic.
🧪 Proposed fix
class FakeProjectDown(nn.Module):
@@
def forward(self, x: torch.Tensor) -> torch.Tensor:
- batch, steps = x.shape[:2]
- assert self.output.shape[:2] == (batch, steps)
- return self.output.to(x.device)
+ flat_output = self.output.reshape(-1, self.output.shape[-1])
+ assert x.shape == flat_output.shape
+ return flat_output.to(x.device)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def forward(self, x: torch.Tensor) -> torch.Tensor: | |
| batch, steps = x.shape[:2] | |
| assert self.output.shape[:2] == (batch, steps) | |
| return self.output.to(x.device) | |
| def forward(self, x: torch.Tensor) -> torch.Tensor: | |
| flat_output = self.output.reshape(-1, self.output.shape[-1]) | |
| assert x.shape == flat_output.shape | |
| return flat_output.to(x.device) |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@test/test_fsq_codebook.py` around lines 13 - 16, The fake forward currently
assumes x is 3D and unflattens it, but FSQCodebook.project_down() receives a
flattened 2D tensor ((batch*steps), dim); update the fake forward to accept x as
2D: read batch_steps = x.shape[0] and dim = x.shape[1], assert
self.output.shape[0] == batch_steps (and optionally self.output.shape[1] == dim)
and return self.output.to(x.device). Change the method referenced in the test
(forward) so its shape contract matches FSQCodebook.project_down().
Summary
Fixes #49
Fix a CUDA-specific precision issue in
FSQCodebook.encode().On some CUDA stacks, floating-point
torch.pow(3, x)does not always returnexact integers. For example,
3**5can become242.99998, and the subsequent.int()cast truncates the packed token id by 1.This caused systematic token mismatches such as:
Root Cause
FSQCodebook.encode()used floating-point arithmetic for code packing:torch.pow(...).int()truncationOn affected GPUs this produced incorrect packed token ids.
Fix
int64self.levelint32Validation
torch.pow(3, x)precision issue locallySummary by CodeRabbit
Bug Fixes
Tests