You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
bucky/model: WithFlashAttn(false) is ignored; flash attention always on #1041
bucky/model: WithFlashAttn(false) has no effect. Flash attention stays on, while the doc comment says it defaults to false and the startup log reports flash-attn=false.
Where
kronk v1.32.8 (also current main), sdk/bucky/model/model.go:59-67:
cp starts from whisper.cpp's defaults, and in whisper.cpp v1.9.5 those set flash_attn = true (src/whisper.cpp:3716, in whisper_context_default_params()).
The FlashAttn branch only ever writes 1. Unlike UseGPU, it has no else branch, so false never reaches whisper.cpp.
sdk/bucky/model/config.go:33-34: the Config.FlashAttn doc says "Defaults to false.", which is wrong in practice.
model.go:101 logs "flash-attn", cfg.FlashAttn, which reports false while the kernel is active.
Repro
Load any whisper model with model.WithFlashAttn(false) (or with the option unset), using either WithUseGPU(true) or WithUseGPU(false). whisper.cpp logs:
whisper_init_with_params_no_state: flash attn = 1
Expected
WithFlashAttn(false) turns flash attention off, and the doc and log match what whisper.cpp actually uses.
Activity
ardan-bkennedy commentedon Oct 9, 2026
Fixed with PR #1043