Question: How to enable offload_kqv?

[michael-shaked]

Hi @kherud ,

Thank you for a great library!!
I am wondering how can I enable offload_kqv?

Regards,
Michael

[bernardladenthin]

I noticed the repository hasn't seen recent activity, so I've taken the liberty of forking it and continuing development at: https://github.com/bernardladenthin/java-llama.cpp

See also #125

[bernardladenthin]

KQV offload is on by default upstream. The downstream fork at
https://github.com/bernardladenthin/java-llama.cpp exposes the inverse —
ModelFlag.NO_KV_OFFLOAD mapped to --no-kv-offload
(args/ModelFlag.java:50, used via ModelParameters.java:739). Leaving the
flag unset preserves offload_kqv=true.