Replies: 1 comment
-
|
SWA is disabled by default, to enable it you use the flag |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I usually run kcpp linux64 and leave all on defaults, changing only context size and now after TurboQuant tend to set KV at Q4. In the launcher I see AutoFit and SWA checkboxes are empty by default in v1.111.2 (running kcpp without arguments from command line in bash). In model load log (terminal output) I see for e.g. Gemma 4 E4B
creating SWA KV cache, size =So I don't understand, is SWA enabled when there is such line in log or not? What does above line mean?
I've asked
Gemma 4 E4B Q4_K_Mabout lines from the log - it tells it means rebuilding KV cache for SWA (it told it's Structured / Sparse Weight Averaging).For AutoFit https://github.com/LostRuins/koboldcpp/wiki
Is 'Force AutoFit' checkbox in the launcher same as
--autofit? If yes, why is it not ON by default when I start the launcher? The launcher shows 'GPU Layers' box = -1 by default.TIA
Beta Was this translation helpful? Give feedback.
All reactions