Official quants?
#2
by
joshuaturner
- opened
I'd love to see the tooling in the repo for "official" quants to be released. My preferred flavour is GGUF, purely for convenience.
active work is happening on it.
https://github.com/ggerganov/llama.cpp/issues/7116
I'm running this model with gguf through ollama now. Thought I should point this out.