VOOZH about

URL: https://huggingface.co/OpenYourMind/Minimax-M3-abliterated-clean/discussions/2

⇱ OpenYourMind/Minimax-M3-abliterated-clean · Please, make GGUF Q2 version

Please, make GGUF Q2 version

#2
by denisvasi - opened

I have only 4 A100 40Gb. Now i use unsloth version

Thanks for the interest, currently i am not making any quantizations of models as there usualy is someone doing them. Usualy a GGUF Q2 version should be doable on CPU / RAM yourself easily.

https://huggingface.co/SicariusSicariiStuff/Minimax-M3-abliterated_GGUF

Notice: this is an experimental version, haven't tested it, please let me know if there are any issues
Q3 is up, soon Q2

Thanks man, much appreciate you taking the time to do GGUF's

@SicariusSicariiStuff Can you provide smaller model like IQ2_M, because Q2_K too big for my server (160GB). It will be great

· Sign up or log in to comment