Configuration Parsing Warning:In config.json: "quantization_config.bits" must be an integer
Configuration Parsing Warning:In config.json: "quantization_config.bits" must be greater than or equal to 2
Note
This was made to test extremely low quants. 70B is NOT usable at 1.6 bpw. A custom strategy for quanting was used, for details check EXL3 repo.
- Downloads last month
- 1
Safetensors
Model size
8B params
Tensor type
F16
·
I16 ·
Model tree for SicariusSicariiStuff/Negative_LLAMA_70B_EXL3_1.6bpw
Base model
meta-llama/Llama-3.1-70B Finetuned
meta-llama/Llama-3.3-70B-Instruct Finetuned
SicariusSicariiStuff/Negative_LLAMA_70B