Uncensored community-governed opinion-rich human-like LLMs. • 6 items • Updated • 142
Dobby-Llama-3.3-70B_GGUF
Dobby-70B high-performance GGUF model based on Llama 3.3 with 70 billion parameters. Designed for efficiency, this model supports quantization levels in 4-bit, 6-bit, and 8-bit, offering flexibility to run on various hardware configurations without compromising performance.
Compatibility
This model is compatible with:
- LMStudio: An easy-to-use desktop application for running and fine-tuning large language models locally.
- Ollama: A versatile tool for deploying, managing, and interacting with large language models seamlessly.
Quantization Levels
| Quantization | Description | Use Case |
|---|---|---|
| 4-bit | Highly compressed for minimal memory usage. Some loss in precision and quality, but great for lightweight devices with limited VRAM. | Ideal for testing, quick prototyping, or running on low-end GPUs and CPUs. |
| 6-bit | Strikes a balance between compression and quality. Offers improved accuracy over 4-bit without requiring significant additional resources. | Recommended for users with mid-range hardware aiming for a compromise between speed and precision. |
| 8-bit | Full-precision quantization for maximum quality while still optimizing memory usage compared to full FP16 or FP32 models. | Perfect for high-performance systems where maintaining accuracy and precision is critical. |
Recommended Usage
Choose your quantization level based on the hardware you are using:
- 4-bit for ultra-lightweight systems.
- 6-bit for balance on mid-tier hardware.
- 8-bit for maximum performance on powerful GPUs.
This model supports prompt fine-tuning for domain-specific tasks, making it an excellent choice for interactive applications like chatbots, question answering, and creative writing.
- Downloads last month
- 18
GGUF
Model size
71B params
Architecture
llama
Hardware compatibility
Log In to add your hardware
4-bit
6-bit
8-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for SentientAGI/Dobby-Unhinged-Llama-3.3-70B_GGUF
Base model
meta-llama/Llama-3.1-70B Finetuned
meta-llama/Llama-3.3-70B-Instruct