gemma-4-12B-it-qat-GGUF

gemma-4-12B-it-qat-q4_0-unquantized is a 12-billion-parameter instruction-tuned vision-language model from Google DeepMind, part of the Gemma 4 family, optimized using Quantization-Aware Training (QAT) to preserve bfloat16-level quality while significantly reducing memory requirements. It features a unified encoder-free architecture that projects raw image patches and audio waveforms directly into the LLM's embedding space, supports text, image, and audio modalities, and offers a 256K token context window. The model employs a hybrid attention mechanism interleaving local sliding window and full global attention, with multilingual support across 140+ languages, native function calling, configurable thinking/reasoning mode, and achieves strong benchmark scores including 77.2% on MMLU Pro, 78.8% on GPQA Diamond, and 77.5% on AIME 2026. The Q4_0 unquantized variant specifically refers to half-precision weights extracted from the QAT pipeline, making it ideal for custom downstream compilation and research rather than direct deployment.

Google DeepMind’s Gemma 4 Quantization-Aware Training (QAT) releases compress models by simulating lower precision during the training process itself. This drastically reduces VRAM requirements and accelerates local inference on consumer hardware and mobile devices while preserving the near-original quality of uncompressed baselines.

Model Files

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp

Base model