gemma-4-E4B-it-qat-GGUF

gemma-4-E4B-it-qat-q4_0-unquantized is a mid-tier on-device-optimized instruction-tuned multimodal model from Google DeepMind, part of the Gemma 4 family, featuring 4.5 billion effective parameters (8B total including embeddings) optimized via Quantization-Aware Training (QAT) to retain near-bfloat16 quality while substantially lowering memory requirements. Designed for efficient local execution on laptops and capable mobile devices, it leverages Per-Layer Embeddings (PLE) for parameter efficiency and supports text, image, and audio modalities with a 128K token context window and 262K vocabulary across 140+ languages, using a hybrid attention mechanism with 512-token sliding window attention across 42 layers alongside a ~150M parameter vision encoder and ~300M parameter audio encoder. Capabilities include ASR, speech translation (CoVoST score of 35.54), image understanding, OCR, video frame analysis, native function calling, and configurable thinking/reasoning mode, with the E4B stepping up noticeably from the E2B in key benchmarks — scoring 69.4% on MMLU Pro, 58.6% on GPQA Diamond, 42.5% on AIME 2026, 52.0% on LiveCodeBench v6, 52.6% on MMMU Pro (vision), and 25.4% on the 128K long-context MRCR v2 task — making it a strong balance between the lightweight E2B and the larger server-class models, with the Q4_0 unquantized variant providing half-precision weights ideal for custom downstream compilation and research.

Google DeepMind’s Gemma 4 Quantization-Aware Training (QAT) releases compress models by simulating lower precision during the training process itself. This drastically reduces VRAM requirements and accelerates local inference on consumer hardware and mobile devices while preserving the near-original quality of uncompressed baselines.

Model Files

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp

Base model