Qwen3.5-27B-MTP-GGUF

Qwen3.5-27B from Alibaba's Qwen team is a 27B-parameter dense (non-MoE) multimodal transformer model—the only full-weight, non-MoE model in the Qwen3.5 Medium Series—featuring 64 layers, 5120 hidden dimension, 248K vocabulary for 201+ languages, native multimodal input (text + images + video via early fusion), and a massive 256K native context window extensible to 1M+ tokens via YaRN. Released February 24, 2026 under Apache 2.0, it achieves 72.4% on SWE-bench Verified (matching GPT-5 mini), 87.8% GPQA Diamond, 66.1% BFCL-V4 for native tool calling, and excels at agentic coding, frontend development, repository-level code comprehension, and complex reasoning while running on 22GB VRAM (Mac M-series, single RTX 4090) with simple deployment free of MoE routing overhead. Its strengths include native multimodal chat, long-context document processing, structured JSON outputs, fine-tuning for specialized domains, and Ollama/vLLM/llama.cpp support, making it ideal for production multimodal assistants combining image understanding with grounded text reasoning at efficient scale.

Multi-Token Prediction (MTP) GGUF is a specialized GGUF model file format extension that integrates speculative decoding directly into the model weights to significantly accelerate local inference. Unlike traditional speculative decoding which requires a separate, smaller "draft" model, MTP GGUF files include additional output heads within the main model architecture that predict multiple future tokens in a single forward pass.

Model Files

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

👁 image.png

Base model