Qwen3.6-27B-MTP-GGUF

Qwen3.6-27B from Alibaba's Qwen team is a 27B-parameter dense multimodal causal language model with an integrated vision encoder, featuring 64 layers, 5120 hidden dimension, 248K vocabulary for 201+ languages, and a native 262K context window extensible to 1M+ tokens via YaRN. As the first dense model in Qwen3.6 to deliver flagship-level agentic coding performance, it outperforms the prior open-source flagship Qwen3.5-397B-A17B MoE across SWE-bench Verified (77.2 vs 76.2), Terminal-Bench (59.3 vs 52.5), and SkillsBench (48.2 vs 30.0), while achieving 87.8 GPQA Diamond and 66.1% BFCL-V4 for native tool calling. The model introduces hybrid thinking modes with "thinking preservation" to retain reasoning context across iterative development sessions, excels at frontend workflows and repository-level reasoning, and runs locally on 18GB VRAM via GGUF quantization with vLLM/SGLang/Unsloth support under Apache-2.0 licensing—eliminating MoE routing complexity for streamlined edge-to-server deployment.

Multi-Token Prediction (MTP) GGUF is a specialized GGUF model file format extension that integrates speculative decoding directly into the model weights to significantly accelerate local inference. Unlike traditional speculative decoding which requires a separate, smaller "draft" model, MTP GGUF files include additional output heads within the main model architecture that predict multiple future tokens in a single forward pass.

Model Files

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

👁 image.png

Base model