Qwen3.6-35B-A3B-MTP-GGUF

Qwen3.6-35B-A3B from Alibaba's Qwen team is an open-weight sparse Mixture-of-Experts (MoE) multimodal model with 35B total parameters but only 3B active per token, combining Gated DeltaNet linear attention with standard gated attention layers for efficient inference at a fraction of compute cost while supporting 262K native context (extensible to 1M via YaRN) across text, image, and video inputs. Released under Apache 2.0, it delivers flagship agentic coding performance at 73.4% SWE-Bench Verified, excels at frontend workflows and repository-level reasoning, and introduces hybrid thinking modes with "thinking preservation" to retain reasoning context across multi-turn conversations for streamlined iterative development. The model features native tool calling, structured output, vision input, and integrated function calling, running locally on ~21-24GB VRAM via GGUF quantization with vLLM/Ollama/LM Studio support while maintaining 38.6B weekly tokens processed on OpenRouter, making it ideal for production coding agents and multimodal workflows at minimal cost.

Multi-Token Prediction (MTP) GGUF is a specialized GGUF model file format extension that integrates speculative decoding directly into the model weights to significantly accelerate local inference. Unlike traditional speculative decoding which requires a separate, smaller "draft" model, MTP GGUF files include additional output heads within the main model architecture that predict multiple future tokens in a single forward pass.

Model Files

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

👁 image.png

Base model