Qwen3.5-35B-A3B-MTP-GGUF

Qwen3.5-35B-A3B from Alibaba's Qwen team is a sparse Mixture-of-Experts multimodal model with 35B total parameters but only 3B active per token, featuring an efficient hybrid architecture of Gated Delta Networks combined with 256 total experts (8 routed + 1 shared active) for high-throughput, low-latency inference at 262K native context length (extensible to 1M+ tokens) across text, image, and video modalities. It outperforms prior-generation models 6× its size (like Qwen3-235B-A22B) across reasoning, coding, agentic tool calling, and multimodal understanding benchmarks, achieving frontier-level performance on vision-language tasks, agentic workflows, and structured outputs while maintaining ~3B active parameter costs for deployment on 21GB VRAM consumer GPUs via GGUF quantization. With expanded 201-language support, toggleable "Enable Thinking" mode for step-by-step reasoning, and Apache 2.0 licensing for vLLM/SGLang/Transformers, it serves as a unified vision-language foundation for production coding agents, frontend workflows, documentation parsing, and multimodal chatbots at minimal compute overhead.

Multi-Token Prediction (MTP) GGUF is a specialized GGUF model file format extension that integrates speculative decoding directly into the model weights to significantly accelerate local inference. Unlike traditional speculative decoding which requires a separate, smaller "draft" model, MTP GGUF files include additional output heads within the main model architecture that predict multiple future tokens in a single forward pass.

Model Files

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):

👁 image.png

Base model