Qwen3.5-4B MTPLX Optimized Speed (Q4 trunk)

Run this with MTPLX

MTPLX is an MLX-native runtime for native Multi-Token-Prediction speculative decoding on Apple Silicon. Up to 2.24× faster decode at real coding temperatures (temp=0.6 / top_p=0.95 / top_k=20) using the model's own built-in MTP heads — no external drafter, no greedy hack.

brew install youssofal/mtplx/mtplx
mtplx start

Project: github.com/youssofal/MTPLX

Other MTPLX checkpoints:

Qwen3.6-27B-MTPLX-Optimized-Speed — 4-bit flagship speed (63 TPS on M5 Max)
Qwen3.5-4B-Optimized-MTPLX — small 8-bit

Small speed-test artifact for MTPLX on Apple Silicon.

This model uses the public mlx-community/Qwen3.5-4B-MLX-4bit MLX affine 4-bit trunk and grafts back the official native MTP head from Qwen/Qwen3.5-4B. The MTP head is stored as mtp/weights.safetensors; layer-0 attention/MLP linears are quantized to 4-bit affine group-64, while mtp.fc and the MTP norms stay BF16.

Intended Use

A quick MTPLX download / load / speed-path test artifact at 4B scale. Once the runtime ships:

mtplx start

Choose Custom Hugging Face repo, then enter:

Youssofal/Qwen3.5-4B-MTPLX-Optimized-Speed

Artifact Layout

Trunk: MLX affine 4-bit, group size 64
MTP sidecar: official Qwen3.5-4B MTP tensors
MTP sidecar quantization: body-int4
Runtime contract: mtplx_runtime.json
MTPLX default: depth 2, target temperature 0.6, draft temperature 0.6

Local Smoke Result

On the local Apple Silicon MTPLX workstation, the depth-2 speed path measured 120.06 tok/s versus 108.41 tok/s AR on the warm-code prompt (max_tokens=48, temperature=0.6, top_p=0.95, top_k=20). Depth 3 is intentionally not the default for this 4B artifact because it over-drafts the small native-MTP head.

Build Stats

{
 "bits": 4,
 "group_size": 64,
 "mode": "affine",
 "output_size_bytes": 86701040,
 "output_tensor_count": 29,
 "quantization": "body-int4",
 "quantized_linears": {
 "mtp.layers.0.mlp.down_proj": {"bits": 4, "group_size": 64, "mode": "affine"},
 "mtp.layers.0.mlp.gate_proj": {"bits": 4, "group_size": 64, "mode": "affine"},
 "mtp.layers.0.mlp.up_proj": {"bits": 4, "group_size": 64, "mode": "affine"},
 "mtp.layers.0.self_attn.k_proj":{"bits": 4, "group_size": 64, "mode": "affine"},
 "mtp.layers.0.self_attn.o_proj":{"bits": 4, "group_size": 64, "mode": "affine"},
 "mtp.layers.0.self_attn.q_proj":{"bits": 4, "group_size": 64, "mode": "affine"},
 "mtp.layers.0.self_attn.v_proj":{"bits": 4, "group_size": 64, "mode": "affine"}
 },
 "source_tensor_count": 15
}

URL: https://huggingface.co/Youssofal/Qwen3.5-4B-MTPLX-Optimized-Speed