✂️ Interlace-Qwen3-VL-4B-20pc

👁 Paper
👁 Project Page
👁 GitHub
👁 Collection
👁 CVPR 2026

This model was produced by INTERLACE, a layer-pruning framework for Vision-Language Models. 20% of the transformer layers in Qwen/Qwen3-VL-4B-Instruct were removed using triplet-based similarity analysis, and the remaining model was fine-tuned on 1% of FineVision for a single epoch.

88.0% average relative performance retained  |  20% layers dropped (7 of 36)  |  29 layers remaining

📋 Model Details

🚀 Usage

from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained(
 "pmadinei/Interlace-Qwen3-VL-4B-20pc",
 dtype="auto",
 device_map="auto",
 attn_implementation="flash_attention_2",
)
processor = AutoProcessor.from_pretrained("Qwen/Qwen3-VL-4B-Instruct")

messages = [
 {
 "role": "user",
 "content": [
 {"type": "image", "image": "path/to/image.jpg"},
 {"type": "text", "text": "Describe this image in detail."},
 ],
 }
]

inputs = processor.apply_chat_template(
 messages, tokenize=True, add_generation_prompt=True,
 return_dict=True, return_tensors="pt",
).to(model.device)

output = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(output[0], skip_special_tokens=True))

📊 Performance

Relative performance compared to the unpruned baseline (% of baseline score, Chain-of-Thought enabled):

🤗 All INTERLACE Models

📝 Citation

@inproceedings{madinei2026interlace,
 title={Interlace: Interleaved layer pruning and efficient adaptation in large vision-language models},
 author={Madinei, Parsa and Solgi, Ryan and Wen, Ziqi and Skaza, Jonathan and Eckstein, Miguel and Pedarsani, Ramtin},
 booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
 pages={2947--2956},
 year={2026}
}

Base model

Qwen/Qwen3-VL-4B-Instruct

(307)

INTERLACE: Interleaved Layer Pruning in VLMs (CVPR 2025). Pruned Qwen3-VL models retaining up to 94% performance. • 8 items • Updated Apr 14