SauerkrautLM-ColLFM2-450M-v0.1

👁 VAGO Solutions Logo

🏆 #1 Small Model (<1B) | Best-in-Class Efficiency

SauerkrautLM-ColLFM2-450M-v0.1 is the #1 small model for visual document retrieval, achieving 83.56 NDCG@5 on ViDoRe v1 - beating colSmol-500M (82.49) with 10% fewer parameters!

👁 ViDoRe v1 Benchmark - 128-dim Models

🎯 Why Visual Document Retrieval?

Traditional OCR-based retrieval loses layout, tables, and visual context. Our visual approach:

• ✅ No OCR errors - Direct visual understanding
• ✅ Layout-aware - Understands tables, forms, charts
• ✅ End-to-end - Single model, no pipeline complexity

🏆 Key Achievements

Small Category Comparison (<1B, 128-dim)

#1 in ALL benchmarks for small models!

Detailed Benchmark Results

Efficiency Comparison

📋 Summary Tables

128-dim Models Comparison

👁 128-dim Models Summary

Comparison vs High-dim Models

👁 High-dim Comparison

✨ Key Features

• 🏆 Best Small Model: #1 in ALL benchmarks for <1B models
• ⚡ Ultra Efficient: Only 450M parameters, ~0.9GB VRAM
• 🎓 Curriculum Learning: Trained with progressive difficulty
• 🔀 Hierarchical Merge: Advanced model merging for optimal performance
• 📐 Native 512x512: Optimized for document resolution
• 🌍 Multilingual: 6 languages (EN, DE, FR, ES, IT, PT)

Model Details

🎓 Advanced Training Methodology

1. Curriculum Learning

Unlike standard training, ColLFM2 was trained with curriculum learning:

Stage 1: Easy examples (high-quality, clear documents)
 ↓
Stage 2: Medium examples (mixed quality)
 ↓
Stage 3: Hard examples (complex layouts, noisy scans)
 ↓
Stage 4: Full mixture with hard negatives

2. Hierarchical Model Merging

Base LFM2-VL-450M
 ↓
 ┌───┴───┐
 ↓ ↓
mMARCO Retrieval
Specialist Model
 ↓ ↓
 └───┬───┘
 ↓
 Hierarchical Merge
 ↓
 Final Model

• mMARCO Specialist: Sub-model trained on mMARCO for retrieval fundamentals
• Retrieval Model: Trained on document retrieval datasets
• Hierarchical Merge: Combined using learned merge weights

Hardware & Configuration

Datasets

Installation & Usage

⚠️ Important: Install our package first before loading the model:

pip install git+https://github.com/VAGOsolutions/sauerkrautlm-colpali

import torch
from PIL import Image
from sauerkrautlm_colpali.models import ColLFM2, ColLFM2Processor

model_name = "VAGOsolutions/SauerkrautLM-ColLFM2-450M-v0.1"

model = ColLFM2.from_pretrained(
 model_name,
 torch_dtype=torch.bfloat16,
 device_map="cuda:0",
).eval()

processor = ColLFM2Processor.from_pretrained(model_name)

images = [Image.open("document.png")]
queries = ["What is the main topic?"]

batch_images = processor.process_images(images).to(model.device)
batch_queries = processor.process_queries(queries).to(model.device)

with torch.no_grad():
 image_embeddings = model(**batch_images)
 query_embeddings = model(**batch_queries)

scores = processor.score(query_embeddings, image_embeddings)

Use Cases

✅ Perfect for:

• Edge deployment (Raspberry Pi, Jetson)
• Mobile applications
• High-throughput batch processing
• Cost-sensitive deployments
• Real-time retrieval systems

📊 Additional Benchmark Visualizations

MTEB v1+v2 Benchmark (128-dim Models)

👁 MTEB v1+v2 Benchmark - 128-dim Models

ViDoRe v3 Benchmark (128-dim Models)

👁 ViDoRe v3 Benchmark - 128-dim Models

Our Models vs High-dim Models

👁 ViDoRe v1 - Our Models vs High-dim

License

This model is licensed under the LFM 1.0 License from LiquidAI. Please review the full license before commercial use.

Citation

@misc{sauerkrautlm-colpali-2025,
 title={SauerkrautLM-ColPali: Multi-Vector Vision Retrieval Models},
 author={David Golchinfar},
 organization={VAGO Solutions},
 year={2025},
 url={https://github.com/VAGOsolutions/sauerkrautlm-colpali}
}

Contact

• VAGO Solutions: https://vago-solutions.ai
• GitHub: https://github.com/VAGOsolutions

Base model

LiquidAI/LFM2-VL-450M

(18)

7 items • Updated Dec 15, 2025 • 9