---

🧠 Overview

Clarivis-R1 is a vision-language model fine-tuned specifically for complex chart reasoning. Given a chart image and an analytical question, the model produces a fully transparent response with an explicit chain-of-thought trace followed by a concise final answer — structured as:

<think> ... step-by-step reasoning ... </think>
<answer> ... final answer ... </answer>

Built on google/gemma-4-E2B-it and trained on 300,000 chart reasoning samples from IBM Granite's ChartNet dataset, Clarivis-R1 targets use cases where interpretability matters: document intelligence, chart QA, RAG pipelines, and agentic workflows.

---

📈 Benchmark Results

Evaluated using greedy decoding (do_sample=False, max_new_tokens=1024). Relaxed accuracy allows ±5% tolerance for numerical answers — the standard ChartQA evaluation protocol.

---

💡 Example Output

Chart: Bar chart — Netflix content ratings by type (Movies vs TV Shows)

Question:

"Which rating category has the most balanced distribution between Movies and TV Shows?"

Response:

<think>
The question asks for the rating category where the number of Movies and
TV Shows are most balanced — i.e., where counts are closest to each other.

From the chart:
- TV-MA: Movies ≈ 2050, TV Shows ≈ 1150 → difference ≈ 900
- TV-14: Movies ≈ 1430, TV Shows ≈ 700 → difference ≈ 730
- TV-PG: Movies ≈ 540, TV Shows ≈ 320 → difference ≈ 220
- TV-Y7: Movies ≈ 150, TV Shows ≈ 200 → difference ≈ 50 ✓
- TVY: Movies ≈ 130, TV Shows ≈ 170 → difference ≈ 40

The smallest absolute difference belongs to TVY and TV-Y7.
Considering both absolute difference and visual proximity, TV-Y7
is the most balanced category.
</think>

<answer>
TV-Y7
</answer>

---

🏋️ Training Details

Base Model

google/gemma-4-E2B-it — 2B parameters, 4-bit quantized via Unsloth

Dataset

LoRA Configuration

r = 32
lora_alpha = 32
lora_dropout = 0.05
target_modules = "all-linear"
bias = "none"

# All layer types fine-tuned:
finetune_vision_layers = True
finetune_language_layers = True
finetune_attention_modules = True
finetune_mlp_modules = True

Training Configuration

per_device_train_batch_size = 1
gradient_accumulation_steps = 4 # effective batch size = 4
learning_rate = 2e-5
lr_scheduler_type = "cosine"
warmup_ratio = 0.03
num_train_epochs = 1
max_seq_length = 2048
optim = "adamw_torch"
max_grad_norm = 0.5
weight_decay = 0.001
dataloader_num_workers = 0 # required for PIL image safety

Infrastructure

Loss Curve

---

🚀 Quick Start

Installation

pip install unsloth transformers torch pillow

Inference

from PIL import Image
from transformers import TextStreamer

# ── Input ────────────────────────────────────────────────────────────────────
from unsloth import FastVisionModel
from PIL import Image
from transformers import TextStreamer

# ── Load model ───────────────────────────────────────────────────────────────
model, processor = FastVisionModel.from_pretrained(
 model_name="mahmoudalrefaey/clarivis-r1",
 load_in_4bit=True,
)
FastVisionModel.for_inference(model)

# ── Input ────────────────────────────────────────────────────────────────────
image = Image.open("your_chart.png").convert("RGB")

instruction = "Your question here."

messages = [
 {
 "role": "user",
 "content": [{"type": "image"}, {"type": "text", "text": instruction}],
 }
]

# ── Tokenize ─────────────────────────────────────────────────────────────────
input_text = processor.apply_chat_template(messages, add_generation_prompt=True)

inputs = processor(
 image,
 input_text,
 add_special_tokens=False,
 return_tensors="pt",
).to("cuda")

# ── Generate ─────────────────────────────────────────────────────────────────
text_streamer = TextStreamer(processor, skip_prompt=True)

result = model.generate(
 **inputs,
 streamer=text_streamer,
 max_new_tokens=2048,
 use_cache=True,
 do_sample=False,
)

⚠️ Important: Always use do_sample=False for chart reasoning tasks. Do not pass temperature=0.0 — it raises a ValueError. Use do_sample=False for greedy (deterministic) decoding instead.

---

✅ Intended Use

---

⚠️ Limitations

• Stacked bar decomposition: Precisely isolating individual colored segments in stacked charts is challenging at the 2B parameter scale — total bar heights are read more reliably than individual segments
• Small value precision: Values in the single-digit to ~20 range on crowded axes may be approximated or hallucinated
• Single epoch training: Each sample was seen exactly once during training
• Out-of-domain charts: Performance may degrade on chart styles not well-represented in ChartNet (e.g. Sankey diagrams, radar charts)

---

📄 Citation

If you use Clarivis-R1 in your research or build on it, please cite:

@misc{clarivis-r1-2025,
 author = {Mahmoud Alrefaey},
 title = {Clarivis-R1: Chart Reasoning Vision-Language Model},
 year = {2025},
 publisher = {HuggingFace},
 howpublished = {\url{https://huggingface.co/mahmoudalrefaey/clarivis-r1}},
}

---

📜 License

This model is built on Gemma 4 and is subject to the Gemma Terms of Use. Please review the license terms before any commercial use.

---

🙏 Acknowledgements

---

Base model

google/gemma-4-E2B

google/gemma-4-E2B-it

(106)