Frankenstein-RL-Frozen-Mid

Frankenstein-RL-Frozen-Mid is the reinforced (reinforcement training after cold-start initialization) with mid 1/3 layers frozen model from the paper:

What does RL improve for Visual Reasoning? A Frankenstein-Style Analysis

Xirui Li*, Ming Li*, Tianyi Zhou

University of Maryland  |  Mohamed bin Zayed University of Artificial Intelligence

(* Co-first Authors)

This model serves as the IN (Instruction-tuned) checkpoint before reinforcement learning, built on the OpenMMReasoner training recipe with Qwen2.5-VL-3B-Instruct as the base model.

Overview

Our paper introduces a Frankenstein-style analysis framework to understand what reinforcement learning (RL) actually improves in vision-language models (VLMs) for visual reasoning. Rather than relying on end-to-end benchmark scores, we decompose VLMs at the granularity of transformer layers and probe their functional roles through:

1. Functional Localization via Causal Probing — localizing vision- and reasoning-related computations along transformer depth
2. Update Characterization via Parameter Comparison — showing that IN and RL differ systematically in update magnitude and geometry
3. Transferability Test via Model Merging — transplanting RL-refined regions into IN models to test causal contributions

Key Findings

• RL does not uniformly improve visual perception or standalone reasoning
• RL induces structured refinements concentrated in mid-to-late layers, improving vision-to-reasoning alignment
• These mid-to-late refinements are both transferable (via merging) and necessary (via freezing) for RL gains
• Freezing late layers during RL training leads to a pronounced drop in reasoning performance

Evaluation Results

Fine-grained and Benchmark Metrics

Parameter Freezing Analysis (RL Training)

Quick Start

Installation

pip install transformers accelerate
pip install qwen-vl-utils[decord]==0.0.8

Inference

from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
 "AIcell/Frankenstein-IN",
 torch_dtype="auto",
 device_map="auto",
)

processor = AutoProcessor.from_pretrained("AIcell/Frankenstein-IN")

messages = [
 {
 "role": "user",
 "content": [
 {"type": "image", "image": "https://your-image-url.jpg"},
 {"type": "text", "text": "Please solve this math problem step by step."},
 ],
 }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
 text=[text],
 images=image_inputs,
 videos=video_inputs,
 padding=True,
 return_tensors="pt",
).to(model.device)

generated_ids = model.generate(**inputs, max_new_tokens=2048)
generated_ids_trimmed = [
 out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
 generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text[0])

Related Resources

Citation

@article{li2026frankenstein,
 title={What does RL improve for Visual Reasoning? A Frankenstein-Style Analysis},
 author={Li, Xirui and Li, Ming and Zhou, Tianyi},
 journal={arXiv preprint arXiv:2602.12395},
 year={2026}
}

License

This model is released under the Apache 2.0 License.

Base model

Qwen/Qwen2.5-VL-3B-Instruct

(793)

The collection for the paper: What does RL improve for Visual Reasoning? A Frankenstein-Style Analysis • 6 items • Updated Feb 17