👁 MapTrace Task Illustration

Qwen3.5-0.8B — MapTrace Path Planning (Partial Fine-Tune)

This model is a Partial Fine-Tune of Qwen/Qwen3.5-0.8B specifically trained for visual path planning on indoor and outdoor maps. Given an image of a traversable map with marked start (green) and end (red) locations, the model predicts a sequence of normalized (x, y) waypoints that connect the two locations while staying on traversable paths.

---

🗺️ Task Description

MapTrace is a visual navigation benchmark that requires a model to:

1. Understand a map image with a marked start location (green dot) and end location (red dot).
2. Trace a realistic, traversable path between the two points.
3. Output the path as a list of normalized (x, y) coordinates: [(x1, y1), (x2, y2), ...]

Example Input

You are provided an image of a path with a start location denoted in green 
and an end location denoted in red. 
The normalized xy-coordinates of the start location are (0.77, 0.47) and 
of the end location (0.54, 0.54). 
Output a list of normalized coordinates in the form of a list [(x1,y1), (x2,y2)...] 
of the path between the start and end location. 
Ensure that the path follows the traversable locations of the map.

Example Output

[(0.7695, 0.4727), (0.7344, 0.5156), (0.5898, 0.5234), (0.543, 0.543)]

---

🏋️ Training Strategy

The model was trained using a Partial Fine-Tuning approach to balance quality and computational efficiency:

Why Partial Fine-Tuning?

Instead of full fine-tuning or LoRA, we opted for unlocking the last 6 transformer layers and the visual encoder for gradient updates. This approach:

• Preserves the model's general language understanding from the foundation weights.
• Directly adapts the "reasoning" layers to the spatial coordinate prediction task.
• Avoids LoRA's approximation error — weights are modified directly.

Training Highlights

• Resumed from checkpoint-50 → checkpoint-100 → final across multiple sessions.
• VRAM stability was achieved by setting max_pixels=1024x768 as a safety cap, preventing OOM spikes from high-resolution map images.
• Loss masking was applied so the model only learns from the assistant's coordinate response, not the user's prompt or image tokens.
• Final Token Accuracy: ~~84.5%, Final Loss: ~0.37.

---

🚀 Inference

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from PIL import Image

model_id = "TurkishCodeMan/Qwen3.5-0.8B-MapTrace-PartialFT"

processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
 model_id,
 torch_dtype=torch.bfloat16,
 device_map="auto",
 trust_remote_code=True,
)
model.eval()

def predict_path(image_path: str, start_xy: tuple, end_xy: tuple) -> str:
 image = Image.open(image_path).convert("RGB")
 
 prompt_text = (
 f"You are provided an image of a path with a start location denoted in green "
 f"and an end location denoted in red. \n"
 f"The normalized xy-coordinates of the start location are ({start_xy[0]}, {start_xy[1]}) "
 f"and of the end location ({end_xy[0]}, {end_xy[1]}). \n"
 f"Output a list of normalized coordinates in the form of a list [(x1,y1), (x2,y2)...] "
 f"of the path between the start and end location. \n"
 f"Ensure that the path follows the traversable locations of the map."
 )

 messages = [
 {
 "role": "user",
 "content": [
 {"type": "image"},
 {"type": "text", "text": prompt_text},
 ],
 }
 ]

 text = processor.apply_chat_template(
 messages,
 tokenize=False,
 add_generation_prompt=True,
 )

 inputs = processor(
 text=[text],
 images=[image],
 return_tensors="pt",
 padding=True,
 min_pixels=256 * 28 * 28,
 max_pixels=1024 * 768,
 ).to(model.device)

 with torch.no_grad():
 generated_ids = model.generate(
 **inputs,
 max_new_tokens=512,
 do_sample=False,
 temperature=0.0,
 )
 
 output = processor.decode(
 generated_ids[0][inputs.input_ids.shape[1]:],
 skip_special_tokens=True
 )
 return output.strip()


# Example usage
result = predict_path(
 image_path="map.png",
 start_xy=(0.7695, 0.4727),
 end_xy=(0.543, 0.543),
)
print(result)
# Expected: [(0.7695, 0.4727), (0.7344, 0.5156), (0.5898, 0.5234), (0.543, 0.543)]

Important: Use the raw prompt format shown above (not apply_chat_template with add_generation_prompt=True). Since the training data did not include <think> tokens, applying the default Qwen3.5 template would inject thinking tags and cause degraded output.

---

📊 Training Metrics

---

🗂️ Dataset

The model was trained on google/MapTrace (the maptrace_20k subset), a dataset containing 20,000 map images paired with path annotations from start to end locations.

---

⚠️ Limitations

• Map-Specific: The model is trained exclusively for top-down map images; it is not a general path planner.
• Coordinate Precision: Predictions are in normalized [0, 1] space and may have small pixel-level deviations.
• Resolution Cap: Images larger than 1024×768 will be downscaled to this resolution during inference to match training conditions.

---

📜 License

This model is released under the Apache 2.0 license, consistent with the base model.

Base model

Qwen/Qwen3.5-0.8B-Base

Qwen/Qwen3.5-0.8B

(232)