You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Model Details

Model Description

This repository provides Asagi-14B, a large-scale Japanese Vision & Language Model (VLM). Asagi-14B has been trained on an extensive Japanese dataset, incorporating a diverse range of data sources.

A significant portion of the training data is synthesized using models such as the Japanese large language model (CALM3-22B-Chat) and the English Vision & Language Model (Phi3.5-vision-instruct).

Importantly, we do not use LLMs that restrict the usage of their outputs in the license terms (e.g., GPT-4) to synthesize the training data.

Usage

Requirements

transformers==4.45.1
accelerate==0.34.2
torch==2.4.0
torchvision==0.19.0

How to use

import requests
import torch
import transformers
from PIL import Image
from transformers import AutoModel, AutoProcessor, GenerationConfig

transformers.set_seed(42)
model_path = "MIL-UT/Asagi-14B"
processor = AutoProcessor.from_pretrained(model_path)
model = AutoModel.from_pretrained(
 model_path, trust_remote_code=True,
 torch_dtype=torch.bfloat16,
 device_map="auto"
)

generation_config = GenerationConfig(
 do_sample=True,
 num_beams=5,
 max_new_tokens=256,
 temperature=0.7,
 repetition_penalty=1.5
)

prompt = ("以下は、タスクを説明する指示です。要求を適切に満たす応答を書きなさい。\n\n"
 "### 指示:\n<image>\nこの画像を見て、次の質問に詳細かつ具体的に答えてください。この写真はどこで撮影されたものか教えてください。また、画像の内容についても詳しく説明してください。\n\n### 応答:\n")

# sample image
sample_image_url = "https://raw.githubusercontent.com/uehara-mech/uehara-mech.github.io/refs/heads/master/images/shibuya.jpg"
image = Image.open(requests.get(sample_image_url, stream=True).raw)

inputs = processor(
 text=prompt, images=image, return_tensors="pt"
)
inputs_text = processor.tokenizer(prompt, return_tensors="pt")
inputs['input_ids'] = inputs_text['input_ids']
inputs['attention_mask'] = inputs_text['attention_mask']
for k, v in inputs.items():
 if v.dtype == torch.float32:
 inputs[k] = v.to(model.dtype)
inputs = {k: inputs[k].to(model.device) for k in inputs if k != "token_type_ids"}

generate_ids = model.generate(
 **inputs,
 generation_config=generation_config
)
generated_text = processor.batch_decode(
 generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False
)[0]

# do not print the prompt
if "<image>" in prompt:
 prompt = prompt.replace("<image>", " ")
generated_text = generated_text.replace(prompt, "")

print(f"Generated text: {generated_text}")

# >>> Generated text: この写真は渋谷スクランブル交差点の夜景を撮影したものです。渋谷スクランブル交差点は、東京都渋谷区に位置する有名な交差点で、多くの人々が行き交う場所として知られています。写真には、高層ビル群やネオンサイン、街灯などが写っており、都会の活気とエネルギーを感じることができます。

Example

👁 Image

More Examples

Training Details

Training Data

Note: ROIS (Ours) is a newly collected dataset crawled from the web specifically for this project. The dataset consists of image and raw text pairs, which are used to synthesize the training data.

Evaluation

We evaluated our model using Heron-Bench, JA-VLM-Bench-in-the-Wild, and JA-VG-VQA-500. We used eval-mm library for this evaluation.

Here, models with "†" are not trained with GPT-generated data. Bold numbers indicate the best performance among all models, and underlined numbers indicate the best performance among models not trained with GPT-generated data.

Risks and Limitations

The models released here are in the early stages of our research and development and have not been tuned to ensure outputs align with human intent and safety considerations.

Model Card Authors

Kohei Uehara

Base model

llm-jp/llm-jp-3-13b-instruct

(7)

Asagi is a Japanese Vision & Language model, trained on a large-scale synthetic dataset. • 4 items • Updated Nov 27, 2025 • 7