Training and evaluation datasets collected for Solaris: Building a Multiplayer Video World Model in Minecraft • 2 items • Updated • 4
Dataset Viewer
Solaris Training Dataset
Project Page | Paper | Github
The training dataset collected via SolarisEngine to train the Solaris multiplayer world model for Minecraft.
Dataset Info
The dataset contains 12.64 M frames across two players. It was recorded at 20 FPS. The observation (video) dimensions are: 1280 × 720.
The action space is presented below:
| Action key | Type | Description |
|---|---|---|
| forward | bool/sustained | Player moving forward (W). |
| back | bool/sustained | Player moving backward (S). |
| left | bool/sustained | Player strafing left (A). |
| right | bool/sustained | Player strafing right (D). |
| jump | bool/sustained | Player jumping. |
| sprint | bool/sustained | Player sprinting. |
| sneak | bool/sustained | Player sneaking. |
| camera | vec2/sustained | Change in player camera orientation (yaw, pitch). |
| attack | bool/once | Player attacks. |
| use | bool/once | Player uses / interacts with the environment. |
| mount | bool/once | Player mounts an entity/vehicle. |
| dismount | bool/once | Player dismounts. |
| place_block | bool/once | Player places a block using the currently selected item. |
| place_entity | bool/once | Player places an entity item. |
| mine | bool/sustained | Player mining a block. |
| hotbar.1 | bool/once | Player selects hotbar slot 1. |
| hotbar.2 | bool/once | Player selects hotbar slot 2. |
| hotbar.3 | bool/once | Player selects hotbar slot 3. |
| hotbar.4 | bool/once | Player selects hotbar slot 4. |
| hotbar.5 | bool/once | Player selects hotbar slot 5. |
| hotbar.6 | bool/once | Player selects hotbar slot 6. |
| hotbar.7 | bool/once | Player selects hotbar slot 7. |
| hotbar.8 | bool/once | Player selects hotbar slot 8. |
| hotbar.9 | bool/once | Player selects hotbar slot 9. |
Usage
The multiplayer Duet dataset is stored in a sharded form on Hugging Face. You can download and unshard it using the following steps:
1. Download the shards
hf download nyu-visionx/solaris-training-dataset --repo-type dataset --local-dir YOUR_STORAGE_PATH/datasets
2. Unshard the dataset
Run the unshard_dataset.py script provided in the Solaris repository:
python unshard_dataset.py --shards YOUR_STORAGE_PATH/datasets/duet_sharded --out YOUR_STORAGE_PATH/datasets/duet
Citation
@article{savva2025solaris,
title={Solaris: Building a Multiplayer Video World Model in Minecraft},
author={Georgy Savva and Oscar Michel and Daohan Lu and Suppakit Waiwitlikhit and Timothy Meehan and Dhairya Mishra and Srivats Poddar and Jack Lu and Saining Xie},
journal={arXiv preprint arXiv:2602.22208},
year={2025}
}
- Downloads last month
- 1,186