Offical checkpoint for Tool-Augmented Reward Modeling (ICLR 2024 spotlight).

Model Description

Themis is a tool-augmented preference model to address these limitations by empowering RMs with access to external environments, including calculators and search engines. It was introduced in the ICLR 2024 paper and first released in this repository. Themis-7b is trained with TARA, achieving a noteworthy overall improvement of 17.7% across eight tasks in preference ranking.

🔥 News

• 9 February, 2024: 🎉 We release the official codebase and model weights of ernie-research/Themis-7b. Stay tuned!🔥
• 16 January, 2024: 🎉 Our work has been accepted to ICLR 2024 spotlight! ✨

Citation

@inproceedings{tarm-2024-ernie,
 author = {Lei Li and
 Yekun Chai and
 Shuohuan Wang and
 Yu Sun and
 Hao Tian and
 Ningyu Zhang and
 Hua Wu},
 title = {Tool-Augmented Reward Modeling},
 booktitle = {The Twelfth International Conference on Learning Representations (ICLR)},
 year = {2024},
 url = {https://openreview.net/forum?id=d94x0gWTUX},
}