Contains some information and experiments fine-tuning LLMs using 🤗 `trl.ORPOTrainer` • 7 items • Updated • 5
ORPO fine-tune of Mistral 7B v0.1 with DPO Mix 7K
Stable Diffusion XL "A capybara, a killer whale, and a robot named Ultra being friends"
This is an ORPO fine-tune of mistralai/Mistral-7B-v0.1 with
alvarobartt/dpo-mix-7k-simplified.
⚠️ Note that the code is still experimental, as the ORPOTrainer PR is still not merged, follow its progress
at 🤗trl - ORPOTrainer PR.
Reference
ORPO: Monolithic Preference Optimization without Reference Model
- Downloads last month
- 6
Safetensors
Model size
7B params
Tensor type
BF16
·