Kite

🎉 You are looking at Kite 2.6, which is now using a Mistral base!

Kite is a small, trained, 13 million parameter language model, without any special optimizations.

Training

It was trained on 50K rows of this dataset using 12500 steps, 1 epoch, 4 batch size, 5e-4 learning rate, and the pika 2 tokenizer.

Limitations

Due to its size, the model is not suitable for production workloads.

Loss

👁 Image