👁 image/png

🔢 EulerMath-Mistral-7B

This model is a full fine-tuned version of meta-math/MetaMath-Mistral-7B on the following datasets:

• 🧮 TIGER-Lab/MathInstruct
• 📐 microsoft/orca-math-word-problems-200k

This model is finetuned using 8xRTX3090 + 1xRTXA6000 using axolotl.

This model's training was sponsored by sablo.ai.

💬 Prompt Template

You can use this prompt template while using the model:

Alpaca

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{instruction}

### Response:

This prompt template is available as a chat template, which means you can format messages using the tokenizer.apply_chat_template() method:

messages = [
 {"role": "system", "content": "You are helpful AI asistant."},
 {"role": "user", "content": "Hello!"}
]
gen_input = tokenizer.apply_chat_template(message, return_tensors="pt")
model.generate(**gen_input)

🔄 Quantizationed versions

GGUF @bartowski

• https://huggingface.co/bartowski/EulerMath-Mistral-7B-GGUF

ExLlamaV2 @bartowski

• https://huggingface.co/bartowski/EulerMath-Mistral-7B-exl2

AWQ @solidrust

• https://huggingface.co/solidrust/EulerMath-Mistral-7B-AWQ

🎯 Evaluation Results

Evaluation Results of this model are low due to the strict requirements for the eval GSM8K eval harness. I evaluated this model using tinyGSM8k which is a streamlined subset of 100 data points from the GSM8K dataset, enabling efficient evaluation of large language models with reduced computational resources.

The results are as follows:

{
 "exact_match,strict-match": 0.02,
 "exact_match_stderr,strict-match": 0.014070529413628952,
 "exact_match,flexible-extract": 0.73,
 "exact_match_stderr,flexible-extract": 0.04461960433384741,
 "alias": "gsm8k"
}

As you can see from the results, this model does not meet the required format for strict-match results but the given answers is actually correct. However, as indicated by the flexible-extract part, this model is actually quite proficient at math.

🤖 Additional information about training

This model is full fine-tuned for 2 epoch.

Total number of steps was 544.

🤝 Acknowledgments

Thanks to sablo.ai for sponsoring this model.

Thanks to all the dataset authors mentioned in the datasets section.

Thanks to axolotl for making the repository I used to make this model.

Thanks to all open source AI community.

👁 Built with Axolotl

If you would like to support me:

☕ Buy Me a Coffee

Base model

meta-math/MetaMath-Mistral-7B

(5)