Run in llama.cpp:

./llama.cpp/llama-cli -hf unsloth/Magistral-Small-2509-GGUF:UD-Q4_K_XL --jinja --temp 0.7 --top-k -1 --top-p 0.95 -ngl 99

Run in Ollama:

ollama run hf.co/unsloth/Magistral-Small-2509-GGUF:UD-Q4_K_XL

Read our in-depth guide about Magistral 1.2: docs.unsloth.ai/basics/magistral

• Fine-tune Magistral 1.2 for free using our Kaggle notebook here!
• View the rest of our notebooks in our docs here.

---

Magistral Small 1.2

Building upon Mistral Small 3.2 (2506), with added reasoning capabilities, undergoing SFT from Magistral Medium traces and RL on top, it's a small, efficient reasoning model with 24B parameters.

Magistral Small can be deployed locally, fitting within a single RTX 4090 or a 32GB RAM MacBook once quantized.

Learn more about Magistral in our blog post.

The model was presented in the paper Magistral.

Updates compared with Magistral Small 1.1

• Multimodality: The model now has a vision encoder and can take multimodal inputs, extending its reasoning capabilities to vision.
• Performance upgrade: Magistral Small 1.2 should give you significatively better performance than Magistral Small 1.1 as seen in the benchmark results.
• Better tone and persona: You should experiment better LaTeX and Markdown formatting, and shorter answers on easy general prompts.
• Finite generation: The model is less likely to enter infinite generation loops.
• Special think tokens: [THINK] and [/THINK] special tokens encapsulate the reasoning content in a thinking chunk. This makes it easier to parse the reasoning trace and prevents confusion when the '[THINK]' token is given as a string in the prompt.
• Reasoning prompt: The reasoning prompt is given in the system prompt.

Key Features

• Reasoning: Capable of long chains of reasoning traces before providing an answer.
• Multilingual: Supports dozens of languages, including English, French, German, Greek, Hindi, Indonesian, Italian, Japanese, Korean, Malay, Nepali, Polish, Portuguese, Romanian, Russian, Serbian, Spanish, Turkish, Ukrainian, Vietnamese, Arabic, Bengali, Chinese, and Farsi.
• Vision: Vision capabilities enable the model to analyze images and reason based on visual content in addition to text.
• Apache 2.0 License: Open license allowing usage and modification for both commercial and non-commercial purposes.
• Context Window: A 128k context window. Performance might degrade past 40k but Magistral should still give good results. Hence we recommend to leave the maximum model length to 128k and only lower if you encounter low performance.

Benchmark Results

Sampling parameters

Please make sure to use:

• top_p: 0.95
• temperature: 0.7
• max_tokens: 131072

Basic Chat Template

We highly recommend including the following system prompt for the best results, you can edit and customise it if needed for your specific use case.

First draft your thinking process (inner monologue) until you arrive at a response. Format your response using Markdown, and use LaTeX for any mathematical equations. Write both your thoughts and the response in the same language as the input.

Your thinking process must follow the template below:[THINK]Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate the response. Use the same language as the input.[/THINK]Here, provide a self-contained response.

The [THINK] and [/THINK] are special tokens that must be encoded as such.

Please make sure to use mistral-common as the source of truth. Find below examples from libraries supporting mistral-common.

We invite you to choose, depending on your use case and requirements, between keeping reasoning traces during multi-turn interactions or keeping only the final assistant response.

Ping model as follows:

Transformers

Make sure you install the latest Transformers version:

pip install --upgrade transformers[mistral-common]

This should also install mistral_common >= 1.8.5

To check:

python -c "import mistral_common; print(mistral_common.__version__)"

Now you can use Transformers with Magistral:

Base model

mistralai/Mistral-Small-3.1-24B-Base-2503

mistralai/Mistral-Small-3.2-24B-Instruct-2506

mistralai/Magistral-Small-2509

(27)