🧠 Dixtral_QA — BUT-FIT Diarization-Conditioned Voxtral for Spoken QA

This repository hosts Dixtral_QA, developed by BUT Speech@FIT. Dixtral couples the Voxtral-Mini-3B spoken-language model with the DiCoW diarization-conditioned encoder, giving the LLM target-speaker awareness in multi-talker audio.

This checkpoint is tuned for spoken question answering over conversational/meeting audio. For pure target-speaker transcription, use Dixtral_TS-ASR instead.

🛠️ Model Usage

from transformers import AutoModel, AutoProcessor

MODEL_NAME = "BUT-FIT/Dixtral_QA"
model = AutoModel.from_pretrained(MODEL_NAME, trust_remote_code=True)
processor = AutoProcessor.from_pretrained(MODEL_NAME)

➡️ For full inference pipelines (diarization → FDDT masks → generation), see the Dixtral GitHub repository.

---

📦 Model Details

• Base Model: Voxtral-Mini-3B-2507
• Encoder: DiCoW v3 large
• Training Datasets:
  • NOTSOFAR-1
  • AMI Meeting Corpus
  • LibriMix / LibriSpeechMix

---

📬 Contact

📧 Email: ipoloka@fit.vut.cz 🏢 Affiliation: BUT Speech@FIT, Brno University of Technology 🔗 GitHub: BUTSpeechFIT

Base model

mistralai/Voxtral-Mini-3B-2507

(19)