Kabyle Sentence Transformer (MPNet)

A sentence embedding model specifically fine-tuned for Kabyle (Taqbaylit) - English cross-lingual semantic similarity.

Model Details

Training Data

Performance

Compared to the base paraphrase-multilingual-mpnet-base-v2 (untrained):

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("boffire/kabyle-sentence-transformer-mpnet")

# Embed English and Kabyle
sentences = ["Hello!", "Azul!"]
embeddings = model.encode(sentences)

# Cross-lingual similarity
from sklearn.metrics.pairwise import cosine_similarity
sim = cosine_similarity([embeddings[0]], [embeddings[1]])
print(sim)

Limitations

• Trained primarily on parallel data; monolingual Kabyle similarity not explicitly optimized
• Best for EN<->KAB cross-lingual tasks; Kabyle<->Kabyle may work but is untested
• Religious text overrepresented in NLLB portion; may underperform on highly technical/modern domains
• Evaluator used constant labels (all 1.0) due to all pairs being positive; correlation metrics were undefined

Future Work

• Train v2 with Davlan/afro-xlmr-large backbone for African-specific pretraining
• Add monolingual Kabyle data for better Kabyle<->Kabyle similarity
• Fix evaluator to use AvgCosineEvaluator instead of correlation-based metrics
• Evaluate against LASER on a proper benchmark

Citation

If you use this model, please cite:

@misc{kabyle-st-mpnet,
 title={Kabyle Sentence Transformer},
 author={boffire},
 year={2026},
 howpublished={\url{https://huggingface.co/boffire/kabyle-sentence-transformer-mpnet}}
}

Acknowledgments

• Imsidag-community for the cleaned parallel corpora
• Tatoeba contributors for community translations
• Meta AI for LASER and NLLB datasets
• boffire community for Kabyle NLP tooling

This is about kabyle language resources. • 24 items • Updated 3 days ago