Multilingual Educational Content Classifier

Trained on full documents of up to 8192 tokens in total. The train set of tartuNLP/fineweb-c-combined-resample was used, which itself is a mix and a resample of HuggingFaceFW/fineweb-edu-llama3-annotations and data-is-better-together/fineweb-c.

Labels

{0: '❗ Problematic Content ❗', 1: 'None', 2: 'Minimal', 3: 'Basic', 4: 'Good', 5: 'Excellent'}

Classification Report

Evaluated on the development set of tartuNLP/fineweb-c-combined-resample organized so that each language appears at least once.

 precision recall f1-score support

 0 0.89 0.78 0.83 602
 1 0.65 0.88 0.75 916
 2 0.41 0.29 0.34 345
 3 0.40 0.30 0.34 179
 4 0.53 0.15 0.23 127
 5 0.55 0.39 0.45 44

 accuracy 0.66 2213
 macro avg 0.57 0.46 0.49 2213
weighted avg 0.65 0.66 0.64 2213

Confusion Matrix

[[471 114 10 6 0 1]
 [ 33 806 59 13 5 0]
 [ 10 204 101 28 2 0]
 [ 7 72 37 53 8 2]
 [ 7 35 27 28 19 11]
 [ 2 7 10 6 2 17]]

Base model