open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-1T-4096-rope_theta-100k
1.7B open-sci-ref model trained on FineWeb-Edu for 1T tokens (sequence length 4096, RoPE theta = 100000).
The main branch holds the final checkpoint (iter 238419). Intermediate checkpoints (iters 58000-238000, every 2000) are available as branches named iter_XXXXXXX.
Evaluation
Final checkpoint on the open-sci-0.01 suite (lm-eval-harness). Metrics collected with oellm collect-results.
| Task | n-shot | Metric | Score |
|---|---|---|---|
| arc_challenge | 10 | acc_norm | 0.4983 |
| arc_easy | 10 | acc_norm | 0.7984 |
| boolq | 10 | acc | 0.7465 |
| commonsense_qa | 10 | acc | 0.2285 |
| copa | 0 | acc | 0.8400 |
| hellaswag | 10 | acc_norm | 0.7008 |
| lambada_openai | 0 | acc | 0.5849 |
| mmlu | 5 | acc | 0.3187 |
| openbookqa | 0 | acc_norm | 0.4360 |
| piqa | 10 | acc_norm | 0.7873 |
| social_iqa | 0 | acc | 0.4365 |
| winogrande | 0 | acc | 0.6425 |
| average | 0.5849 |
- Downloads last month
- 65
Safetensors
Model size
2B params
Tensor type
BF16
·