open-sci-ref-v0.02-1.7b-nemotron-hq-1T-4096-rope_theta-100k
1.7B open-sci-ref model trained on Nemotron-CC HQ for 1T tokens (sequence length 4096, RoPE theta = 100000).
The main branch holds the final checkpoint (iter 238419). Intermediate checkpoints (iters 58000-238000, every 2000) are available as branches named iter_XXXXXXX.
Evaluation
Final checkpoint on the open-sci-0.01 suite (lm-eval-harness). Metrics collected with oellm collect-results.
| Task | n-shot | Metric | Score |
|---|---|---|---|
| arc_challenge | 10 | acc_norm | 0.5316 |
| arc_easy | 10 | acc_norm | 0.8190 |
| boolq | 10 | acc | 0.7615 |
| commonsense_qa | 10 | acc | 0.6003 |
| copa | 0 | acc | 0.8600 |
| hellaswag | 10 | acc_norm | 0.7440 |
| lambada_openai | 0 | acc | 0.6136 |
| mmlu | 5 | acc | 0.5212 |
| openbookqa | 0 | acc_norm | 0.4360 |
| piqa | 10 | acc_norm | 0.8041 |
| social_iqa | 0 | acc | 0.4458 |
| winogrande | 0 | acc | 0.6606 |
| average | 0.6498 |
- Downloads last month
- 61
Safetensors
Model size
2B params
Tensor type
BF16
·