They may be small, but they're training like giants! • 9 items • Updated • 20
A Llama Chat Model of 160M Parameters
- Base model: JackFram/llama-160m
- Datasets:
- Availability in other ML formats:
Recommended Prompt Format
<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant
Recommended Inference Parameters
penalty_alpha: 0.5
top_k: 4
repetition_penalty: 1.01
Usage Example
from transformers import pipeline
generate = pipeline("text-generation", "Felladrin/Llama-160M-Chat-v1")
messages = [
{
"role": "system",
"content": "You are a helpful assistant who answers user's questions with details and curiosity.",
},
{
"role": "user",
"content": "What are some potential applications for quantum computing?",
},
]
prompt = generate.tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True
)
output = generate(
prompt,
max_new_tokens=1024,
penalty_alpha=0.5,
top_k=4,
repetition_penalty=1.01,
)
print(output[0]["generated_text"])
Old Open LLM Leaderboard Evaluation Results
| Metric | Value |
|---|---|
| Avg. | 30.27 |
| AI2 Reasoning Challenge (25-Shot) | 24.74 |
| HellaSwag (10-Shot) | 35.29 |
| MMLU (5-Shot) | 26.13 |
| TruthfulQA (0-shot) | 44.16 |
| Winogrande (5-shot) | 51.30 |
| GSM8k (5-shot) | 0.00 |
New Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 4.10 |
| IFEval (0-Shot) | 15.75 |
| BBH (3-Shot) | 3.17 |
| MATH Lvl 5 (4-Shot) | 0.00 |
| GPQA (0-shot) | 1.01 |
| MuSR (0-shot) | 3.17 |
| MMLU-PRO (5-shot) | 1.51 |
- Downloads last month
- 758
Safetensors
Model size
0.2B params
Tensor type
F32
·
Model tree for Felladrin/Llama-160M-Chat-v1
Base model
JackFram/llama-160mDatasets used to train Felladrin/Llama-160M-Chat-v1
Spaces using Felladrin/Llama-160M-Chat-v1 17
Collection including Felladrin/Llama-160M-Chat-v1
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard24.740
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard35.290
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard26.130
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard44.160
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard51.300
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard0.000
- strict accuracy on IFEval (0-Shot)Open LLM Leaderboard15.750
- normalized accuracy on BBH (3-Shot)Open LLM Leaderboard3.170