Dataset and Model for "The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning" • 5 items • Updated • 5
YAML Metadata Warning:The pipeline tag "text2text-generation" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, audio-text-to-text, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-ranking, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, image-text-to-image, image-text-to-video, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, video-text-to-text, keypoint-detection, visual-document-retrieval, any-to-any, video-to-video, other
Links for Reference
- Homepage:https://github.com/kaistAI/CoT-Collection
- Repository:https://github.com/kaistAI/CoT-Collection
- Paper:https://arxiv.org/abs/2305.14045
- Point of Contact:seungone@kaist.ac.kr
TL;DR
CoT-T5 is a language model using Flan-T5 as a base model, and CoT fine-tuned on 1.84 million rationales across 1,060 tasks from the CoT Collection. Since it was CoT fine-tuned on a large amount of rationales, it shows superior performance with CoT compared to Flan-T5. One could use CoT-T5 for (1) Solving unseen tasks in zero-shot setting, and (2) Adapting to new tasks with CoT fine-tuning.
Model Details
Model Description
- Model type: Language model
- Language(s) (NLP): English
- License: Apache 2.0
- Related Models: All CoT-T5 Checkpoints
- Resources for more information:
CoT-T5 is trained with two different sizes (3B and 11B). You could check the 11B sized LM on this page. Also, check out our dataset as well on this page.
License
CoT Collection and CoT-T5 is subject to OpenAI's Terms of Use for the generated data. If you suspect any violations, please reach out to us.
Usage
Find below some example scripts on how to use the model in transformers:
Using the Pytorch model
Running the model on a CPU
Running the model on a GPU
Running the model on a GPU using different precisions
FP16
INT8
Citation
If you find the following model helpful, please considering citing our paper!
BibTeX:
@article{kim2023cot,
title={The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning},
author={Kim, Seungone and Joo, Se June and Kim, Doyoung and Jang, Joel and Ye, Seonghyeon and Shin, Jamin and Seo, Minjoon},
journal={arXiv preprint arXiv:2305.14045},
year={2023}
}
- Downloads last month
- 17