SemCoT: Accelerating Chain-of-Thought Reasoning through Semantically-Aligned Implicit Tokens

SemCoT is a framework that improves the efficiency of Chain-of-Thought (CoT) reasoning by encoding reasoning steps inside hidden representations ("implicit reasoning") instead of generating long textual explanations. This approach significantly speeds up inference while maintaining high reasoning performance by ensuring semantic alignment with ground-truth reasoning.

This specific checkpoint is based on princeton-nlp/Sheared-LLaMA-1.3B and fine-tuned on the ChilleD/SVAMP dataset using the SemCoT framework.

• Paper: SemCoT: Accelerating Chain-of-Thought Reasoning through Semantically-Aligned Implicit Tokens
• Code: Official GitHub Repository

🎯 Key Features

• 🗣️ Semantic Alignment: Uses a contrastively trained sentence transformer to ensure that implicit reasoning remains semantically consistent with human-readable CoT explanations.
• ⚡ Efficiency Optimization: Introduces a lightweight implicit reasoning generator, fine-tuned via knowledge distillation, to reduce token generation time and enhance inference speed.
• 🧩 Joint Optimization: SemCoT is the first approach that enhances CoT efficiency by jointly optimizing token-level generation speed and preserving semantic alignment.

🚀 Usage

This model is built using PyTorchModelHubMixin. Because SemCoT uses a custom implicit reasoning framework, please refer to the official GitHub repository for instructions on how to load and run the model.

Citation

@inproceedings{he2025semcot,
 title={SemCoT: Accelerating Chain-of-Thought Reasoning through Semantically-Aligned Implicit Tokens},
 author={He, Yinhan and Zheng, Wendy and Zhu, Yaochen and Zheng, Zaiyi and Su, Lin and Vasudevan, Sriram and Guo, Qi and Hong, Liangjie and Li, Jundong},
 booktitle={39th Conference on Neural Information Processing Systems (NeurIPS 2025)},
 year={2025}
}

Base model