Language Decoded LoRA — Condition 2: Chinese Reserved-Word Translation (Phase 2 · The Stack v1)

⚠️ Deprecated — preliminary (Phase 2 · The Stack v1)

This adapter is the original March-2026 hackathon (Phase 2) model, trained on bigcode/the-stack (v1, non-dedup). It is superseded by the paper's Phase 3 adapter, which was re-trained from scratch on the cleaner bigcode/the-stack-v2-dedup corpus. For paper-grade use, load the Phase 3 adapter from the umbrella repo:

PeftModel.from_pretrained(base_model, "legesher/language-decoded-lora", subfolder="tiny-aya-base/condition-2-zh-5k-seed42")

This repo is kept for reproducibility of the preliminary results only — do not cite it for the paper. It was renamed from legesher/language-decoded-lora-condition-2-zh-5k; the old URL continues to resolve via a Hugging Face redirect.

Chinese reserved-word-translated Python via Legesher (Phase 2: v0.5.1 / v0.6.0), 5k subset of bigcode/the-stack (v1). Tests whether the language of Python's reserved words matters for reasoning benefits.

Part of the Language Decoded project (Cohere's Tiny Aya Expedition).

For the full adapter inventory across both phases, see the Language Decoded LoRA hub and its MANIFEST.md.

Training Data

legesher/language-decoded-data / phase-2-the-stack-v1-condition-2-zh-5k — the Phase 2 / The Stack v1 config.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-base")
tokenizer = AutoTokenizer.from_pretrained("CohereLabs/tiny-aya-base")
# Preliminary Phase 2 adapter (kept for reproducibility):
model = PeftModel.from_pretrained(base_model, "legesher/language-decoded-lora-phase-2-the-stack-v1-condition-2-zh-5k")

Citation

@misc{language-decoded-2026,
 title={Language, Decoded: Exploring the Impact of Fine-Tuning a Multilingual Model on Native-Language Code},
 author={Madison Edgar and Saad Ahmed Bazaz and Tom Sherborne and Rashik Shahjahan and Khojasteh Mirza and Sarah Jawaid and Rafay Mustafa and Sohaib Ahmed Bazaz},
 year={2026},
 publisher={Hugging Face},
 url={https://huggingface.co/legesher/language-decoded-lora}
}

License

Apache 2.0

Base model