Voozh

WizardCoder: Empowering Code Large Language Models with Evol-Instruct

📃 [WizardLM] • 📃 [WizardCoder] • 📃 [WizardMath]

👋 Join our Discord

News

[2024/01/04] 🔥 We released WizardCoder-33B-V1.1 trained from deepseek-coder-33b-base, the SOTA OSS Code LLM on EvalPlus Leaderboard, achieves 79.9 pass@1 on HumanEval, 73.2 pass@1 on HumanEval-Plus, 78.9 pass@1 on MBPP, and 66.9 pass@1 on MBPP-Plus.

[2024/01/04] 🔥 WizardCoder-33B-V1.1 outperforms ChatGPT 3.5, Gemini Pro, and DeepSeek-Coder-33B-instruct on HumanEval and HumanEval-Plus pass@1.

[2024/01/04] 🔥 WizardCoder-33B-V1.1 is comparable with ChatGPT 3.5, and surpasses Gemini Pro on MBPP and MBPP-Plus pass@1.

Model	Checkpoint	Paper	HumanEval	HumanEval+	MBPP	MBPP+	License
GPT-4-Turbo (Nov 2023)	-	-	85.4	81.7	83.0	70.7	-
GPT-4 (May 2023)	-	-	88.4	76.8	-	-	-
GPT-3.5-Turbo (Nov 2023)	-	-	72.6	65.9	81.7	69.4	-
Gemini Pro	-	-	63.4	55.5	72.9	57.9	-
DeepSeek-Coder-33B-instruct	-	-	78.7	72.6	78.7	66.7	-
WizardCoder-33B-V1.1	🤗 HF Link	📃 [WizardCoder]	79.9	73.2	78.9	66.9	MSFTResearch
WizardCoder-Python-34B-V1.0	🤗 HF Link	📃 [WizardCoder]	73.2	64.6	73.2	59.9	Llama2
WizardCoder-15B-V1.0	🤗 HF Link	📃 [WizardCoder]	59.8	52.4	--	--	OpenRAIL-M
WizardCoder-Python-13B-V1.0	🤗 HF Link	📃 [WizardCoder]	64.0	--	--	--	Llama2
WizardCoder-Python-7B-V1.0	🤗 HF Link	📃 [WizardCoder]	55.5	--	--	--	Llama2
WizardCoder-3B-V1.0	🤗 HF Link	📃 [WizardCoder]	34.8	--	--	--	OpenRAIL-M
WizardCoder-1B-V1.0	🤗 HF Link	📃 [WizardCoder]	23.8	--	--	--	OpenRAIL-M

Our WizardMath-70B-V1.0 model slightly outperforms some closed-source LLMs on the GSM8K, including ChatGPT 3.5, Claude Instant 1 and PaLM 2 540B.
Our WizardMath-70B-V1.0 model achieves 81.6 pass@1 on the GSM8k Benchmarks, which is 24.8 points higher than the SOTA open-source LLM, and achieves 22.7 pass@1 on the MATH Benchmarks, which is 9.2 points higher than the SOTA open-source LLM.