👁 Image

GLM-4 with hopefully fixed context and some Alpaca eval maxxing. This checkpoint is SFT only and I'm planning to do APO, but still need to gen ~10K responses for training and finish up my anti-rep preference dataset.

32K context requires only 2GB VRAM, so you can fit Q4_K_M + 32k context on a single 24GB GPU, best in class for 32B dense models.

Chat Template:

GLM-4.1

Major Thanks

Thudm for the excellent GLM-4 Base

nyu-dice-lab for WildChat-50M

AI2 for the Tulu dataset

Base model

zai-org/GLM-4-32B-Base-0414

arcee-ai/GLM-4-32B-Base-32K

(2)