Gemma4-31b-Gembrain-Equinox

A V2 Fisher-protected community merge of two Gemma-4 31B creative-writing variants on top of stock google/gemma-4-31b-it. Built for the jaxxks / twisted / toasty research thread on instruct-preserving merges.

What's inside

Recipe sketch: TIES-style merge of the two community deltas (Gembrain − IT, Equinox − IT) into IT, with Fisher importance + layer-importance damping applied to the combined delta to protect the high-importance instruct-following parameters. Scale 0.3.

Per jaxxks's characterization (2026-05-23): MeroMero would be more attention-to-detail-leaning; Gembrain reinforces user-prompt attention; Equinox carries a neutral/realistic finishing edge but is best at low percentages — the Fisher+layer damping is what lets us include Equinox at a non-trivial weight without losing instruct adherence.

Chat template — important

This repo ships a modified chat_template.jinja that prefills <|channel>thought\n on assistant turns when add_generation_prompt=True and enable_thinking=True. This patches a previously-observed "merges skip thinking" regression where the assistant would open a generation without entering the thought channel first.

• enable_thinking=True → assistant turn begins inside the thought channel; produces a thought trace then folds back to the answer.
• enable_thinking=False → behaves identically to stock Gemma-4 (empty thought-block convention preserved).

If you're loading via transformers, you'll pick this up automatically because chat_template.jinja is in the repo root. If you're loading via llama.cpp from a re-quantized GGUF, make sure the GGUF was built after the chat template was placed in the source dir, or apply it post-hoc via gguf_new_metadata.py --chat-template-file.

GGUF

A Q4_K_M GGUF of this exact merge (with the chat-template fix) is mirrored at:

• ToastyPigeon/Gemma4-Test-GGUFs/g4_31b_v2_gembrain_equinox.Q4_K_M.gguf

Recommended llama-server flags for Gemma 4 31B at Q4_K_M on a dual-3090 box (subject to verification — see EVAL_GUIDELINES):

CUDA_VISIBLE_DEVICES=0,1 llama-server \
 --model g4_31b_v2_gembrain_equinox.Q4_K_M.gguf \
 --tensor-split 1,1 \
 --ctx-size 8192 --n-gpu-layers 999 \
 --parallel 8 --no-warmup --no-mmap --jinja \
 -fa on -ctk q8_0 -ctv q8_0

Client requests: cache_prompt: true. Do not pass --swa-full (default SWA-on is correct for Gemma 4).

Known caveats

• This is a V2 merge. The V3 fisher+layer family (ToastyPigeon/g4-31b-v3-fisher-layer-test) supersedes it in our internal release thread but V2 remains an interesting "low-scale community blend on IT" data point.
• Eval-side regression vs the local 31B-IT baseline was measured at roughly -19.78pp IFEval-strict in our first end-to-end test (2026-05-23). Most of that gap traced back to a chat-template-missing bug at the GGUF level (now fixed); a re-eval against the template-fixed GGUF is on the post-eval-opt queue.
• The two source community models have their own licensing terms; check both before redistribution.

Reproduction notes (high level)

1. Pull stock google/gemma-4-31b-it weights.
2. Pull the two source community 31B variants (Gembrain, Equinox).
3. Compute Δ_GB = W_GB − W_IT, Δ_EQ = W_EQ − W_IT.
4. TIES-merge the two deltas (sign-conflict resolution + magnitude pruning).
5. Apply a Fisher + layer-importance mask to damp instruct-critical parameters, then add to IT at scale 0.3.
6. Copy chat_template.jinja (with the thought-channel prefill) into the merged dir before any quant step.
7. Save merged bf16 safetensors. Optionally convert to GGUF + quant.

For the masking technique itself, see INSTRUCT_MASKING_TECHNIQUES.md (cross-reference) or the project's instruct_mask/INSTRUCT_MASKING_TECHNIQUES.md.

Credits

• Merge recipe: jaxxks (characterization) + twisted (mask infra) + toasty (training / orchestration)
• Stock base: Google's Gemma 4 31B IT
• Community variants: Gembrain, Equinox (originating authors retain credit for the source models)

Some stuff that's kinda usable • 2 items • Updated 27 days ago