Darwin-35B-A3B-Opus

👁 Gen1
👁 Gen2
👁 Gen3

👁 9B
👁 9B Space
👁 31B
👁 31B Space

👁 35B
👁 35B Space
👁 Q8 GGUF
👁 bartowski GGUF

👁 FINAL Bench
👁 ALL Bench

35B MoE (3B active) | GPQA Diamond 90.0% (Father 84.2%, Mother 85.0%) | MMMLU 85.0% | Multimodal | 201 Languages | 262K Context | 147.8 tok/s | Apache 2.0

---

Technical Definitions

Before describing the methodology, we define the terms used throughout this document. These are not metaphors — they refer to specific, measurable quantities.

---

Benchmark Results

GPQA Diamond (198 Questions, Graduate-Level Reasoning)

Evaluation: SGLang, context 32768, temperature 0, greedy decoding, official GPQA prompt format

MMMLU (Multilingual Knowledge, 29 Languages)

• GPQA vs Father: +6.9% relative improvement
• GPQA vs Mother: +5.9% relative improvement
• MMMLU: Father-level multilingual knowledge preserved (85.0% vs 85.2%)

---

Parent Models

Both parents share the identical Qwen3.5-35B-A3B architecture (40 layers, 256 experts, GDN+MoE hybrid). The Mother is a LoRA SFT on the same base — not a different architecture. "Text-only" refers to the training data (Claude 4.6 Opus reasoning chains), not the model structure.

---

Methodology: Darwin V5

Relationship to Existing Tools

Darwin V5 uses mergekit as its merge backend. We do not claim to have invented evolutionary merging — mergekit's evolve feature already provides this capability. What Darwin adds is a three-phase diagnostic pipeline that wraps mergekit with pre-merge profiling and post-merge verification.

Pipeline

Standard mergekit evolve:
 Random initial params --> Evolve --> Best score

Darwin V5:
 Phase 0: Profile both parents (40 layers x 256 experts)
 | Measure: expert activation frequency, routing entropy,
 | probe cosine distance per layer
 v
 Phase 1: Evolution with diagnostic-informed initial genome
 | Search space constrained by dead expert map + layer importance
 v
 Phase 2: mergekit DARE-TIES merge + benchmark evaluation
 | (same merge backend as standard mergekit)
 v
 Phase 3: Profile the child, compare against both parents
 | Detect: interference, function loss, dead expert inheritance
 v
 Final model

What Darwin V5 Adds Over Standard mergekit evolve

How Diagnostics Changed the Merge

Without diagnostics (V4 blind evolution):

• ratio=0.481, attn=0.168, ffn=0.841
• Uniform across all 40 layers

With diagnostics (V5):

• L0-L37: t=0.599 (Mother 60%), Mother's router
• L38: t=0.900 (Mother 90%), Mother's router — identified as reasoning core by probe cosine distance
• L39: t=0.534 (Father 47%), Father's router — preserves output/multimodal routing

The diagnostic profile identified L38 as having the highest cosine distance on REASONING and CODE probes. This informed the per-block strategy rather than relying on blind search to discover it.

---

Parent Model Diagnostics

Mother: Expert Activation Analysis

👁 Mother MoE Health

👁 Mother Expert Utilization

👁 Mother Probe Cosine Distance

L34-L38 shows high cosine distance across REASONING, CODE, LOGIC probes — this is where the Claude distillation concentrated its reasoning patterns.

Father: Baseline Profile

👁 Father MoE Health

👁 Father Expert Utilization

👁 Father Layer Importance by Probe

The Father shows uniform expert activation across all 40 layers — all experts active. This makes it suitable as a donor for the Mother's inactive expert slots.

Parent Comparison

👁 Parent A vs B Layer Advantage

• Above zero: Father stronger — L0-L5 (embedding/early layers)
• Below zero: Mother stronger — L5-L35 consistent advantage
• L34-L38: Mother peaks on REASONING and CODE probes
• L39: Father recovers — output layer

This advantage map directly informed the 3-block merge recipe.

---

Merge Configuration

👁 MRI-Guided Genome

👁 Merge Ratio per Layer

# Darwin V5 diagnostic-guided layer-wise merge
# Method: DARE-TIES via mergekit
# Genome: ratio=0.800 attn=0.320 ffn=0.590 density=0.799

L0-L37: t=0.5988 (Mother 60%) — router from Mother
L38: t=0.9000 (Mother 90%) — reasoning core
L39: t=0.5336 (Father 47%) — router from Father (output routing)

---

Post-Merge Health Check

👁 Darwin Health Check

Layer-by-layer importance comparison between the child and both parents:

• Layer 0 (Embedding): Child 0.42, parents 0.35-0.50. No interference.
• Layers 1-33: Near-zero across all three. Normal for MoE middle layers.
• Layers 34-39: Importance rises. Child matches or exceeds parents — reasoning transfer confirmed.
• Layer 39 (Output): Child 0.48, matching parents. Output intact.

No interference detected. No function loss detected.

---

Inherited Capabilities

From Father (Qwen3.5-35B-A3B):

• Multimodal: Image and video understanding
• 201 Languages: Multilingual coverage
• 262K Context: Native long-context (extendable to 1M via YaRN)
• Gated DeltaNet + MoE architecture
• Multi-Token Prediction

From Mother (Claude 4.6 Opus Distilled):

• Structured step-by-step reasoning within <think> tags
• Coding agent compatibility
• Tool calling stability

---

Performance

---

Model Specifications

---

Usage

SGLang (Recommended)

python -m sglang.launch_server \
 --model-path FINAL-Bench/Darwin-35B-A3B-Opus \
 --tp 1 \
 --mem-fraction-static 0.90 \
 --context-length 32768 \
 --trust-remote-code

vLLM

vllm serve FINAL-Bench/Darwin-35B-A3B-Opus \
 --trust-remote-code \
 --enforce-eager

Transformers

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained(
 "FINAL-Bench/Darwin-35B-A3B-Opus",
 trust_remote_code=True,
 use_fast=True,
)
model = AutoModelForCausalLM.from_pretrained(
 "FINAL-Bench/Darwin-35B-A3B-Opus",
 dtype="bfloat16",
 device_map="auto",
 trust_remote_code=True,
)

---

Evolution Details

---

Acknowledgements

• Korean Government — GPU Support Program research grant
• Qwen Team — Qwen3.5-35B-A3B base architecture
• Jackrong — Claude 4.6 Opus Reasoning Distilled model
• mergekit — Merge backend infrastructure
• nohurry, TeichAI — Distillation datasets

---

Citation

@misc{vidraft_darwin_35b_opus,
 title = {Darwin-35B-A3B-Opus: Diagnostic-Guided Evolutionary Merge},
 author = {VIDRAFT},
 year = {2026},
 publisher = {Hugging Face},
 howpublished = {\url{https://huggingface.co/FINAL-Bench/Darwin-35B-A3B-Opus}}
}

---

FAQ

This model is introduced in Darwin Family.

비드래프트 • 47 items • Updated 4 days ago • 22