🧬 Helix SCE 12B

👁 helix-sce

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the SCE merge method using MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy as a base.

Models Merged

The following models were included in the merge:

• MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy
• Retreatcost/Evertide-RX-12B
• yamatazen/Aurora-SCE-12B
• Vortex5/Elysian-Sunrise-12B
• Vortex5/Mythic-Fabulist-12B
• Vortex5/Prototype-X-12b

Configuration

The following YAML configuration was used to produce this model:

architecture: MistralForCausalLM
base_model: B:\12B\MuXodious--Mistral-Nemo-Instruct-2407-absolute-heresy
models:
 - model: B:\12B\Retreatcost--Evertide-RX-12B
 - model: B:\12B\yamatazen--Aurora-SCE-12B
 - model: B:\12B\Vortex5--Elysian-Sunrise-12B
 - model: B:\12B\Vortex5--Mythic-Fabulist-12B
 - model: B:\12B\Vortex5--Prototype-X-12b
merge_method: sce
dtype: float32
out_dtype: bfloat16
parameters:
 normalize: true
 select_topk: 0.5
tokenizer:
 source: union
chat_template: auto