wizard-tulu-dolphin-70b-v1.0-slerp

I used this blend as part of Midnight Rose v2.0.3. I haven't tested it much on its own, but I created this blend to be good at instruction following. It's nothing fancy. I just figured that WizardLM, Tulu, and Dolphin 2.2 together would be a heavy hitter for smarts.

Merge Details

Merge Method

This model was merged using the SLERP merge method.

Models Merged

The following models were included in the merge:

• /home/llm/mergequant/models/BASE/ehartford_dolphin-2.2-70b-32000vocab
• /home/llm/mergequant/models/wizard-tulu-70b-v1.0

Configuration

The following YAML configuration was used to produce this model:

models:
 - model: /home/llm/mergequant/models/wizard-tulu-70b-v1.0
 - model: /home/llm/mergequant/models/BASE/ehartford_dolphin-2.2-70b-32000vocab
merge_method: slerp
base_model: /home/llm/mergequant/models/wizard-tulu-70b-v1.0
parameters:
 t:
 - value: 0.5
dtype: float16

models:
 - model: /home/llm/mergequant/models/BASE/NousResearch_Llama-2-70b-hf
 # no parameters necessary for base model
 - model: /home/llm/mergequant/models/BASE/allenai_tulu-2-dpo-70b
 parameters:
 density: 0.35
 weight: 0.75
 - model: /home/llm/mergequant/models/BASE/WizardLM_WizardLM-70B-V1.0
 parameters:
 density: 0.35
 weight: 0.5
merge_method: dare_ties
base_model: /home/llm/mergequant/models/BASE/NousResearch_Llama-2-70b-hf
parameters:
 normalize: true
 int8_mask: true
dtype: float16
tokenzer_source: union