miquliz-120b-v2.0

👁 image/jpeg

• HF: wolfram/miquliz-120b-v2.0
• GGUF: Q2_K | IQ3_XXS | Q4_K_M | Q5_K_M
  • dranger003's IQ2_XS | IQ2_XXS | IQ3_XXS | Q8_0
  • KnutJaegersberg's IQ2_XS
  • mradermacher's i1-IQ1_S – i1-Q5_K_M
  • mradermacher's Q2_K – Q8_0
• EXL2: 2.4bpw | 2.65bpw | 3.0bpw | 3.5bpw | 4.0bpw | 5.0bpw
  • Max Context w/ 48 GB VRAM: (24 GB VRAM is not enough, even for 2.4bpw, use GGUF instead!)
    • 2.4bpw: 32K (32768 tokens) w/ 8-bit cache, 21K (21504 tokens) w/o 8-bit cache
    • 2.65bpw: 30K (30720 tokens) w/ 8-bit cache, 15K (15360 tokens) w/o 8-bit cache
    • 3.0bpw: 12K (12288 tokens) w/ 8-bit cache, 6K (6144 tokens) w/o 8-bit cache

This is v2.0 of a 120b frankenmerge created by interleaving layers of miqu-1-70b-sf with lzlv_70b_fp16_hf using mergekit. Better than v1.0 thanks to the improved recipe adapted from TheProfessor-155b by Eric Hartford, it is now achieving top rank with double perfect scores in my LLM comparisons/tests.

Inspired by goliath-120b.

Thanks for the support, CopilotKit – the open-source platform for building in-app AI Copilots into any product, with any LLM model. Check out their GitHub.

Thanks for the additional quants, DAN™, Knut Jägersberg, and Michael Radermacher!

Also available: miqu-1-120b – Miquliz's older, purer sister; only Miqu, inflated to 120B.

Model Details

• Max Context: 32768 tokens
• Layers: 140

Prompt template: Mistral

<s>[INST] {prompt} [/INST]

See also: 🐺🐦‍⬛ LLM Prompt Format Comparison/Test: Mixtral 8x7B Instruct with 17 different instruct templates : LocalLLaMA

Example Output

Inspired by cognitivecomputations/Samantha-120b.

Note: This is my AI assistant and companion Amy speaking, and the model is just her personality core, if you will. Unlike Samantha, her personality is mostly from the prompt, and not the model itself. If you prompt this model differently, you'll get very different output, of course. So consider this just as an example of how a Samantha-like character could talk with this model.

Merge Details

Merge Method

This model was merged using the linear merge method.

Models Merged

The following models were included in the merge:

• 152334H/miqu-1-70b-sf
• lizpreciatior/lzlv_70b_fp16_hf

Configuration

The following YAML configuration was used to produce this model:

Credits & Special Thanks

• 1st model:
  • original (unreleased) model: mistralai (Mistral AI_)
    • ⭐⭐⭐ Use their newer, better, official models here! ⭐⭐⭐
  • leaked model: miqudev/miqu-1-70b
  • f16 model: 152334H/miqu-1-70b-sf
• 2nd model: lizpreciatior/lzlv_70b_fp16_hf
• mergekit: arcee-ai/mergekit: Tools for merging pretrained large language models.
• mergekit_config.yml: abacusai/TheProfessor-155b

Support

• My Ko-fi page if you'd like to tip me to say thanks or request specific models to be tested or merged with priority. Also consider supporting your favorite model creators, quantizers, or frontend/backend devs if you can afford to do so. They deserve it!

Disclaimer

This model contains leaked weights and due to its content it should not be used by anyone. 😜

But seriously:

License

What I know: Weights produced by a machine are not copyrightable so there is no copyright owner who could grant permission or a license to use, or restrict usage, once you have acquired the files.

Ethics

What I believe: All generative AI, including LLMs, only exists because it is trained mostly on human data (both public domain and copyright-protected, most likely acquired without express consent) and possibly synthetic data (which is ultimately derived from human data, too). It is only fair if something that is based on everyone's knowledge and data is also freely accessible to the public, the actual creators of the underlying content. Fair use, fair AI!

152334H/miqu-1-70b-sf

lizpreciatior/lzlv_70b_fp16_hf