Jon Durbin's Airoboros 7B GPT4 fp16

These files are fp16 pytorch format model files for Jon Durbin's Airoboros 7B GPT4.

They are the result of converting the original fp32 model to fp16 for quicker download and easier storage.

Repositories available

• 4-bit GPTQ models for GPU inference
• 4-bit, 5-bit, and 8-bit GGML models for CPU+GPU inference
• Unquantised fp16 model in pytorch format, for GPU inference and for further conversions

Prompt template

Uses the Vicuna 1.1 format:

USER: prompt
ASSISTANT:

Discord

For further support, and discussions on these models and AI in general, join us at:

TheBloke AI's Discord server

Thanks, and how to contribute.

Thanks to the chirper.ai team!

I've had a lot of people ask if they can contribute. I enjoy providing models and helping people, and would love to be able to spend even more time doing it, as well as expanding into new projects like fine tuning/training.

If you're able and willing to contribute it will be most gratefully received and will help me to keep providing more models, and to start work on new AI projects.

Donaters will get priority support on any and all AI/LLM/model questions and requests, access to a private Discord room, plus other benefits.

• Patreon: https://patreon.com/TheBlokeAI
• Ko-Fi: https://ko-fi.com/TheBlokeAI

Patreon special mentions: Aemon Algiz, Dmitriy Samsonov, Nathan LeClaire, Trenton Dambrowitz, Mano Prime, David Flickinger, vamX, Nikolai Manek, senxiiz, Khalefa Al-Ahmad, Illia Dulskyi, Jonathan Leane, Talal Aujan, V. Lukas, Joseph William Delisle, Pyrater, Oscar Rangel, Lone Striker, Luke Pendergrass, Eugene Pentland, Sebastain Graf, Johann-Peter Hartman.

Thank you to all my generous patrons and donaters!

Original model card: Jon Durbin's Airoboros 7B GPT4

Overview

This is a fine-tuned 13b parameter LlaMa model, using completely synthetic training data created gpt4 via https://github.com/jondurbin/airoboros

The context size has been increased to 4096.

The dataset used to fine-tune this model is available here, with a specific focus on:

• trivia
• math/reasoning (although it still sucks)
• coding
• multiple choice and fill-in-the-blank
• context-obedient question answering
• theory of mind
• misc/general

This model was fine-tuned with a fork of FastChat, and therefore uses the standard vicuna template:

USER:
[prompt]

<\s>
ASSISTANT:

The most important bit, to me, is the context obedient question answering support, without extensive prompt engineering.

Usage

The easiest way to get started is to use my fork of FastChat, which is mostly the same but allows for the increased context length and adds support for multi-line inputs:

pip install git+https://github.com/jondurbin/FastChat

Then, you can invoke it like so (after downloading the model):

python -m fastchat.serve.cli
 --model-path airoboros-13b-gpt4 \
 --temperature 0.5 \
 --max-new-tokens 4096 \
 --context-length 4096 \
 --conv-template vicuna_v1.1 \
 --no-history

Context obedient question answering

By obedient, I mean the model was trained to ignore what it thinks it knows, and uses the context to answer the question. The model was also tuned to limit the values to the provided context as much as possible to reduce hallucinations.

The format for a closed-context prompt is as follows:

BEGININPUT
BEGINCONTEXT
url: https://some.web.site/123
date: 2023-06-01
... other metdata ...
ENDCONTEXT
[insert your text blocks here]
ENDINPUT
[add as many other blocks, in the exact same format]
BEGININSTRUCTION
[insert your instruction(s). The model was tuned with single questions, paragraph format, lists, etc.]
ENDINSTRUCTION

Here's a trivial, but important example to prove the point:

BEGININPUT
BEGINCONTEXT
date: 2021-01-01
url: https://web.site/123
ENDCONTEXT
In a shocking turn of events, blueberries are now green, but will be sticking with the same name.
ENDINPUT
BEGININSTRUCTION
What color are bluberries? Source?
ENDINSTRUCTION

And the response:

Blueberries are now green.
Source:
date: 2021-01-01
url: https://web.site/123

Other stuff

Better coding

Example:

Create a python application with the following requirements:
- Asyncio FastAPI webserver
- ping endpoint that returns the current date in JSON format
- file upload endpoint, which calculates the file's sha256 checksum, and checks postgres to deduplicate

Word games / trivia

USER: This anagram of "silent" is what you do when paying attention and hearing someone.
ASSISTANT: listen

USER: Give me a list of 7 words that start with EN
ASSISTANT: 1. Endlessly
2. Enthusiastically
3. Enormously
4. Enveloping
5. Enduring
6. Enlightening
7. Encouraging

Multiple choice

USER: Which of the following astronomical objects is composed primarily of ice and rock, and is located in the Kuiper Belt?

A) Mercury
B) Venus
C) Pluto
D) Jupiter
ASSISTANT: [C, Pluto]

Writing