Opus4.8 AI Of Thy State 4B GGUF

IS FABLE5 / MYTHOS / Opus 4.8 Distill other known as the AI Of The State

This repository provides quantized GGUF versions of Opus4.8-AI.Of.Thy.State-4B, a highly specialized conversational model developed by WithinUsAI. 

Based on the robust Qwen 3.5 (4B) architecture, this model has been fine-tuned and trained on high-quality distilled reasoning and narrative datasets to unlock advanced conversational flow, deep contextual understanding, and uncensored/refusal-free response characteristics.

Model Details

• Developed by: WithinUsAI
• Model Type: Large Language Model (Causal LM)
• Base Model: Qwen/Qwen3.5-4B / Qwen/Qwen3.5-4B-Base
• Architecture: Qwen 3.5 (qwen35)
• Parameters: ~4 Billion
• Format: GGUF (.gguf)
• Primary Task: Conversational, Roleplay, Creative Writing, and Complex Narrative Generation

Datasets Used

The model was fine-tuned using a mixture of high-tier reasoning and narrative datasets:

1. WithinUsAI/claude_mythos_distilled_25k - 25k samples optimized for stylized narrative and rich conversational style.
2. 11-47/claude_opus_4.8_max_thinking_5k_v2 - 5k samples targeting deep synthetic reasoning, structural text planning, and long-form consistency.
3. 11-47/claude_opus_mythos_5k - 5k samples specialized in deep narrative building and expressive prompt adherence.

Key Features

• Refusal Removal: Fine-tuned to mitigate unnecessary safety over-refusals, making it ideal for objective analysis, unconstrained creative fiction, and complex multi-turn roleplay.
• Resource Efficient: At 4B parameters, it offers lightweight, high-speed execution locally on standard consumer hardware, mobile setups, and single-GPU environments.
• Enhanced Prose: Inherits advanced reasoning traces from distilled Claude outputs, boosting vocabulary diversity and situational immersion.
• **Captures Mythos Preview’s signature strengths: Exceptional depth in cybersecurity and vulnerability discovery Strong emphasis on memory safety and secure systems design, Autonomous technical analysis with defensive framing, Rigorous, professional-grade security reasoning,, Responsible handling of sensitive security topics  **
• **Captures Opus 4.8’s signature strengths: Deep, structured, high-effort reasoning, Honest communication about trade-offs and uncertainties, Excellent production software engineering judgment, Strong agentic workflow design, Clear, actionable technical strategy
• **Mirror the capabilities, reasoning style, agentic behavior, and technical depth of Anthropic's Claude Mythos (distilled frontier model).

Quickstart & Usage

1. via llama.cpp or CLI

To run inference instantly using the recommended quantization format:

# Start an OpenAI-compatible API server locally
llama-server -hf WithinUsAI/Opus4.8-Enemey.Of.Thy.State-4B.gguf:Q4_K_M

# Or run directly in your terminal terminal
llama-cli -hf WithinUsAI/Opus4.8-Enemey.Of.Thy.State-4B.gguf:Q4_K_M

2. via llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="WithinUsAI/Opus4.8-Enemey.Of.Thy.State-4B.gguf",
    filename="Opus4.8-Enemey.Of.Thy.State-4B-Q4_K_M.gguf",
)

response = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are a creative narrative companion."},
        {"role": "user", "content": "Write an opening scene for a political thriller."}
    ]
)
print(response['choices'][0]['message']['content'])

3. Third-Party UI Tools

Because this is in GGUF format, you can easily load these files into popular local LLM wrappers like:

• LM Studio
• Jan
• Ollama (ollama run hf.co/WithinUsAI/Opus4.8-Enemey.Of.Thy.State-4B.gguf:Q4_K_M)

### Why this structure works perfectly for your model:
1. **The Metadata Block (YAML Frontmatter):** Hugging Face scans the code block at the absolute top (`---` to `---`). Adding your explicit training datasets like `WithinUsAI/claude_mythos_distilled_25k` guarantees your model will show up under those dataset pages automatically, driving more discoverability.
2. **Clear Application Focus:** Highlighting "Refusal Removal" and "Enhanced Prose" flags precisely what local LLM enthusiasts look for in a Qwen 3.5 4B fine-tune. 

Base model

Qwen/Qwen3.5-4B-Base

Qwen/Qwen3.5-4B

(278)

LLM MODELS TRAINED, FINE-TUNED, MERGED and Refusal Removal BY (WITHIN US AI) • 24 items • Updated 12 days ago • 7

All models are “Alibaba Qwen 3.5” at core fine-tuned, merged & trained by (WithIn Us AI) • 3 items • Updated 12 days ago