Qwen3-Coder-Next-NVFP4 👁 Model Icon

👁 Validated Badge

Model Overview

• Model Architecture: Qwen3NextForCausalLM
  • Input: Text
  • Output: Text
• Model Optimizations:
  • Weight quantization: FP4
  • Activation quantization: FP4
• Release Date:
• Version: 1.0
• Model Developers:: Red Hat
• ModelCar Storage URI: oci://registry.redhat.io/rhai/modelcar-qwen3-coder-next-nvfp4:3.0
• Validated on vLLM: 0.14.1
• Validated on RHAIIS: 3.4 EA1
• Validated on RHOAI: 3.4 EA1

Quantized version of Qwen/Qwen3-Coder-Next.

Model Optimizations

This model was obtained by quantizing the weights and activations of Qwen/Qwen3-Coder-Next to FP4 data type. This optimization reduces the number of bits per parameter from 16 to 4, reducing the disk size and GPU memory requirements by approximately 75%. Only the weights and activations of the linear operators within transformers blocks of the language model are quantized.

Deployment

Use with vLLM

1. Initialize vLLM server:

vllm serve inference-optimization/Qwen3-Coder-Next-NVFP4 --port 8000 --tensor-parallel-size 2 --enable-auto-tool-choice --tool-call-parser qwen3_coder
 

2. Send requests to the server:

# Your tool implementation
def square_the_number(num: float) -> dict:
 return num ** 2

# Define Tools
tools=[
 {
 "type":"function",
 "function":{
 "name": "square_the_number",
 "description": "output the square of the number.",
 "parameters": {
 "type": "object",
 "required": ["input_num"],
 "properties": {
 'input_num': {
 'type': 'number', 
 'description': 'input_num is a number that will be squared'
 }
 },
 }
 }
 }
]

from openai import OpenAI
# Define LLM
client = OpenAI(
 # Use a custom endpoint compatible with OpenAI API
 base_url='http://localhost:8000/v1', # api_base
 api_key="EMPTY"
)
 
messages = [{'role': 'user', 'content': 'square the number 1024'}]

completion = client.chat.completions.create(
 messages=messages,
 model="RedHatAI/Qwen3-Coder-Next-NVFP4",
 max_tokens=65536,
 tools=tools,
)

print(completion.choices[0])

Creation

This model was quantized using the llm-compressor library as shown below.

Evaluation

The model was evaluated on the OpenLLM leaderboard task, using lm-evaluation-harness. vLLM was used for all evaluations.

Accuracy

Base model

Qwen/Qwen3-Coder-Next

(96)

March 2026 Collection of third-party generative AI models validated by Red Hat AI for use across the Red Hat AI Product Portfolio. • 5 items • Updated Apr 30 • 4