Microsoft MAI-Thinking-1 is the company's highest-performing reasoning model to date. It scores 97.0% on AIME 2025 and matches Claude Opus 4.8 on SWE-Bench Pro. The catch: it is not open-weight. Access is through Microsoft Foundry (private preview as of June 2026; broader access has not been officially confirmed). There are no downloadable weights and no HuggingFace release.

This guide covers what MAI-Thinking-1 actually is, how to access it via the Foundry API, and which open-weight reasoning models you can self-host on GPU cloud when you need full inference control. For context on what reasoning tokens actually cost at scale, see the reasoning model inference cost guide.

What Is Microsoft MAI-Thinking-1

MAI-Thinking-1 is Microsoft's first reasoning model trained entirely in-house. Unlike earlier Microsoft open releases (Phi series, MAI-DS-R1, which used distillation), this model was trained end-to-end without borrowing outputs from third-party models. That gives it a clean training lineage for organizations with strict enterprise licensing requirements.

Architecture: sparse Mixture-of-Experts. The 35B active parameter figure reflects how MoE routing works: each token activates only a subset of experts, so only 35B parameters compute per forward pass. The total parameter count, including all expert weights that must reside in VRAM, is approximately 1T. At FP8 (1 byte per parameter), that is roughly 1 TB of weight storage before KV cache or framework overhead. This is why Microsoft distributes MAI-Thinking-1 as an API service rather than open weights.

Key specs:

For a comparable open-model deployment walkthrough, see deploying GPT-OSS on GPU cloud.

Benchmark Results: MAI-Thinking-1 vs Open Reasoning Models

Published scores from Microsoft's Build 2026 announcement:

The gap between MAI-Thinking-1 (97.0% AIME 2025) and the best self-hostable models (~72.5% for Nemotron Ultra) is large on pure math reasoning. For teams that can accept API latency and do not require on-premises deployment, MAI-Thinking-1 via Foundry is significantly stronger on reasoning tasks.

For a broader comparison across the reasoning model landscape, see DeepSeek vs Llama 4 vs Qwen 3.

Why MAI-Thinking-1 Is Not Self-Hostable

The ~1T total parameter count is the core constraint. At FP8, weights alone require approximately 1 TB of VRAM. With 15% framework overhead, the minimum before KV cache is roughly 1.15 TB.

To put that in context:

MAI-Thinking-1 at 1T parameters exceeds what a single 8x H200 SXM5 node (1,128 GB total VRAM) could serve without multi-node NVLink fabric. Microsoft has not announced a timeline for open-weight release. If that changes, the self-hosting math would need to be redone from the actual confirmed parameter count and architecture details.

For the theory behind why MoE total parameter count drives GPU selection, see the MoE inference optimization guide.

Accessing MAI-Thinking-1 via Microsoft Foundry

Step 1: Request Access

Go to microsoft.ai/models/mai-thinking-1 and apply for private preview access through Microsoft Foundry. Access is currently in private preview; broader availability has not been officially confirmed from the Build 2026 coverage. Enterprise access through Azure AI is expected to follow any future public release.

Step 2: Call the API

MAI-Thinking-1 is served via a Chat Completions-compatible endpoint. Once you have Foundry credentials:

import openai

client = openai.OpenAI(
 base_url="<your-microsoft-foundry-endpoint>",
 api_key="<your-foundry-api-key>"
)

response = client.chat.completions.create(
 model="mai-thinking-1",
 messages=[
 {
 "role": "user",
 "content": "Solve: A train travels 120 miles at 60 mph, then 80 miles at 40 mph. What is the average speed for the entire trip?"
 }
 ],
 max_tokens=4096,
 temperature=0.1
)

print(response.choices[0].message.content)

The model generates an extended reasoning trace before the final answer. If responses do not include thinking tokens, check the model card for any required system prompt or template format once it becomes available.

What About Fine-Tuning?

As of Build 2026, Microsoft has not announced fine-tuning access to MAI-Thinking-1 weights. Access is inference-only via the Foundry API. If fine-tuning support is added, Microsoft will announce it through official Foundry documentation.

Self-Hostable Open Reasoning Alternatives

For teams that need full inference control, on-premises deployment, or data residency requirements, the current top open-weight reasoning models are DeepSeek R2 and Nemotron Ultra 253B.

DeepSeek R2

DeepSeek R2 is a large sparse MoE model expected to have open weights available for self-hosting. At FP8 with selective INT4 quantization on some layers, it fits on a 4x H200 SXM5 node (564 GB combined VRAM). It is the closest open alternative to MAI-Thinking-1 in terms of reasoning tier.

For the full deployment setup, see the DeepSeek R2 deployment guide.

Quick start for 4x H200 SXM5:

vllm serve /models/deepseek-r2 \
 --dtype fp8 \
 --quantization compressed_tensors \
 --tensor-parallel-size 4 \
 --max-model-len 65536 \
 --kv-cache-dtype fp8_e5m2 \
 --gpu-memory-utilization 0.90 \
 --enable-expert-parallel \
 --trust-remote-code \
 --port 8000

Important: --enable-expert-parallel requires 2+ GPUs. For a single-GPU setup, drop --tensor-parallel-size to 1 and also remove --enable-expert-parallel. Running expert parallelism on a single GPU causes an error. For the full vLLM setup on Spheron including tensor parallel configuration options, see the vLLM server guide.

Nemotron Ultra 253B

NVIDIA's Nemotron Ultra 253B is a dense transformer (not MoE) with 253B parameters. Scores: 76.01% GPQA Diamond, 72.5% AIME 2025. As a dense model, it does not have MoE routing complexity and is simpler to serve.

At FP8, 253B parameters require about 253 GB of VRAM. A 2x H200 SXM5 node (282 GB) fits the weights with some KV cache headroom for practical context lengths.

For the full setup, see the Nemotron Ultra 253B deployment guide.

GPU Hardware and Pricing for Open Alternatives

Pricing fetched from the Spheron API on 16 Jun 2026.

H200 SXM5 for DeepSeek R2

DeepSeek R2 needs 4x H200 SXM5 at FP8 with selective INT4 quantization minimum. On-demand pricing at $4.96/hr per GPU:

H200 SXM5 on Spheron includes NVLink by default on SXM5 configurations, which reduces all-reduce latency per layer compared to PCIe interconnect. This matters for tensor parallel efficiency at TP=4.

B200 SXM6 for Smaller Open Models

Nemotron Ultra 253B at FP8 fits on 2x B200 SXM6 (384 GB combined) with good KV cache budget. A single B200 SXM6 (192 GB) is only viable via MXFP4 (Blackwell FP4, ~130 GB), not FP8, since 253B at FP8 requires ~253 GB of weight storage alone.

B200 SXM6 on Spheron also supports MXFP4 (0.5 bytes per parameter), which is Blackwell-only and can halve the weight footprint again, enabling Nemotron Ultra 253B on a single B200 at ~130 GB.

H100 SXM5 for Cost-Optimized Deployment

At $3.98/hr per GPU, 4x H100 SXM5 ($15.92/hr combined) is a cost-efficient option for Nemotron Ultra 253B at FP8 (320 GB combined VRAM covers 253B FP8 weights with headroom). DeepSeek R2 at FP8 with selective INT4 quantization needs 8x H100 SXM5 minimum.

Pricing fluctuates based on GPU availability. The prices above are based on 16 Jun 2026 and may have changed. Check current GPU pricing for live rates.

Cost Comparison: MAI-Thinking-1 API vs Self-Hosting an Open Alternative

This is a trade-off between capability and control. MAI-Thinking-1 at 97% AIME 2025 leads on reasoning accuracy, but it is API-only. The open alternatives score lower on benchmarks but offer full control.

For sustained high-volume inference where DeepSeek R2 or Nemotron Ultra benchmark scores are sufficient, self-hosting on Spheron typically beats per-token API pricing at roughly 100-300 million tokens per month of sustained output, depending on what Microsoft prices Foundry access at.

When to Use MAI-Thinking-1 API vs Self-Host an Open Alternative

MAI-Thinking-1 sets a new bar for reasoning model accuracy at 97.0% AIME 2025, but it runs as an API service for now. For self-hosted inference with open weights, H200 SXM5 instances on Spheron support both DeepSeek R2 and Nemotron Ultra 253B with per-minute billing and no contracts.

B200 SXM6 on Spheron | View all GPU pricing

Start deploying now