 |
VOOZH | about |
|
VOOZH | about |
We’re so glad you’re here. You can expect all the best TNS content to arrive Monday through Friday to keep you on top of the news and at the top of your game.
Check your inbox for a confirmation email where you can adjust your preferences and even join additional groups.
Follow TNS on your favorite social media networks.
Become a TNS follower on LinkedIn.
Check out the latest featured and trending stories while you wait for your first TNS newsletter.
Last week, Inception launched Mercury 2, a large language model based on diffusion rather than the autoregressive approach used by every major AI lab. And on this week’s episode of The New Stack Agents, Inception CEO and co-founder Stefano Ermon explains how the diffusion model of generative AI could reshape how we build AI applications.
But first, some background: Traditional LLMs generate text one token at a time, left to right, a system that Ermon calls “fancy autocomplete.” Meanwhile, diffusion models work differently: They start with a rough answer and refine it in parallel, much like image models like Stable Diffusion crystallize images from noise. The result is a model that produces over 1,000 tokens per second — five to ten times faster than speed-optimized models from OpenAI, Anthropic, and Google, according to Inception’s own testing.
“What we’re seeing is that our Mercury 2 model, which is a reasoning model, is actually able to match the quality of these speed optimized models from [frontier labs OpenAI, Anthropic, Meta, and Google], while being five to 10x faster in terms of, like, the end to end latency, how long you need to wait before it gives you an answer,” Ermon tells TNS Senior Editor for AI Frederic Lardinois.
Autoregressive models are slower because they move data through memory instead of doing math. Diffusion models focus on parallel computation, which is what GPUs were built for. And GPU giant Nvidia, an investor in Inception, is helping optimize the serving engine, Ermon says.
Ermon, who pioneered diffusion models for images at Stanford and published the foundational text diffusion paper that won Best Paper at ICML 2024, is candid about the trade-offs: Mercury 2 matches the quality of Claude Haiku and Google Flash-class models, not Claude Opus or OpenAI GPT-4. But he argues the economics will win out as models scale. Reinforcement learning, the technique behind today’s reasoning models, is also naturally faster on diffusion architectures since its bottleneck is inference.
Inception is the only company shipping a production diffusion LLM — Google’s text diffusion model is still “experimental.” Mercury 2 is now available via an OpenAI-compatible API, with AWS Bedrock integration coming soon.
Listen to the full conversation on The New Stack Agents.
