 |
VOOZH | about |
|
VOOZH | about |
We’re so glad you’re here. You can expect all the best TNS content to arrive Monday through Friday to keep you on top of the news and at the top of your game.
Check your inbox for a confirmation email where you can adjust your preferences and even join additional groups.
Follow TNS on your favorite social media networks.
Become a TNS follower on LinkedIn.
Check out the latest featured and trending stories while you wait for your first TNS newsletter.
For this episode of The New Stack Makers, we speak with Matthew O’Riordan, CEO and co-founder of Ably.
Ably’s platform handles trillions of transactions a month for customers like HubSpot and Intercom. Originally built for human collaboration, its technology now also nicely enables long-running AI agents.
“We’re just very well suited to solving this communication layer between agents and humans.”
In this episode, we discuss why HTTP starts to break for long-running agents, how a “durable session” layer is emerging in AI infrastructure, and why Ably wants developers to stop thinking about pub/sub.
About 18 months ago, O’Riordan says, investors and his own team started pressing him for an AI story, but he initially resisted. Ably wasn’t an AI company, after all. It was an infrastructure play built for human collaboration with a focus on presence, ordering, state, and reconnection.
Then agents came along. Once those became long-running processes that reason and call tools, they also needed to stay alive for hours
“This is what we’ve been building for humans to communicate. And agents are becoming more human-like. […] I’d love to say we had the vision, and that’s why we built it all, but it’s not,” O’Riordan says. “We’re just very well suited to solving this communication layer between agents and humans.”
Most AI applications start on HTTP. It’s the default for developers on frameworks like LangGraph, and as even Ably would concede, it’s just fine for short, one-shot completions from chatbots. But a long-running agent that calls dozens of tool calls over multiple reasoning steps creates problems that a standard request/response flow wasn’t designed for. The system needs to handle connection drops, users switching tabs or devices, or interruptions by users wanting to interrupt the agent mid-stream.
HTTP isn’t the wrong transport tool, O’Riordan says. “It is exactly what you need to get up and running.” But expectations have shifted because, as he puts it, “we’re all engaging with ChatGPT and Claude.”
“What most people are doing is they click between tabs and they just expect the experience,” he says. “You ask it something, it takes a while, you go away, you come back five minutes later to a different tab, and you expect everything to just be in sync.”
The vocabulary developers have started using for this layer is “durable sessions.” O’Riordan credits EMQX, the MQTT broker, with originating the term and ElectricSQL with popularizing it for AI. He prefers it over “durable streams” because streams are only part of what’s needed. A durable session also covers presence, shared state, storage to rehydrate after crashes, and push notifications for offline users.
To support this AI use case, Ably extended its primitives, O’Riordan says. Token streaming requires moving from immutable to mutable messages, so that a client reconnecting mid-stream can request the current state rather than replaying every token. The company also introduced “live objects,” collaborative storage that keeps an agent and a human in sync on shared state, even when the agent has reasoned for so long that the user has moved on.
The other constraint O’Riordan kept hearing from developers was that any new transport layer had to be effectively invisible. The goal, he says, was a layer developers could drop into their existing infrastructure without changing their overall stack and design patterns.
Ably’s answer is AI Transport, which is now in its second iteration. It leaves the client-to-agent path on HTTP, where developers already are, and moves only the response path to a durable session. The agent essentially acknowledges the request over HTTP and then streams back over the session.
Frameworks are baking in plug-in points for exactly this layer. Vercel and TanStack both already expose transport abstractions for durable session providers. Ably’s Vercel AI SDK plug-in adds multi-device session resumption, interruption support, and persistent storage.
O’Riordan doesn’t want developers thinking in old-school pub/sub terms either. “I feel strongly that people should not have to think about pub/sub.”
