Confluent Provides a Real-Time Context Engine for AI

NEW ORLEANS — Confluent Inc. wants to help organizations bring their AI efforts up to speed so that AI analysis can be done and actions can be taken as events are unfolding.

Confluent is, of course, all about real-time processing, with its Confluent Platform and Confluent Cloud. They are both based on the open source Kafka event streaming platform and Kafka’s frequent companion, the Apache Flink data processing platform.

With enterprises worldwide pouring $30-$40 million into generative AI (GenAI) efforts with thus far little to show for it, Confluent sees the missing ingredient as being timeliness.

“Off-the-shelf models are powerful, but without the continuous flow of data, they can’t deliver decisions that are timely and uniquely valuable to a business,” said Confluent CEO and co-founder Jay Kreps, in a statement. “That’s where data streaming becomes essential.”

Running AI is a bit like driving a car, further advised Sean Falconer, Confluent’s head of AI, in an interview with TNS.

You need the historical information: everything you need to know about how to drive a car that you’ve learned over the years. But you also need a constant stream of new data about the road you’re driving on itself, he said.

Confluent wants to help your AI tap into that real-time data.

Confluent Intelligence

At the company’s annual Current conference this week in New Orleans, Confluent revealed a new set of features for its streaming platform, called Confluent Intelligence.

With today’s AI systems, “There’s a lot of demand to engineer that data up front to be able to serve whatever tasks that you want to perform,” Falconer said. “You’re essentially steering the logic of the systems through that data and context.”

Confluent Intelligence will provide Confluent users with a set of tools to easily derive and package data from their streaming platforms so it can easily be used in AI operations.

One component is a real-time context engine, a fully managed service that streams materialized views to any AI agent or application. Kafka provides the data, and Flink joins data sets into a new common topic.

Available in early access release, the context engine can communicate this data to either internal AI apps built on Kafka/Flink or to external applications by way of the Model Context Protocol (MCP).

The developer doesn’t need to know about working with streaming data specifically; they can get a context snapshot, or materialized view, directly from MCP.

The Need for Real-Time Context Data

Typically, context data is provided to a large language model (LLM) through the user prompt, perhaps calling in the services of a number of MCP-based agents. This approach is expensive in terms of spending tokens to run multiple calls for data.

“The biggest problem is that a lot of those API access points aren’t really designed for contextualizing data for AI systems. They’re designed for consuming API endpoints. So you end up with a large, exploding token cost in a lot of times serving data that doesn’t necessarily make sense to AI,” Falconer said.

Rarely does all that data reside in one database. So in some cases, an analyst may build a derived data set that joins the data needed and stores it in a database. But then that data is stale, Falconer explained.

Many AI operations require real-time or near-real-time access to data. Think of a bot that helps a customer rebook a flight. Or a car sharing app that requires the current status of both customer and driver locations.

“Ultimately, what you need is a purpose-built data set that represents the current state of the business. It has historical reference but also [has] what’s happening in the moment,” Falconer said.

For an organization, another advantage of deriving all the context data from the existing data processing platform is that it can apply all its governance, compliance and security rules to the data that goes to the AI, without building these safeguards again into another platform.

Use the Data Platform

Users can build event-driven agents that run natively on Flink, through Streaming Agents, a new development environment that Confluent introduced in August. The idea is to put AI reasoning — observe, decide and act — directly at the data processing level.

With this feature, agents become a core feature of Flink.

“So I can essentially define an agent, define its roles, what tools it has available, the models that I want to use, and then I have it act directly against the real-time stream,” Falconer said.

This is a new kind of agent, not waiting on user input, but rather one able to take action when it sees events happening as they unfold.

Anthropic’s Claude will serve as the default LLM for Streaming Agents.

👁 Image

To further help with AI, the company has also introduced a set of built-in machine learning (ML) functions, written in Flink SQL, and covering tasks such as anomaly detection, forecasting, model inference and real-time visualization. Currently, this set of functions is available on Confluent Cloud.

Stay tuned, as TNS will be at Current this week to find out more details about Confluent Intelligence.