VOOZH about

URL: https://thenewstack.io/databricks-is-rebuilding-the-data-stack-for-ai-agents/

⇱ Databricks wants to merge the two databases every company runs - The New Stack


TNS
SUBSCRIBE
Join our community of software engineering leaders and aspirational developers. Always stay in-the-know by getting the most important news and exclusive content delivered fresh to your inbox to learn more about at-scale software development.
REQUIRED
It seems that you've previously unsubscribed from our newsletter in the past. Click the button below to open the re-subscribe form in a new tab. When you're done, simply close that tab and continue with this form to complete your subscription.
The New Stack does not sell your information or share it with unaffiliated third parties. By continuing, you agree to our Terms of Use and Privacy Policy.
Welcome and thank you for joining The New Stack community!
Please answer a few simple questions to help us deliver the news and resources you are interested in.
REQUIRED
REQUIRED
REQUIRED
REQUIRED
REQUIRED
Great to meet you!
Tell us a bit about your job so we can cover the topics you find most relevant.
REQUIRED
REQUIRED
REQUIRED
REQUIRED
REQUIRED
Welcome!

We’re so glad you’re here. You can expect all the best TNS content to arrive Monday through Friday to keep you on top of the news and at the top of your game.

What’s next?

Check your inbox for a confirmation email where you can adjust your preferences and even join additional groups.

Follow TNS on your favorite social media networks.

Become a TNS follower on LinkedIn.

Check out the latest featured and trending stories while you wait for your first TNS newsletter.

PREV
1 of 2
NEXT
VOXPOP
As a JavaScript developer, what non-React tools do you use most often?
Angular
0%
Astro
0%
Svelte
0%
Vue.js
0%
Other
0%
I only use React
0%
I don't use JavaScript
0%
Thanks for your opinion! Subscribe below to get the final results, published exclusively in our TNS Update newsletter:
NEW! Try Stackie AI
From clobbered drafts to real-time sync
Apr 14th 2026 10:00am, by David Moore
TypeScript 6.0 RC arrives as a bridge to a faster future
Mar 14th 2026 9:00am, by Darryl K. Taft
Mastra empowers web devs to build AI agents in TypeScript
Jan 28th 2026 11:00am, by Loraine Lawson
2026-06-16 17:30:00
Databricks wants to merge the two databases every company runs
AI / AI Agents / AI Infrastructure

Databricks wants to merge the two databases every company runs

Its new LTAP architecture is a bet that AI agents, not people, are now the database's primary users.
Jun 16th, 2026 5:30pm by Frederic Lardinois
👁 Featued image for: Databricks wants to merge the two databases every company runs

Databricks wants to erase the divide between the databases that run a business and the systems that analyze it. At its Data + AI Summit in San Francisco on Tuesday, the company introduced an architecture it calls Lake Transactional/Analytical Processing, or LTAP, built to collapse that split for AI agents.

Databricks started going down this path a while ago but made it concrete when it bought the serverless Postgres startup Neon and, later, Mooncake Labs in 2025. The bet here is that AI agents, not people, will become the primary users of the enterprise data stack, and that the infrastructure beneath them has to be rebuilt for them.

👁 Image
Credit: The New Stack.

A breakthrough 40 years in the making

“For decades, complicated data infrastructure was a tax that teams were forced to pay,” said Ali Ghodsi, co-founder and CEO of Databricks, in the announcement. “Then agents arrived. In a matter of months, organizations effectively doubled their workforce, just not with humans. Agents write code, make calls, and run loops at a pace human teams never could. The infrastructure that powered the last era of computing is now the bottleneck that no one can afford. LTAP removes it.”

LTAP, Ghodsi said in his conference keynote on Tuesday, is “a breakthrough the industry has been working on for 40 years. We think we finally pulled it off.”

👁 Image
Credit: The New Stack.

Historically, companies have had to run two kinds of databases. Online transactional processing systems handle the live operations of a business, like orders, payments, and inventory, in row-based formats tuned for fast writes. Online analytical processing systems then use what is essentially the same data for reporting and analysis in column-based formats specifically tuned for large scans. The two were kept apart for performance and reliability, and enterprises bridged them with ETL pipelines and replicas..

Databricks argues that agents need a different system because they can read live transactional data, reasoning over historical context, and act on both of them at once.

Earlier attempts to merge the two layers never quite worked, the company says, because hybrid transactional and analytical processing (HTAP) systems carried high costs and proprietary lock-in, while “zero-ETL” tools amounted to hidden change data capture, still leaving two copies of the data and the problem of data going stale.

👁 Image
Credit: The New Stack.

What is LTAP?

LTAP unifies transactional and analytical data in a single storage layer, governed once and stored in open formats on cloud object storage, while keeping separate compute engines for each kind of work.

The design builds directly on Lakebase, the Postgres-based operational database Databricks introduced in June 2025, which the company describes as a “new category” that separates compute from storage and places the data in the lake in open formats.

Now, the company is extending Lakebase for what it calls business-critical workloads, adding native vector and full-text search, real-time event ingestion through Zerobus, part of its Lakeflow Connect ingestion service, and Git-style branching that lets an agent copy a database to experiment and then discard it.

“Agents love to just branch out and experiment with the data, try something else, and they want to do it quickly,” Ghodsi said. “They don’t want to wait ten minutes on a database to come up.”

👁 Image
Credit: The New Stack.

Lakehouse//RT

The second piece is Lakehouse//RT, a real-time analytics engine, powered by a vectorized engine Databricks calls Reyden, that runs directly on Delta and Iceberg tables in the lakehouse.

Companies have long stood up separate, specialized systems to get millisecond query speeds, duplicating data into a “serving layer” that sits alongside the lakehouse. Databricks says Lakehouse//RT removes that layer, delivering millisecond-level latency on lakehouse data with no extra copies, pipelines, or governance gaps.

Databricks stresses the engines high concurrency. Mehrshad Setayesh, SVP of engineering at PointClickCare, says Lakehouse//RT “ran more than a third faster on average than our prior warehouse on our healthcare dataset, with 10x faster queries,” and that it removed the company’s need for a dedicated real-time system alongside its lakehouse.

Mooncake and Neon to the rescue

LTAP’s main pitch is that a single copy of the data can be stored once in open formats without the need for complex data pipelines. The Lakebase architecture, the company wrote last year, shares one storage layer across transactional and analytical workloads “without moving or duplicating it.”

Lakebase’s analytical speed comes from Mooncake, the startup Databricks bought to accelerate it. Mooncake mirrors Postgres changes into the lakehouse in real time, which is how transactions and analytics run on the same fresh data.

“Postgres changes are mirrored in real time to the lakehouse,” the company wrote when it announced the deal. Mirroring produces a second, columnar copy of the data, which is what makes the analytical queries fast.

Security, governance, auditing, and high availability, the company wrote, “only need to be implemented and managed once, on a single open foundation.”

The branching is the feature of this that was built specifically for agents — and a feature that is core to Neon, too. Because the data sits on object storage, an agent can fork a full database, test against it, and discard it, the way it would a Git branch. Databricks says even petabyte-scale databases can be copied in seconds, while on a traditional database, provisioning an instance takes minutes or hours and cloning production risks taking it down.

As Ghodsi noted in his keynote, agents love Postgres, but they do need better tools to work with them — and maybe better databases, too. “In the next 12 months, we’re going to see more software written than ever in the history of mankind,” he said. “All that software that your organizations are going to write using LLMs and coding tools need the database behind the scenes.”

What else is new?

LTAP was only one part of the company’s three-hour keynote. Like so many other enterprise vendors, Databricks is also thinking about how to get agent sprawl — and cost — under control. Databricks’ answer is Unity AI Gateway, a single control point for every model, agent, MCP server, and skill running in an organization. Among other features, it offers spending dashboards, budgets that can be set per team or per user, rate limits, and single sign-on across MCP servers.

The company also introduced Genie One, a general-purpose agent for business teams, fed by Genie Ontology, a new layer that builds a ranked graph of a company’s data with a PageRank-style algorithm it calls OntoRank.

Ghodsi also highlighted OpenSharing, a new protocol for sharing data, models, and agent skills across platforms (you may remember its predecessor Delta Sharing, but it is now a project under the Linux Foundation).

Databricks also debuted CustomerLake, a customer data platform aimed at marketing teams and announced an agreement to acquire Panther, a Python-based security company, to feed its Lakewatch security information and event management service.

Databricks’ moat?

It’s the data layer, though, and its data science history, where Databricks can really differentiate. At this point, it feels like every enterprise vendor, no matter their expertise, is adding agent builders, agent orchestration and governance tools. Databricks can be a relatively neutral player in this space — something Ghodsi also stressed in a press conference after the keynote.

But the company is also seemingly aware that while many of the other enterprise SaaS vendors can use their expertise and existing customer data that can feed AI agents as a moat, Databricks functions as more of a utility layer. It’s maybe no surprise then, that it is launching an industry-specific product like CustomerLake for the marketing industry that adds a pre-made product layer on data its customers already store on its platform.

TRENDING STORIES
Before joining The New Stack as its senior editor for AI, Frederic was the enterprise editor at TechCrunch, where he covered everything from the rise of the cloud and the earliest days of Kubernetes to the advent of quantum computing....
Read more from Frederic Lardinois
SHARE THIS STORY
TRENDING STORIES
TNS owner Insight Partners is an investor in: Databricks.
SHARE THIS STORY
TRENDING STORIES
TNS DAILY NEWSLETTER Receive a free roundup of the most recent TNS articles in your inbox each day.
The New Stack does not sell your information or share it with unaffiliated third parties. By continuing, you agree to our Terms of Use and Privacy Policy.