Google Cloud Therapist on Bringing AI to Cloud Native Infrastructure

At Google Cloud Next, I sat down with Google’s Bobby Allen. While he describes himself as a “cloud therapist,” his official title is Group Product Manager for Google Kubernetes Engine. And while Google’s current focus seems to be almost 100% on AI, it’s these cloud native technologies like Kubernetes that are at the core of what makes these systems work.

GKE As An AI Platform

“GKE is a phenomenal platform for AI in so many ways, directly and indirectly,” Allen said. “So let’s hit the indirectly first. Indirectly, GKE powers things like Vertex AI. So, Vertex AI runs directly on GKE, which to me is part of our superpower, because when you think about us drinking our own champagne, if you will, we have customers like DeepMind and like Vertex AI that leverage, directly or indirectly, our technology and our expertise.”

Those groups run their model training and inference on top of GKE, as do plenty of gaming, health care, financial services, life sciences and other businesses.

“A lot of digital natives and startups really kind of trust GKE to be the way that they orchestrate a lot of the newer types of resources they need,” he said.

This current focus on AI is also changing how the team thinks about the future of GKE. Over time, Allen argued, Google went from specialized hardware to commoditized hardware and is now back to specialized hardware with the advent of AI accelerators. For all of this, he said, you need an orchestrator that can help manage these resources.

“AI is special, but it’s not,” he said. “I’m kind of contradicting myself, because we see AI as another type of modern workload. So if you think about things that are cloud native, that are scalable, that need to be elastic — AI has a lot of those characteristics, but it also has some other distinguishing characteristics, like leveraging hardware accelerators, like GPUs and TPUs, sitting at the intersection of high availability, HPC and a lot of innovation. GKE underpins all that well, because a lot of the things that GKE naturally does is literally built for a lot of the types of things we want to do.”

So many of the primitives built into Kubernetes, he said, are tailor-made for those users who want to tune their systems for AI workloads.

Thanks to a lot of this early work that went into Kubernetes, engineers solved issues around scaling those platforms and securing them, Allen noted. And because the range of users today is also extremely broad, services like GKE have the tools built in to handle virtually any workload, with adjacent systems for storage, networking databases, etc. tuned for them.

“GKE is the most control with the least technical debt, in my opinion,” he said. “Because it’s future-proofed in so many ways. Because we know it can scale. We know it can grow. We know the commitment that we have in the open source community, where Google continues to invest at a very aggressive rate.”

GKE as Model Router

Looking ahead, Allen and his team are thinking about how to better integrate AI models into the overall workflow. Most enterprises, he argued, will not train their own models but they will want model choice.

“On that model as a service offering, GKE will support the ability to route you to the model based on the purpose, not necessarily the name,” he said. “So maybe you want Meta. Maybe you want Gemma or Gemini. Maybe you just want to say: Hit this inference endpoint, if you’re a developer, for image generation, and I don’t care what’s underneath, right? GKE will allow you to set up that scaffolding to give people the output they want, without knowing the brand or the version of the model.”