“An agent is an LLM and a harness”: What Nvidia really thinks about OpenClaw

How much of Nvidia is reflected by their visionary CEO, Jenson Huang? With his praise and later support of OpenClaw, Huang took a big step beyond the corporate boundary to embrace the “bad boy” of the agent world. Where exactly does Nvidia fit in here?

The New Stack put that question to Nader Khalil, Director of Developer Technologies at Nvidia, as well as how Nvidia is working with developers on agentic AI projects.

Khalil, co-founder of Brev.dev, found himself acquired by Nvidia about two years ago. His company helped startups access Nvidia AI chipsets. He is still excited by the possibilities of AI, and his energy is proof that Nvidia is enjoying the moment. Khalil was expansive, showing a startup’s keenness for the pace of change around them.

Before anything else, Khalil defines what he believes an agent is. “I have some slides,” he threatens. But these are more to organize his thoughts on an oft-asked question- not an attempt to lecture.

“An agent is an LLM and a harness… Each loop should take us closer to our goal.”

“An agent is an LLM and a harness. And if you think about that, it involves two things. It involves the loop and the LLM. And obviously you don’t want each loop to do the same thing. You want to leverage the results from the LLM. That might include reasoning on new tools to use. Each loop should take us closer to our goal.”

Nader praises the early OpenAI initiative. “So ChatGPT innovated outside of the model. It was not just a great model they made; they also added prompts. There was a system prompt and then the user prompt; there was multimodal, and suddenly that felt really good as a way for me to use the LLM. Every user could benefit from a system prompt that OpenAI had written while you were using your individual prompt.” Khalil continues, “Then they added memory.”

“Suddenly my assistant became really useful because it remembers things about me. ChatGPT knows that I really like to barbecue. So when I ask a question, it remembers what my smoker is,” recalls Khalil. “The thing that I was missing was files.”

Of course, the story continues through Cursor to Claude. “But this is the harness. Everything here is the harness,” he says.

Khalil moves on to how Nvidia works today. “The way to get your product into this rapidly growing market is with skills. Hence the CUDA X library.”

These are the implementations of use cases that target GPU acceleration, usually for compute-intensive applications.

“And so we look at every product we build now, it needs to have a skill because you need to cater to this growing audience,” he says.

This is how Nvidia first works with in-house experts, and connects to their edge hardware.

Supporting OpenClaw

Khalil was happy with the wording that Nvidia are “supporting” OpenClaw. “We’re just squarely in the community”, agrees Khalil. “We do this by the way, through a lot of projects that are very important in the open source ecosystem.”

But OpenClaw is not just any project and could be considered quite a risk to associate with. “We have a couple of developers at the company that contribute to OpenClaw full time.” Pushed on the nature of the relationship a little more, Khalil offers, “I think we just try to contribute wherever we can. I think what’s very clear is that harnesses had a moment, right?”

“We have a couple of developers at the company that contribute to OpenClaw full time.”

It has been quite a moment. “There is a lot of change happening right now, and we’re really thankful to [Peter Steinberger], OpenClaw, and the community for creating this moment around agents and harnesses. We of course want to contribute.”

Related to this, the OpenClaw project currently has many unresolved pull requests (PRs). In fact, there were rumors that new PRs are no longer accepted at all.

“We saw Peter tweeting about some of the issues they had, and we just rolled up our sleeves and were eager to help. They bless us by allowing our contributions.”

“You know,” says Khalil, “We saw Peter tweeting about some of the issues they had, and we just rolled up our sleeves and were eager to help. They bless us by allowing our contributions.”

Khalil reflects on things a little more: “You know the cardinal rule of code. It is easier to write than it is to read.”

And at over 800,000 lines of code, this must be true. Khalil continues, “It is easier not to have to process this complicated codebase, but every successful project right now has the same issue. It is easier to enlist many agents to help write code and build these PRs. The bottleneck is in merging the PRs through.” As well as dealing with the fallacies.

“OpenClaw was a major change for the industry. It was a huge moment, and everyone’s eyes are on it. It got more stars than Linux in months. Developers care deeply about the project because it was influential, and so I think you’re gonna see a mountain of PRs, right?”

“It got more stars than Linux in months… so I think you’re gonna see a mountain of PRs, right?”

Their attitude to OpenClaw is clearly to accept its problems, like that raucous friend who seems to wind up in police custody after a wild party, but is good at heart.

Blueprints and microwaves

Hermes is one of the newer projects in the wake of OpenClaw (like NanoClaw) that wants to bottle the lightning but in a safer way. Nvidia is also embracing it, but Khalil backs up to explain how Nvidia looks at projects in general.

“So, NemoClaw is our blueprint. When we see amazing harnesses, we try to figure out how we can help enterprises adopt them. Consumers sometimes want the security to run any agent; then there’s the model and harness. Then there are the skills, right. You have to give it access to your terminal.’

The term “blueprint” takes on a bit more formality in Nvidia, meaning the structure for building AI agents and systems. And of course Khalil needs to show these working with the Nemotron model and other Nvidia solutions.

“There’s a blueprint for Hermes and a blueprint for OpenClaw”. It sets up the runtime, enables the policies if there’s a local GPU, and runs the model.

Working with agents in the enterprise is seen as a significant risk. “There are a bunch of camps,” says Khalil. “There are teams within enterprises who are more worried. We have a project called OpenShell that is our security runtime and we’ll work with.”

“Our goal is to create the tooling that’s needed in the ecosystem. Developers in industry and enterprises have actually been adopting agents. And we have been building for this audience. One way to do so is to build a specialized agent or a sub-agent. “

So Nvidia doesn’t offer a big takeover solution, but fits in with where teams already are.

Your microwave, your agent

“The way to think about it is like when you use a microwave that you haven’t used before, you have to press a lot of buttons or spend time figuring it out. But when it’s your microwave at home, you just go ‘Boop, boop. Done.’ Right?”

“So every industry in enterprise will be building these specialized agents, and many already have. Nvidia is already working with CrowdStrike and Cadence, Palantir, among many others.”

The future will be agents

Khalil believes a lot of the concern over long-running agents is slowly petering out. Which leads to the final question: is Nvidia looking to stay in the open sea where there may be dragons, or become a calm port for developers to work in?

“So our approach is: Who can we help and how?” Khalil shows no fear, or lack of sea legs. “The inflection point happened months ago, so we ask what can we do to usher in all of this technology.” Here, Khalil ties his — and, to a degree, Nvidia’s — future to green-field developers.

“There are gonna be some people quick to adapt. And some people that aren’t; and what we’re noticing, if you look at the adoption curve, many people have yet to experience this. So there’s much work in helping make sure that we deliver this safely.”