Agents build things. Enterprises have to run them

I’ve never worked in the Kubernetes space, so KubeCon, its big event, is rarely top of mind, even when it’s in my backyard. Last week it attracted more than 10,000 attendees, yet a good friend still questioned its relevance in the age of AI. I’d mentioned KubeCon in a post and he texted back, “No one cares about the primitives anymore. Agents just build things!”

I found myself agreeing with him, but only for a moment. For those of us who live in AI, it’s easy to forget the reality of enterprise adoption. I’ve spent years warning against fashion-driven Kubernetes adoption, and I’m hardly nostalgic for developers spending their afternoons deciphering deployment configurations. If an agent can spare them that work, wonderful.

But there’s a rather large difference between no longer wanting to configure infrastructure and no longer needing infrastructure that works. For those who think Kubernetes’ obsolescence is just a prompt away, well, reality bites.

Indeed, many of the people embracing AI are precisely the people who understand why the primitives still matter. I posited this to Linux Foundation CEO Jim Zemlin, arguing, “AI changes how we create software faster than it eliminates the work of operating it.” “Yep,” he texted back, “Code is cheap [but] trust is expensive.” That’s a much better explanation of KubeCon’s relevance than nostalgia for YAML.

The AI crowd is already here

It’s not as if the Kubernetes crowd is allergic to AI. Far from it. As noted in a recent CNCF and SlashData report, 75% of cloud-native developers use AI coding tools, compared with 55% of their non-cloud-native peers. Of course, cloud-native developers aren’t synonymous with Kubernetes operators. Even so, it’s hard to find data to support the idea that the Kubernetes infrastructure community is stubbornly resisting AI. The two interests overlap.

Nor should we pretend enterprise agents are mostly imaginary. Zapier’s November 9 survey found that 71% of respondents’ organizations had agents in production. This may not be an entirely representative sample, given that it was 513 AI-experienced US managers at companies with at least 1,000 employees. And it may also not be the most interesting finding in the report. What drew my attention was that only 30% of leaders at organizations using agents said those agents completed more than 60% of assigned tasks without human intervention.

That’s cool. It’s real adoption. But it’s a long, long way from handing over the business for robots to run.

Indeed, both surveys challenge my friend’s cavalier “AI will do it all” response. Enterprises can enthusiastically use AI while insisting that their applications remain secure, recoverable, and affordable. This doesn’t make them slow. It shows maturity and a willingness to take responsibility beyond an impressive screen recording.

After all, an agent generating a service doesn’t settle which customer records it should access, how much capacity it deserves, or which other application gets priority when resources run short. Those decisions remain, even when software increasingly executes them.

We still have to pay for inference

Zemlin says the people at KubeCon were focused on security and getting more performance from less hardware and power. “That is not easy,” he adds.

Consider China Merchants Bank. In a September case study published by CNCF, the bank describes supporting AI training and inference across nearly 10,000 accelerator cards. Its problem wasn’t simply finding more hardware. Instead it was a matter of predictability and responsiveness: Training needs predictable capacity, and customer-facing inference needs to respond to changing demand. Keeping capacity available for peak traffic wastes money during quiet periods, while scaling too slowly leaves users waiting.

The bank combined Kubernetes with tools for scheduling, monitoring, scaling, and moving model data, with significant impact. It reported average accelerator compute utilization had risen from 35% to more than 60%, with inference cost per million tokens falling by over 60% for comparable models and services. Suddenly, not-so-cool Kubernetes became very cool indeed, because it helped to dramatically lower the cost of AI.

Apparently someone cared about the primitives.

That’s the part the “agents just build things” argument skips. An application can work perfectly well in a demonstration while consuming far more resources than it should. A deployment can succeed while leaving the next team to discover its permissions are too broad. Generating the configuration faster doesn’t establish that it’s the right configuration. Etc., etc.

As Isolvalent cofounder Thomas Graf told me in 2023, platform teams are increasingly expected to meet enterprise requirements “without disrupting and undoing the gains that have been made on agility and developer efficiency.” The job is to deliver the controls while preserving developers’ freedom to build. That remains a pretty good description of what agents need from a platform.

AI brings its own infrastructure woes

Newer workloads also require some newer answers. On November 5, Google announced Agent Substrate 1.0 and its general availability on Google Kubernetes Engine, alongside donations of Agent Substrate, gVisor, and OpenRL to CNCF. Look past the fancy language and the problems are familiar. Agents execute code, wait for model responses, retain state, and need limits on what they can access. Keeping dedicated compute running while an agent waits is wasteful; running generated code without adequate isolation is dangerous.

Google’s approach separates persistent agent state from reusable execution environments and enforces access through gateways. Google has long been on the cutting edge of AI. Despite its AI sexiness (or maybe because of it), Google clearly thinks infrastructure still matters.

Then there’s Nvidia. Despite its CEO’s recent problems, the company continues to demonstrate that AI is only as cool as it is boring (a theme I’ve addressed before). Hence, Nvidia contributed OpenShell to the CNCF Sandbox on November 10. Nvidia knows that such things benefit from community involvement. As Zemlin told me, OpenShell is “critical sandbox software that needs peer review and collective development.” The runtime is designed to enforce boundaries around agents’ access to data, secrets, and networks.

It’s a really important and highly consequential use of open source. This approach means that companies can share the work of testing and improving a security boundary instead of each trusting a fresh implementation because an agent produced it. Open code doesn’t guarantee safety; instead, it gives people a common place to scrutinize the controls.

When I interviewed Kubernetes creators Craig McLuckie and Joe Beda earlier this year, accountability was central to the conversation. Enterprises still own the outcome of work they delegate to agents. This is what that ownership looks like in practice: permissions enforced, state preserved, resources allocated, and failures contained. Even if an agent eventually handles most configuration and troubleshooting, the underlying platform still determines what it can accomplish. Stable interfaces, enforced permissions, and accurate feedback make automation possible. Giving a machine access to an undocumented collection of scripts doesn’t magically turn that collection into a reliable platform.

Give the agents a platform

By all means, let agents build things. But don’t do so foolishly. Give them a platform with enforced permissions, spending limits, and a way to undo a bad deployment. Asking an agent to generate the application and then reassure you that everything is fine isn’t much of an operating strategy. I’ve had enough experience with AI confidently making shocking errors to know that “trust but verify” has never been more needed.

That doesn’t mean more Kubernetes for every project. Sometimes a managed service that hides the infrastructure may be the better answer. My point is simply that we should reuse capabilities somebody knows how to operate rather than inventing another pile of infrastructure just because an agent can generate it.

That’s why KubeCon still matters. The work is to make shared infrastructure safer and cheaper to operate so the rest of us can get on with building. My friend is welcome to ignore the primitives. He just needs someone else to care about them.

By jawad