Why AI infrastructure must evolve for agents — Akshat Bubna, CTO Modal

Akshat Bubna, CTO of Modal cloud platform, returned to the Latent Space podcast after two years to discuss why the current cloud infrastructure paradigm is ill-suited for the era of AI-agents.

From Stateless Functions to Long-Lived Agents

Cloud infrastructure for the last 15 years has been built around the paradigm of short functions: a request arrives, the server executes a task in milliseconds or seconds, returns a result. This model works perfectly for web applications, microservices, and data processing.

AI-agents work completely differently. Instead of a one-off API request, an agent launches and can work for hours or days, making decisions, calling tools, analyzing results, and adapting to new information. It critically needs to maintain state, remember context, coordinate long-term plans.

• An agent does not simply call an LLM once — it iterates
• Requires a long-lived process that is not killed by the server
• Must remember the entire context between steps
• Interacts with multiple tools, APIs, and services in parallel

Why Serverless Architecture Does Not Fit

When you run a function on AWS Lambda or Google Cloud Functions, the cloud can kill the container at any moment after execution. State is not preserved. The next function call — a blank slate.

For agents, this means each iteration loses the context of previous steps. The agent cannot work long-term. Scaling becomes a problem: you run many independent functions, instead of managing a single coordinated long-lived process. Resources compete. Reliability suffers.

Current cloud platforms simply were not designed for this.

What Agent Infrastructure Requires

A platform supporting agents must provide:

• Long-lived containers that do not lose state
• Guaranteed context preservation between iterations
• Simple scaling of multiple concurrent agents
• Built-in coordination and fault tolerance
• Easy access to GPU/CPU without infrastructure management

Modal positions itself as such a platform — a cloud that first asks "what computations does the agent need?" instead of "what functions do I scale?".

What This Means

Cloud infrastructure enters its third evolution cycle. The first was about simply moving servers to the cloud. The second — microservices and function scaling. The third — is the era of long-lived AI-agents, which require a new kind of architecture. Platforms like Modal show that infrastructure must evolve together with the types of applications developers build.