Choose Kubernetes for agents only after an isolation and GPU trial
The direct answer: choose the smallest demonstrably manageable path
Choose Kubernetes only if you can demonstrate for one specific agent workflow that scheduling, GPU serving, state, isolation, and fallback are manageable together. This is not a judgment that Kubernetes is unsafe or unsuitable; the supplied excerpts do not assess Kubernetes, gVisor, or specific network policies. They do, however, establish two relevant constraints. Agentic workloads consist of short reasoning-action loops with external calls and, according to AgentServe, require a balance of low latency, stable token emission, and throughput across multiple agent requests. The same excerpt describes head-of-line blocking when long prefills and short decodes compete on a single GPU. Treat bursty GPU capacity as a workload characteristic to measure, not as a reason to reject a platform in advance.
Isolation is a separate design decision
The excerpt on Trusted AI Agents in the Cloud places agents in a multiparty ecosystem in which untrusted components can lead to data leakage, tampering, or unintended behavior. It also states that existing Confidential Virtual Machines provide protection only per binary and offer no guarantees for cross-principal trust, accelerator isolation, or supervision of agent behavior. This does not imply a universal prescription for a sandbox or runtime, but it does provide a practical design rule: for each executing agent, define which code, data, tools, credentials, and outbound destinations are permitted. Persistent state is part of this: determine which data may remain after a run, who can read it, and when it is removed. A permitted external destination remains an explicit route that must be assessed; isolation alone does not determine whether an agent result is correct or desired.
Use one decision matrix before expanding
Create a decision matrix for an existing agent flow with five columns: workload peak, execution boundary, state, permitted external actions, and fallback. Under a realistic peak, measure latency and throughput; run representative tool calls in the intended isolation layer; check which state and credentials remain; and force a model, tool, and network failure to verify the fallback. Only then choose between Kubernetes, stronger separation, or a simpler managed path. The decision matrix also makes ownership visible: each column has one owner and one acceptance criterion. Practical artifact: “For each agent flow, record the GPU peak, execution boundary, permitted state, egress, fallback test, owner, and acceptance criterion.” This tool is intended for this infrastructure decision and does not replace a full security assessment. Limitation: “The supplied excerpts do not compare Kubernetes runtimes, sandbox products, persistent storage patterns, or fallback implementations; that choice requires a separate test in the intended environment.”
