TPU and Accelerator Access in Cloud Sandbox Environments
Agents need accelerators to run untrusted code safely and fast.
TPU and accelerator access in cloud sandboxes requires architectural design. As autonomous agents become primary consumers of accelerator hardware, their execution environments must reconcile speed, isolation, and continuity. The runtime must give untrusted agent code productive access to specialized hardware without weakening the boundary that makes that access safe.
Why AI agents changed the hardware and sandbox design problem at the same time
Agents plan, act, and loop, placing sustained, concurrent demands on hardware as tasks unfold. Google’s decision to split its eighth-generation TPU into training and inference architectures responds to that pattern, with TPU 8i specifically sized to serve many agents concurrently. The workload changes the design target from an isolated computation to a continuing sequence of decisions and actions.
That sequence also includes running machine-generated code. Because the code is untrusted, the environment in which it runs becomes a production security boundary. Hardware designers and sandbox engineers therefore face the same pressure from different directions. One must keep inference moving, while the other must contain the actions that inference produces.
What the TPU 8t / TPU 8i split reveals about inference architecture for agents
Google’s TPU split makes the distinction concrete at the silicon level. TPU 8t targets pre-training and embedding-heavy workloads, scaling to 9,600 chips per superpod through a 3D torus interconnect. Each chip has 128 MB of on-chip SRAM, with the architecture prioritizing sustained matrix throughput over fast decoding. Training and agent inference demand different balances. A design that works well for one may work less well for the other.
TPU 8i instead trades compute density for memory architecture, providing 384 MB of on-chip SRAM per chip and 288 GB HBM. Its SRAM capacity is three times the previous inference generation’s, supporting on-chip storage of the KV cache used during multi-step agent sessions and avoiding spills of that state to HBM. The aim is to prevent cores from stalling for data during long-context decoding. Communication receives its own specialized treatment. The new Boardfly network topology reduces chip-to-chip latency for the all-to-all traffic typical of mixture-of-experts routing, while Google’s Collectives Acceleration Engine handles synchronization across that fabric.
Both chips use Arm Axion, based on Neoverse V2, as the host CPU because orchestration, tool calls, and data preparation influence how often the accelerator actually runs.
Why shared-kernel containers cannot safely mediate accelerator access for agent code
What stands between that hardware and untrusted code? The sandbox grants or denies access, so its isolation model becomes part of the accelerator’s security architecture. Docker/runc containers share the host’s operating-system kernel. A sandbox escape or privilege escalation in one container can therefore affect the host and other tenants, rather than remaining confined to a separate guest kernel.
Accelerator access widens that exposure beyond CPU-only execution because it creates a path from untrusted code to privileged hardware. A June 2026 comparative study of code sandbox engines examined host attack surface, information leakage, defense-in-depth stackability, public CVE history, patch cadence, and upstream fuzzing posture. Those dimensions expose why “isolated” is not a sufficient description of an execution environment. What isolation must do depends on the threat model. Containers designed for trusted developer workloads are not, by that fact alone, adequate boundaries for code written by an LLM.
The isolation primitives the industry has converged on for agent sandboxes
The industry has adopted a small set of isolation approaches. gVisor intercepts system calls in user space without requiring a full VM, offering strong isolation with lower memory overhead than a microVM while trading some performance for security. Google’s GKE Agent Sandbox is built on gVisor with additional Kata Containers support. Google reports better price-performance on Axion processors than comparable hyperscaler cloud providers. That result connects the host CPU choice to the isolation design in purchasing decisions.
Google’s Agent Compute Engine uses Cloud Hypervisor with its own guest kernel and root access, supported by Titanium, Google’s custom offload architecture of purpose-built silicon, security microcontrollers, and tiered scale-out offload processors. Freestyle provides a full hardware-virtualized Linux VM with its own kernel, nested virtualization, and full Linux networking. It can fork a running VM in milliseconds. That stronger separation comes with higher resource costs.
Cloudflare Sandboxes occupy another point on the trade-off curve, emphasizing persistent Python and JavaScript interpreters, with TypeScript compiled to JavaScript, PTY support, and credentials injected through an egress proxy. These differences shape more than security posture. MicroVMs suit code-executing agents with moderate isolation needs, hardware hypervisor boundaries address strict enterprise governance, and kernel-level virtualization favors density and latency. None of those labels alone establishes accelerator readiness, because the boundary and its runtime overhead still have to fit the workload.
The cold-start versus isolation trade-off that determines whether agents can actually use accelerators at scale
A separate guest kernel takes time to start. Compared with a shared-kernel container, a microVM introduces provisioning overhead that compounds if an agent needs a fresh sandbox for every tool call. For accelerator-backed work, that delay can erode the gains delivered by faster silicon. The relevant interval is not merely VM boot time, because GPU or TPU initialization adds latency before useful work can begin.
Fly.io’s Sprites / Freestyle makes VMs cost nothing while paused and supports a 100–500ms warm wake, while allowing a running VM to be forked in milliseconds without pausing it. Those resume and fork capabilities have timings distinct from cold starts. Google’s GKE Agent Substrate addresses another part of the problem, having been engineered to run millions of sandboxes at 10× higher density than standard container runtimes.
Treat provisioning latency as a first-class runtime requirement. But which latency should govern the decision? For accelerator workloads, the useful measure must account for readiness to submit work. Fast boot cannot compensate for an execution environment that still waits for its hardware context.
Why stateful sandboxes matter more than ephemeral ones when agents drive long-running accelerator workloads
When an ephemeral environment loses state between steps, it must repeat setup, including package installation, model-weight loading, and hardware-context initialization. During that reconstruction, the accelerator can sit idle. A stateful runtime avoids repeating setup by preserving a continuous workspace in which files remain available, packages stay installed, and processes keep running.
Cloudflare provides persistent interpreters across calls, snapshot-based recovery stored in R2, and real-time filesystem watching. Freestyle preserves VMs and allows a running state to be forked, providing a restorable checkpoint without rebuilding the environment. These capabilities make continuity an operational property and improve the agent's interface. They address what survives between actions. They do not, by themselves, establish which actions safely completed.
Conversation history can tell an agent what happened, but it does not prove which shell command ran, which side effect was committed, or whether a retry would duplicate work. Durable execution requires an execution journal and idempotent tool boundaries.
How accelerator access is expanding across cloud boundaries, and what that means for sandbox portability
Oracle and Google said on October 27, 2026 at Oracle AI World in Las Vegas that Google TPUs would be available through Oracle Cloud Infrastructure. Availability is planned for calendar year 2027, making OCI the first hyperscale cloud outside Google to offer them. Both TPU 8t and TPU 8i are included for training, post-training, and production inference involving reasoning, agentic, and long-context applications. Accelerator choice is becoming less cloud-bound.
The operating split gives OCI responsibility for power, cooling, and datacenter networking, while Google provides TPU silicon and the software stack, including JAX, PyTorch, OpenXLA, and vLLM support. Mahesh Thiagarajan, president of Oracle Cloud Infrastructure, frames the value as matching different workloads to compute with suitable performance, throughput, and economics. Yet the deployment constraint remains explicit. TPUs are accessible only through rented cloud capacity, so strict on-premise or fully self-managed deployments require GPUs.
The announcement also leaves governance questions unresolved. Who answers a data-residency audit for Google TPUs running inside OCI infrastructure? It does not specify regions or settle that responsibility. Meanwhile, a sandbox tied to GKE-native primitives cannot simply follow the workload to another cloud. Docker-native compatibility offers a practical portability approach by preserving what teams already run, though compatibility with container workloads is not the same as relying on shared-kernel isolation.
The compliance and security floor that agent sandboxes running accelerator workloads must clear
SOC 2 Type II is effectively mandatory for serious enterprise conversations about sandbox infrastructure, but a vendor’s certification may cover its managed cloud without covering BYOC or on-premises deployments. The deployment model is therefore part of the claim.
Persistent environments can be as safe as ephemeral ones when isolation, patching, and monitoring are solid, because agents need consistent boundaries. Agent Compute Engine is built to meet that need with its hardware-isolated boundary and silicon-rooted defense-in-depth.
With a black-box sandbox, security teams must take claims about containment on trust, while an open-source runtime lets them inspect the code, audit it, and check those guarantees. Even with that transparency, compliance coverage must apply to the deployment. It gives the security review something concrete to examine beyond the provider’s description of its boundary.
Why purpose-built sandbox infrastructure is the architectural conclusion, not a product preference
Taken together, strong isolation, low provisioning latency, persistent state, accelerator-aware execution, portability, and deployment-specific compliance constrain the same runtime. Each requirement can be addressed separately. The difficulty is satisfying them simultaneously, especially when a design target such as sub-90ms provisioning must coexist with a meaningful security boundary. Shared-kernel containers favor startup speed, full VMs provide separation but historically impose startup costs, and developer sandboxes prioritize ergonomics over handling untrusted code at agent scale.
Google’s additions illustrate why this calls for new infrastructure. Agent Compute Engine, GKE Agent Substrate, and GKE Agent Sandbox are purpose-built responses to agent workloads. Vercel Agent Desktop, released open-source in October 2026, applies the same reasoning to computer-use agents. Built in Rust, it uses OS accessibility trees rather than screenshot-based pixel inference to provide stable action references and safe-to-retry operations. The abstraction changes to fit the consumer.
Arm’s Performix toolkit, announced independently by Arm on April 28, 2026, extends that shift into performance analysis through structured outputs intended for automated agent tuning loops as well as human dashboards. The architecture requires an execution layer that can make hardware access, isolation, continuity, and governance work together throughout an autonomous task, beyond enabling an existing container configuration to acquire a TPU.
