cold start latency benchmarks across AI agent sandbox platforms

A cold start is not one thing. What happens in that window, and how long it takes, depends entirely on which isolation primitive sits underneath the sandbox, and the industry has settled on four broad approaches that trade speed for security boundary strength in different places. That tradeoff deserves top billing, not a footnote: most teams evaluating these platforms look at the cold-start number first and the isolation model second, and that ordering is backwards. Get the isolation model wrong and the cold-start number stops mattering.

Standard containers, the Docker and runc model most engineers already know, start fast because they share the host kernel. That speed carries a real cost: the boundary between one container and another, or between a container and the host, is a software convention the kernel enforces, not a hardware guarantee. Recent vulnerabilities make that distinction concrete rather than academic. CVE-2025-31133 and CVE-2025-52881, both in runc, plus a February 2026 vulnerability in n8n (CVSS score of 9.9) and the OpenClaw incident that same year, which exposed more than 135,000 agent framework instances, were not hypothetical scenarios security teams worried about in the abstract. They were container escapes that happened, on infrastructure people were running production agents on. Anyone running untrusted, agent-generated code inside a bare Docker container and calling it "isolated" is trusting a convention, not a wall, and that distinction is the whole section in miniature.

gVisor sits a step further along. It intercepts syscalls in userspace rather than letting a container talk directly to the host kernel, shrinking the kernel's attack surface without requiring hardware virtualization. That buys a real security improvement for moderate overhead, and gVisor runs fine without KVM, which makes it a reasonable choice for compute-bound work where the code being run is not fully adversarial.

Firecracker microVMs go further still. A Firecracker sandbox boots an actual lightweight virtual machine on top of hardware virtualization, Intel VT-x or AMD-V, so the CPU itself enforces the separation between tenants rather than the OS taking that job on faith. Baseline boot cost for a cold Firecracker microVM runs around 125 milliseconds, slower than a container, but the boundary it draws is hardware-enforced. Breaking out of it requires a hypervisor-level vulnerability, and that class of bug is expensive: Google's kvmCTF program alone offers $250,000 for a working KVM escape, and exploit brokers price that category of bug in the $250,000 to $500,000 range. Raise the cost of an exploit that high and the pool of people who bother trying shrinks with it.

WebAssembly takes a different route entirely: near-zero overhead per invocation, a capability-first security model where code can only touch what it is explicitly granted, and sandboxing that runs at very low overhead per tool call on platforms built around WASM orchestration, such as those using Cosmonic or wasmCloud. The tradeoff is language support. WASM is polyglot in principle but constrained in practice, and not every runtime an agent might need to call into compiles cleanly to it yet.

Here is what actually breaks the apparent tradeoff between speed and security, and vendors love to blur this point, so it is worth being precise. It is not a faster kernel. It is snapshotting. Pausing a running Firecracker microVM preserves its memory and filesystem state exactly as it was, and resuming from that snapshot skips the full boot sequence entirely. Resume times of 5 to 30 milliseconds are achievable from a snapshot, even though creating that same microVM from scratch takes 125 milliseconds or more. That is the engineering move that lets a platform claim both hardware-level isolation and a cold-start number competitive with a bare container. The isolation model did not get weaker. The boot path got shorter.

Cloudflare's approach with V8 isolates sits at a different point on this curve entirely. Rather than booting any kind of kernel, a V8 isolate shares a single JavaScript runtime process across many sandboxed executions, which gets startup times under 5 milliseconds. The tradeoff is confinement to JavaScript and TypeScript, and the loss of in-memory state the moment the isolate goes to sleep. It is a legitimate architecture, but it solves a narrower problem than a general-purpose code execution sandbox. Treating it as a like-for-like competitor to a platform built on microVM isolation is comparing a scalpel to a multi-tool and asking which one is sharper.

The rule holds, then: faster cold starts generally mean a weaker isolation boundary, with one specific exception. The platforms that appear to defy that rule are not weakening their security model to get there. They are engineering around the boot sequence itself through snapshotting, which is a fundamentally different move than just trusting the kernel more. Any platform that claims sub-10ms starts without saying which of these two moves it made is a platform worth pushing back on before signing anything.

How session duration and state persistence change the calculus beyond first boot

A cold-start benchmark measures the first few hundred milliseconds of a sandbox's life. It says nothing about the next hour, and for most real agent workloads, the next hour is where the actual cost lives. Teams that pick a platform off the cold-start chart alone, and skip this part of the evaluation, are optimizing the wrong number entirely.

Every time a sandbox expires, or gets garbage collected because it sat idle, whatever state accumulated inside it disappears: crawled data, installed packages, conversation history, warm filesystem caches that made the last several tool calls fast. Rebuilding all of that costs something real. Full-context memory approaches, where an agent reconstructs its working state from scratch, run around 26,000 tokens per conversation on standard benchmarks, and every cold restart that forces a rebuild burns those tokens again on top of whatever latency the restart itself adds. That cost compounds quietly across a long-running agent's lifetime in a way a single cold-start benchmark will never show.

Here is the part that ought to reframe how teams think about sandbox pricing: tool-augmented agents spend as much as 54.5% of their total execution time sitting in GPU idle periods, waiting on a model response, waiting on a human, waiting on some external system to finish. Most of an agent's life is not spent computing. It is spent waiting. Why does that matter more than the boot number? Because a platform that charges for standby time, or imposes a minimum billing window on every invocation, is charging the most for the exact state agents spend the majority of their time in. Optimize purely for cold-start speed and ignore standby pricing, and the result is a fast, expensive sandbox that bills for silence.

That pricing shape has a direct engineering consequence. When a sandbox platform imposes a hard session limit, teams do not just accept shorter workflows. They build around it: serializing state out to object storage before the session dies, then re-hydrating that state on the next invocation. It works, but it adds a layer of engineering complexity that purpose-built stateful infrastructure would have made unnecessary. It also introduces a new failure mode, since a serialization step that fails silently becomes a bug buried in the agent's core loop rather than a platform limitation someone can work around from the outside.

Platform-by-platform breakdown: cold start numbers, isolation model, session behavior, and compliance

Reading the platforms side by side is where the isolation-versus-speed tradeoff from the first section, and the session-duration problem from the second, stop being abstract and start looking like actual product decisions. Some of these platforms are built for one shape of workload and quietly wrong for another, no matter how good the marketing page reads, and one platform in particular below has a session-limit problem worth flagging before it costs anyone a production incident.

One open-source platform runs Docker containers by default, with Kata Containers available as an opt-in for microVM-level security. Sessions run unlimited in duration, sandboxes are stateful with persistent workspaces, and the platform supports desktop environments for computer-use agents. Compliance covers SOC 2, HIPAA, and GDPR, and the platform offers BYOC (bring your own cloud), letting customers run the compute inside their own environment. This is the fastest startup profile in the comparison, and also the weakest default isolation of the major platforms here. Teams running genuinely untrusted code need to opt into Kata Containers rather than accept the Docker default as sufficient, and skipping that step is the single most common mistake in this whole category.

Blaxel reports cold creation from a template in the 200 to 600 millisecond range, with resume from standby landing between 4 and 10 milliseconds, putting effective latency across the platform under 25 milliseconds in most cases. Isolation runs on Firecracker microVMs, giving it a hardware-enforced kernel boundary per sandbox. Standby is perpetual and free of compute charges, with automatic shutdown after 15 seconds of network inactivity, and filesystem plus memory state survives that shutdown cycle. Sandboxes are co-located with agent hosting on the same infrastructure, which removes the network round trip described earlier as a compounding cost. Compliance includes SOC 2 Type II, HIPAA with a signed BAA, and ISO 27001, and the platform has been a first-class native sandbox provider in OpenAI's Agents SDK since April 2026. Concurrency scales across multiple tiers, with higher limits negotiable at the enterprise level.

Another microVM-based platform, built around a pre-built notebook-style Code Interpreter SDK, reports use by 94% of Fortune 100 companies for frontier agentic workflows and offers minimal integration effort as its core pitch. But session limits are a hard ceiling here, and this is where the pitch and the reality pull apart. The Hobby plan caps out at one hour, the Pro plan at 24 hours, and hitting that ceiling terminates the session with no recovery path and no shutdown signal sent to the agent beforehand. That last detail matters more than it sounds: an agent mid-task simply loses its environment, with nothing telling it the clock ran out. The core is open-source under Apache 2.0, with BYOC available for enterprise customers, and the company raised a $21 million Series A led by Insight Partners in July 2025. It is also a native sandbox provider in OpenAI's Agents SDK as of April 2026. Strong open-source traction and fast integration, undercut by session caps that make long-running agent workflows structurally unworkable without external state management bolted on afterward, and that bolt-on is exactly the kind of failure mode described in the section above.

A separate platform runs gVisor for syscall interception rather than a microVM hybrid in its standard sandbox path, with VM Sandboxes still in alpha. Session duration extends to 24 hours, and longer workflows lean on snapshot and state restoration rather than an extended session window. Its differentiator is GPU access: T4, L4, A10, L40S, and A100 variants, plus H100, H200, B200, B300, and RTX PRO 6000, all without quota restrictions, which matters for ML-heavy agent workloads that need more than CPU cycles. CPU pricing starts around $0.047 per vCPU-hour. Compliance covers SOC 2 Type II, with HIPAA available on enterprise plans via BAA, and enterprise tiers include Okta SSO and audit logging. Production users include Lovable and Quora, and the platform serves more than 10,000 teams. It fits teams that need GPU access alongside code execution and are comfortable working inside a Python-first, fully managed environment.

Northflank reports fast cold starts with Docker compatibility, and it stands out for offering a genuine choice of isolation backend rather than locking customers into one model: Firecracker, gVisor, and Kata Containers (the last running on Cloud Hypervisor as its primary VMM backend) are all available, and the user picks. Sessions run unlimited, execution is stateful, and BYOC deployment runs inside the customer's own AWS, GCP, Azure, Oracle, CoreWeave, or on-premises environment via BYOK. Compliance includes SOC 2 Type 2 across both managed cloud and BYOC deployments, plus HIPAA with a BAA. Scale-wise, the platform processes more than 2 million isolated workloads monthly across more than 70,000 developers in production, with customers including Writer and Sentry. Northflank named itself "best overall" in its own 2026 comparison guide, and on isolation-backend flexibility alone, that claim actually holds up: no other platform in this set lets the customer choose the isolation primitive per workload.

Vercel's Sandbox product starts quickly, and its persistent sandboxes, currently in beta, auto-save state on stop and resume from where they left off. Isolation runs on Firecracker microVMs, and Docker can run inside the sandbox itself. Session limits are 45 minutes on the Hobby plan and 5 hours on Pro and Enterprise, with concurrency capped at 10 sandboxes on Hobby and scaling to 2,000 on Pro and Enterprise. Compliance covers SOC 2 Type 2, with SSO and audit logs available on enterprise plans. The product is clearly built around v0 and AI-SDK use cases, meaning code generation, live previews, and fast experimentation, and the 5-hour ceiling makes it a poor fit for anything that runs longer than that by design rather than by accident.

Cloudflare's Sandbox product reports sub-50 millisecond cold starts and runs on edge infrastructure, executing code physically close to the end user across Cloudflare's global network. Isolation combines per-sandbox VMs on Cloudflare's Containers infrastructure with V8 isolates for JavaScript and TypeScript workloads specifically, and its Dynamic Workers feature lets an LLM spawn code directly into child isolates at runtime. The catch is not minor: when a Cloudflare sandbox goes to sleep, everything inside it, files and running processes alike, is gone. Persistence across sessions requires explicitly wiring up R2 mounts, the backup and restore APIs, or persistent storage. The product is currently in beta, available on the Workers Paid plan, and Cloudflare reports an 81% reduction in token usage when using Dynamic Workers compared to standard tool-calling. Strong fit for Workers-native teams running short, stateless JavaScript or TypeScript executions at the edge. Wrong fit for anything that needs to remember something between sessions, and no amount of edge speed changes that math.

A final platform in this set reports 100 millisecond command execution on a custom bare-metal hypervisor built to deliver roughly twice the vCPU performance of standard virtualized instances. Isolation runs a dual layer combining a micro-VM with an additional container boundary inside it. Its real differentiator is not the sandbox at all: it ships a built-in agent evaluation and benchmarking layer, covering SWE-Bench, SWE-Smith, and custom benchmark suites, alongside the execution infrastructure itself, aimed at closing the gap between agents that work in a demo and agents that survive enterprise deployment. Compliance covers SOC 2 Type II, HIPAA, and GDPR, and the company raised a $7 million seed round. It suits teams that want execution infrastructure and evaluation tooling in one stack rather than stitched together from separate vendors.

What the numbers reveal when read together: the three performance tiers and what separates them

Line these platforms up by raw cold-start number and a ranking falls out easily enough. Trusting that ranking is the biggest analytical mistake a team can make here, and it is worth walking through exactly why before picking anything off the list above.

A platform reporting a 200 to 600 millisecond cold creation time paired with a 4 to 10 millisecond resume from standby can outperform a platform with a flat 90 millisecond cold start, in practice, for a specific kind of workload: one where most sandboxes are resumed rather than created fresh. The number that actually predicts real-world latency is not the cold-start figure in isolation. It is the ratio of new-sandbox creations to resumed-sandbox invocations in the actual traffic pattern a team runs. An agent that spins up a fresh environment for every single task pays the cold-creation cost constantly, while an agent reusing a small pool of long-lived sandboxes across many tool calls barely pays it at all. For that second workload, the resume number is the only column that matters. Anyone comparing platforms on cold-start alone, without asking which of these two patterns their own agent follows, is reading the wrong row of the spreadsheet.

Concurrency complicates the picture further. A cold-start figure benchmarked against a single sandbox spinning up in isolation does not necessarily hold once hundreds or thousands of sandboxes start at once, since shared infrastructure, scheduler contention, and resource allocation all behave differently under load than they do in a clean single-instance test. This kind of degradation under concurrent load in virtualized and containerized systems is a known challenge in production environments, and the general finding holds across the industry: a benchmark number measured in isolation is a ceiling on performance, not a guarantee of it once real traffic shows up.

Read all fourteen platforms together and three rough tiers emerge. At the fast end sit the sub-100 millisecond and sub-50 millisecond platforms, an edge-based V8 isolate service and a container-default open-source option among them, where speed comes from either sharing a runtime or accepting a software-level security boundary rather than a hardware one. That speed is real, but it is bought with a specific, named tradeoff, not a free lunch, and treating it as one misreads the entire first section of this piece.

In the middle sit the microVM-based platforms that use snapshotting to push resume times into the single-digit or low double-digit milliseconds while still preserving a hardware-enforced isolation boundary. This is arguably the more interesting engineering achievement of the three tiers, since it refuses the tradeoff outright instead of accepting it. And at the slower end, in terms of what actually matters for long-running work, sit the platforms whose real constraint is not the cold-start number but the session ceiling sitting behind it: fast enough to boot, but capped at an hour, or five hours, or twenty-four, in a way that rules out entire categories of agent work regardless of how quick that first response feels.

None of this points to one correct platform, and any comparison that claims otherwise is selling something. It points to a question most teams skip: does the workload look bursty and stateless, or long-running and stateful? Answer that first, and the fourteen platforms above stop looking like a leaderboard and start looking like a set of tools built for genuinely different jobs, each one right for exactly the workload it was built around and wrong for most of the others.