Sandbox Providers That Handle Hundreds of Concurrent AI Agents

This piece looks at what it takes to run hundreds of AI agents at once, and why spinning up more containers, the standard answer, stops working once those agents start executing real commands. A single coding agent might clone a repository, install dependencies, run a build, read the error output, and try again, all inside one session that could last minutes or hours. That workload alone is manageable. Multiply it by hundreds of agents running concurrently, each one improvising shell commands and pulling in packages nobody has reviewed, and the demands on the underlying infrastructure do not just add up, they compound.

Part of the problem is structural. Generic containers share the host kernel, so if one tenant's workload suffers a container escape, it can reach into another tenant's workload on the same machine. That risk is tolerable for code a company wrote and reviewed internally. It is a different calculation entirely for code an AI agent generated seconds ago and is about to execute without a human in the loop.

Part of the problem is timing. Running many sandboxes at once is a separate challenge from creating many sandboxes at once, and most infrastructure built for steady workloads was never tested against the second one. Northflank's own evaluation criteria treats sustained concurrent capacity, creation throughput during bursts, P95/P99 readiness latency, and failure rates under load as four separate measurements, because a platform can handle one without handling the others. An agent fleet that scales up to handle a sudden spike in user requests needs hundreds of sandboxes to appear within seconds, not minutes, and a platform that provisions smoothly under steady load can still buckle the moment that load arrives all at once.

There's also a gap between how providers report their own scale and what that reporting actually tells a buyer. A platform that has processed millions of sandbox executions over its lifetime has told a prospective customer very little about how many of those sandboxes it can run live, at the same moment, under the same account. Monthly volume accumulates over weeks; concurrent capacity is a snapshot of a single instant. Engineers evaluating sandbox providers for agent workloads should ask about the second number specifically, because a production fleet of agents will only actually run if that number is high enough.

The four architectural properties that determine whether a sandbox scales to concurrent agents

Four properties separate sandbox platforms that hold up under hundreds of concurrent agents from those that only look like they do on a spec sheet: how fast they provision, how strongly they isolate tenants, how well they manage state across an agent's lifecycle, and how they handle bursts of simultaneous demand. None of the four stands on its own. A platform can win on any single axis and still be the wrong choice if it fails on the others.

Provisioning speed is the clearest of the four to measure and the easiest to misread. Cold-start latency, the delay between requesting a sandbox and successfully running the first command inside it, matters on its own, but it matters far more once hundreds of requests arrive within the same few seconds. Vendors often measure different units of work when they publish creation-time figures, so you cannot assume a number quoted by one provider is comparable to a number quoted by another. The only reliable test is benchmarking cold starts on the actual workload and the actual container image a team plans to run, not accepting a vendor's headline figure at face value. Snapshot-resume adds a further wrinkle: a sandbox waking from a saved memory snapshot starts meaningfully faster than one booting from nothing, and conflating the two measurements produces a badly distorted picture of how a platform will behave in production.

Tenant isolation strength is where the security conversation and the performance conversation meet. For code an AI agent wrote and is about to run without review, the isolation model is the actual security boundary, and a dedicated kernel per workload closes off escape paths that a shared-kernel container runtime leaves open. That said, the runtime is only one layer. Network policies that restrict what a sandbox can reach, credentials scoped narrowly to what a task requires, and hard resource limits on CPU and memory all contribute to the isolation posture independent of which runtime technology sits underneath. A team that picks a strong isolation runtime and then grants every sandbox broad network access and a long-lived API key has undone much of what that runtime bought them.

Stateful lifecycle management is the property most likely to get overlooked during a proof of concept and most likely to cause pain in production. A coding agent that has to re-clone its repository and reinstall every dependency on each tool call is not a production system, it is a demo that happens to work. Three distinct levels of persistence matter here, and they are not interchangeable: filesystem persistence, where files survive a pause; directory snapshots, which can be mounted into a running sandbox; and full memory snapshots, which restore running processes mid-execution so an agent resumes exactly where it stopped. The hibernate-then-wake pattern built on top of these snapshot types is what makes concurrent agent fleets financially workable. A sandbox snapshots its full state, compute billing pauses, and the sandbox resumes on the agent's next turn, so the agent experiences a continuous workspace while the platform stops burning compute cycles during the long stretches an agent spends waiting on a language model's response. It also turns out that persistent sandboxes are not inherently less secure than ephemeral ones. Security depends far more on isolation, patching discipline, and monitoring than on how long a sandbox stays alive, so if those three things are in place, the assumption that only short-lived compute passes a security review does not hold.

Burst-ready scheduling is the property that ties the other three together under real load. Account-tier quotas complicate this further: a platform's overall capacity can be enormous while a specific account's tier still caps concurrent sandboxes at a number far below what a production fleet needs, so the responsible move is checking both the platform's demonstrated ceiling and the limits attached to the specific plan being purchased. A platform optimized for one will not automatically perform well on the other, and only testing both reveals which failure mode a team is exposed to.

Isolation technology choices at high concurrency

The isolation technology underneath a sandbox determines not just its security posture but its scalability. At hundreds of concurrent agents, that same choice governs how many sandboxes fit on a given piece of hardware, how fast new ones can be created, and which agent workloads can run. What stays invisible during a ten-sandbox pilot can decide everything once the number climbs into the hundreds.

Firecracker, the microVM technology AWS built and open-sourced, boots in approximately 125 milliseconds and adds under 5MB of memory overhead per instance, achieved by stripping out the PCI bus, USB support, and other legacy hardware emulation that a general-purpose VM carries by default. That per-instance overhead figure looks academic at small scale. It stops being academic once hundreds of these instances are live on the same fleet simultaneously, because overhead multiplied by hundreds becomes real memory pressure on the host. Kata Containers takes a different path to a similar goal: it wraps microVM technology, including Firecracker, Cloud Hypervisor, QEMU, or Dragonball, behind a container-orchestrator-compatible interface, which gives a team hardware-level isolation without forcing them to abandon a container-based orchestration layer they already operate.

gVisor sits in a different spot on the spectrum. It intercepts system calls in a user-space kernel called Sentry rather than running a full separate kernel, which makes it stronger than a bare container and lighter than a full microVM. The tradeoff appears in compatibility rather than in raw performance: some Linux syscalls are not implemented in Sentry, and workloads relying on ptrace, raw sockets, or certain ioctl calls can fail, sometimes silently. If a team adopts gVisor for moderate isolation at lower overhead, it needs to test its actual agent workload against that runtime instead of assuming general Linux compatibility, because the failure may only appear when an agent hits exactly the operation Sentry doesn't support.

Full KVM virtual machines sit at the heavier end of the spectrum. They support nested virtualization and a broader range of operating system features than microVMs do, but they carry more overhead per instance, which caps how many a single host can run side by side. Copy-on-write snapshot forking softens that cost: each forked sandbox shares the same base memory pages as its parent until it writes new data, so the marginal memory cost of each additional fork is only the incremental data that instance changes, not a full duplicate of the entire VM. Standard OCI containers remain the fastest and lightest option on this list by a wide margin, and for trusted internal code that speed advantage is hard to beat. For untrusted, AI-generated code running across hundreds of tenants, that same lightness is the liability: a kernel exploit inside one container's workload can cross into the host and from there into every other tenant sharing it.

Teams benefit from treating isolation technology as a dial, choosing a different setting for each workload rather than applying one fixed decision to all of them. A platform that lets engineers choose Kata for a task that demands maximum isolation and gVisor for a lower-stakes task that only needs moderate isolation with lighter overhead gives them room to match the runtime to the risk, rather than paying the heaviest possible overhead on every single sandbox regardless of what it's actually running.

Leading sandbox providers against those four criteria

No sandbox provider currently leads on provisioning speed, isolation strength, stateful lifecycle management, and burst-ready scheduling all at once. Each platform's architecture reflects a deliberate set of tradeoffs, and those tradeoffs make a given platform a strong fit for some agent workloads and a poor fit for others. If you compare providers against the four-property framework instead of a single headline number, you get a clearer picture of which platform fits a given fleet.

Northflank positions itself as a strong overall choice for fast, high-concurrency sandboxes spanning coding agents, APIs, browser automation, and both CPU and GPU workloads. Its isolation layer uses Kata Containers, giving each workload a dedicated kernel through a microVM, where nested virtualization is available on the underlying hardware, and falls back to gVisor where it isn't, with any standard OCI container image running unmodified either way. Deployment supports bring-your-own-cloud across AWS, GCP, Azure, and bare metal, with orchestration running either on Northflank's managed infrastructure or inside a customer's own cloud account. On scheduling, Northflank is built for high concurrency specifically, supporting thousands of concurrent sessions across coding agents, scripts, APIs, and GPU tasks.

Daytona, the platform behind this publication, started life as a developer environment manager, but it was rebuilt specifically around the demands of autonomous AI systems. Sandbox creation runs in under 90 milliseconds, which keeps agents responsive even as hundreds of concurrent sessions accumulate, since cold-start penalties that would otherwise stack up across a fleet don't have room to compound at that speed. Isolation runs through fully separated runtime environments, so AI-generated code cannot reach the underlying infrastructure or other tenants that share the platform. Sandboxes are stateful by design and can run indefinitely, letting agents resume long-running work exactly where they left off without re-cloning a repository or reinstalling dependencies on every turn. Deployment also includes an optional bring-your-own-cloud tier, where sandbox runners execute inside a customer's own cloud account while Daytona's control plane stays on Daytona's infrastructure; it ships as an enterprise, invite-only option. Compliance coverage, SOC 2, HIPAA, and GDPR, is built into the base platform. Daytona's core development moved to a private codebase in June 2026, with its public repository frozen at version 0.190.0 under the AGPL-3.0 license, and that stands in contrast to execution services that offer no public runtime history for teams to inspect. Organizations running agent workloads on the platform include SambaNova, LangChain, Turing, and Writer.

Blaxel's core differentiator is standby cost: sandboxes can hibernate indefinitely, and billing during that standby period covers only snapshot storage, never compute. Resume latency comes in under 25 milliseconds, with filesystem state, memory, and running processes all intact on wake. Isolation runs on a microVM per workload, so it closes the same kernel escape paths that a shared-kernel container runtime leaves open. Compliance coverage includes SOC 2 Type II, ISO 27001, and HIPAA support through a business associate agreement. Storage runs through Agent Drive, a distributed filesystem currently in private preview, and it mounts to multiple sandboxes at once with concurrent read-write access, alongside Volumes for guaranteed long-term persistence. Blaxel is also a first-class sandbox provider inside the OpenAI Agents SDK, and its compute primitives, Sandboxes and Batch Jobs, support parallel, asynchronous work running anywhere from minutes to hours.

Upstash Box takes a different approach: it places the agent inside the sandbox itself. The coding harness, whether Claude Code, Codex, OpenCode, or a custom build, along with the model, shell, files, and Git integration, are all provisioned together as a unit, so sending a single prompt triggers cloning, fixing, testing, and pull request creation without a separate orchestration loop sitting outside the sandbox. Isolation runs through a dedicated container per sandbox, each with its own filesystem, process tree, and network namespace. Billing covers active CPU time only, so the idle stretches an agent spends waiting on an LLM response go unbilled, a structural advantage for agent workloads that spend most of their wall-clock time blocked waiting. Concurrency starts at ten simultaneous boxes on the free plan, with paid tiers scaling beyond that ceiling. SDKs are available in TypeScript and Python.

Vercel Sandbox fits teams already building inside Vercel's application platform, offering SDK-managed microVMs with configurable sessions and snapshots that plug into an existing Vercel workflow. Its billing model, like Upstash's, charges for active CPU only and excludes I/O wait time, benefiting agent workloads where most of the clock time is spent waiting on a model or an external API.

Cloudflare Sandbox reached general availability on April 13, 2026, and it runs on the Workers Paid plan. It's built for teams that run agents from Cloudflare Workers, and it supports command execution, file operations, and application previews within that environment. Billing again charges for active CPU only, and its fine-grained billing granularity aims to lower costs for workloads that spend most of their time waiting on external calls.

Taken together, these platforms show that the four-property framework from earlier in this piece, provisioning speed, isolation strength, stateful lifecycle management, and burst-ready scheduling, produces genuinely different answers depending on which property a given agent workload weights most heavily. A team running thousands of short-lived, trusted automation tasks has different priorities than a team running long-lived coding agents against untrusted, AI-generated code at high concurrency, so the right sandbox provider follows from that distinction, not from whichever platform publishes the biggest headline number.