An agent is only as capable as the environment it can safely act in.
To work autonomously, agents need a way to verify their own work. The agent can have a rich action space and a perfect reward signal, but without a real environment to execute in, those capabilities remain largely theoretical. As models become more capable, they're trusted with increasingly consequential work: modifying databases, running arbitrary code, deploying infrastructure, and interacting with production APIs. With a full runtime workspace, agents can verify their own work and iterate until success.
The challenge is to build a workspace that is indistinguishable from a developer's local environment while safely isolating the consequences of everything an agent does. A sandbox provides an isolated execution environment where an agent can safely run code, interact with services, and recover from failure. A well-designed sandbox gives an agent a real playground while ensuring every action remains isolated, reproducible, and disposable.
A great execution environment should have the following core properties:
- 1.
Control plane: keeping execution ready with minimum latency
Provisioning must feel almost instantaneous, making the experience responsive enough to become part of a developer's everyday workflow.
- 2.
Execution plane: a real machine, not a shell
The environment should feel indistinguishable from a developer's local setup. Agents need a real filesystem, Docker, customer repositories, databases, development services, networking, tools, and the same setup scripts they would execute in a real development environment.
- 3.
Security and Network plane: isolating capability without restricting it
Agents need the freedom to take meaningful actions, but every action should be isolated behind strong security boundaries. The goal is not to restrict what the agent can do inside its workspace, it's to ensure those actions execute safely within well-defined security, access, and network controls.
- 4.
Data plane: deterministic state at production scale
Every execution should begin from a known state, and every action should be recoverable. Identical inputs should produce identical environments, making failures debuggable, evaluations repeatable, and successful trajectories reusable.
Every design decision ultimately optimizes for the Pareto frontier across these core principles.
In this post, we walk through the core infrastructure powering the workspaces where agents execute, explore, and learn from experience. We talk about how we provision the workspace with the same capabilities as a real environment, including a real filesystem, Docker, networking, representative data, and production tooling, while ensuring every run remains isolated, reproducible, and disposable.
Overview
The control plane minimizes latency by ensuring execution environments are ready before they're needed. The execution plane provides the fidelity of a real developer workstation rather than a constrained runtime. The security plane enforces strong isolation without limiting what an agent is capable of doing. Finally, the data plane guarantees every sandbox begins from an identical, reproducible state.
Control plane: keeping execution ready with minimum latency
Agents are interactive systems, and every second spent preparing an execution environment is a second the user spends waiting. Booting a microVM, provisioning storage, cloning repositories, initializing services, loading tools, and performing health checks can easily dominate the critical path of a request. If every interaction requires creating a sandbox from scratch, even the most capable model quickly feels sluggish.
The obvious solution is to avoid creating sandboxes on demand.
The control plane orchestrates the lifecycle of execution environments, ensuring the right sandbox is provisioned, initialized, and assigned to an agent with minimal latency. It also maintains a pool of pre-provisioned execution environments that are ready to accept work immediately. Instead of booting a new sandbox for every request, the scheduler attempts to attach an agent to an existing warm sandbox. Cold starts occur only when the warm pool is exhausted or when a workload requires resources or configuration that are not already available. Daytona provides native support for warm sandbox pools, allowing us to pre-provision execution environments and attach agents to them with minimal startup latency.
A warm sandbox is much more than just a running virtual machine. By the time it enters the pool, the operating system has booted, Docker is running, repositories have been cloned, dependencies installed, the agent harness runtime initialized, and supporting services, including MCP servers, are already available. From the agent's perspective, execution begins almost immediately because the expensive initialization work has already been completed.
Maintaining that pool is fundamentally a scheduling problem rather than a provisioning problem. Holding too few warm sandboxes increases cold-start latency, while holding too many wastes compute on idle capacity. The scheduler continuously predicts demand, provisions new sandboxes ahead of time, and reclaims unused capacity through an LRU-style eviction policy.
A live model of a smaller scale warm pool. Requests arrive and attach to warm sandboxes instantly, the scheduler refills the pool in the background, and a cold boot happens only when the pool has no ready sandboxes.
The optimal pool size depends on workload patterns, repository characteristics, dependency graphs, user behavior, and resource requirements. The scheduler becomes a prediction system: anticipating future demand, preparing resources ahead of time, and continuously balancing responsiveness against cost. The biggest latency improvements do not come from making every provisioning step faster. They come from avoiding unnecessary provisioning work entirely.
One of the highest-leverage optimizations comes from intent prediction. Because user interaction begins before a request is submitted, the control plane can start preparing a sandbox while the user is still composing their prompt. By overlapping initialization with user think time, many potential cold starts become simple attachments, reducing perceived latency without changing the execution environment itself.
When a warm sandbox is unavailable, the fallback path remains deterministic. Base images are restored from versioned snapshots, provisioned with workload-specific CPU, memory, and storage allocations, initialized, health-checked, and then handed off to the agent. The fast path and the fallback path therefore produce equivalent execution environments, differing only in how much work has already been completed before the request arrives.
Execution plane: a real machine, not a shell
Low latency in sandboxes is only valuable if the environment behaves like the one the agent is ultimately targeting. An execution environment that omits services, simplifies the filesystem, or replaces production tooling with mocks may be faster to provision, but it fundamentally changes the agent's behavior which isn't useful. Successful trajectories in an artificial environment often fail when transferred to real infrastructure.
For that reason, our goal is fidelity rather than minimalism. Each sandbox is designed to resemble a developer workstation as closely as possible. Agents operate inside a complete environment with a writable filesystem, cloned repositories, project configuration, developer tooling, Docker, MCP servers, and the same supporting services they would encounter during production development. The objective is straightforward: if a task succeeds inside the sandbox, it should succeed when executed against the customer's actual environment.
One consequence of this philosophy is treating containerized workloads as first-class citizens. Modern software development rarely consists of a single process running on a host machine. Applications depend on databases, caches, message queues, background workers, and other services that are typically orchestrated through Docker Compose or similar tooling. Supporting these workflows requires more than the Docker CLI, it requires a functioning Docker daemon running inside the sandbox itself. Rather than reasoning about whether a change should work, the agent can directly verify its behavior by building and running the application it has modified.
Providing this level of fidelity inevitably expands the execution surface. Running nested containers, exposing a Docker daemon, and supporting arbitrary developer tooling all introduce additional security considerations. Rather than restricting these capabilities, we isolate them through the security architecture described later in this post. The goal is not to reduce the agent's capabilities, but to ensure those capabilities remain safely secured and isolated.
Ultimately, the execution plane exists to eliminate the gap between evaluation and deployment. Agents should develop, test, and debug software inside the same kind of environment where that software will eventually run. The closer the sandbox mirrors reality, the more reliable the agent becomes.
Security and Network plane: isolating capability without restricting it
The execution plane intentionally gives agents broad capabilities. They can execute arbitrary code, interact with Docker, modify repositories, access representative data, and communicate with external services. Those capabilities are what make agents useful, but they also make sandbox security fundamentally different from traditional application security. The goal is not to prevent execution. It is to ensure that execution remains fully isolated. Daytona helps us improve security between different sandboxes by supporting both kernel level isolation and hardware level isolation.
No single security boundary is sufficient. A sandbox simultaneously executes untrusted, model-generated code while interacting with customer infrastructure and credentials. Protecting against privilege escalation, secret exposure, lateral movement, and data exfiltration requires multiple independent layers of isolation rather than a single perimeter.
Secretless sandboxes
The safest secret is one the agent can never access.
This principle shapes the entire credential architecture. Any credential exposed to the execution environment, whether through environment variables, mounted files, or process memory, can ultimately be exfiltrated if an attacker gains control of the agent through prompt injection or model manipulation. Short-lived credentials reduce exposure but do not eliminate the underlying risk. If the agent can read a secret, it can leak it.
Instead, credentials must be outside the sandbox entirely. Requests requiring authentication are routed through a credential-injecting proxy that attaches the appropriate credentials immediately before forwarding the request. The agent issues an ordinary network request without ever observing the underlying secret, while the proxy becomes the only component responsible for credential management. Secrets never become part of the execution environment, eliminating an entire class of exfiltration attacks.
Requests leave the sandbox with no credentials and pick them up at the proxy, outside the isolation boundary, so no secret ever exists inside. Additionally, prompt-injected request aimed at an unapproved host never gets past our strict egress allowlist.
Network Isolation
Protecting the sandbox requires controlling traffic in both directions.
Inbound traffic is restricted through an allowlist that limits which systems are permitted to communicate with running sandboxes. This protects the control plane and prevents unauthorized access to active execution environments.
Outbound traffic is equally important. By default, agents may communicate only with explicitly approved destinations. Restricting egress prevents arbitrary data exfiltration and ensures authenticated requests pass through the credential-injecting proxy rather than bypassing it directly. Both controls are necessary. Inbound filtering prevents unauthorized access into the sandbox; outbound filtering prevents unauthorized data from leaving it.
Data plane: reproducibility without sacrificing fidelity
A faithful execution environment is only useful if the data inside it is equally representative. Real applications depend on databases, caches, analytics stores, and application state that cannot realistically be recreated from scratch for every execution. At the same time, every agent run must begin from an identical starting point so experiments remain reproducible and independent of one another. The data plane exists to satisfy both requirements simultaneously.
Each sandbox is provisioned with representative datasets before an agent ever begins execution. Depending on the workload, this may include a transactional PostgreSQL database, Redis for application state, and analytical stores such as DuckDB or externally hosted OLAP systems accessed through a restricted network policy. Rather than asking the agent to construct its own environment, it arrives in a system that already resembles the one it is expected to operate on.
Naively restoring these datasets for every execution quickly becomes prohibitively expensive. Even moderately sized databases require seconds, or minutes, to reload from backups, placing that work directly on the critical path of every request. The latency is particularly wasteful because the overwhelming majority of data remains unchanged across executions.
Instead, every sandbox begins from a shared reference snapshot. Rather than physically copying the underlying database, the data plane creates a database branch for each execution. The branch initially shares all storage with the reference dataset, allocating new storage only for pages modified during the run. From the agent's perspective, the database is a completely independent, fully writable instance. Underneath, only the differences consume additional storage.
Every run starts from the same reference database, instantly. Instead of copying it, each run gets its own branch that shares the original data and stores only the pieces it actually changes. So every run behaves like a full, private database, the extra storage stays tiny, and cleanup is just deleting the branch.
This approach changes the economics of reproducibility. Fresh databases are no longer something that must be restored, they are effectively created instantaneously. Every execution begins from the same reference state, agents are free to modify their local copy without affecting other runs, and cleanup becomes as simple as discarding the branch when execution completes. Reproducibility, isolation, and low startup latency all emerge from the same underlying mechanism rather than requiring separate systems.
Before a sandbox is handed to an agent, the data plane performs a final readiness check across every required service. A virtual machine that has successfully booted but whose supporting databases are still initializing is functionally no different from a failed environment. Verifying service health before execution ensures failures occur deterministically during provisioning rather than unpredictably halfway through an agent trajectory.
Pareto frontier between control, execution, security and data planes
A normal sandbox is stuck on the inner frontier, so making it more realistic means giving up isolation or speed. The mechanisms in this post push the frontier outward, letting the workspace be realistic, isolated, and fast all at once. Database branching then means every run still starts from the same point.
Every design decision in this post trades between these four planes, and the tension is sharpest between fidelity and isolation. Giving agents access to container tooling dramatically improves fidelity. Agents can build images, launch services, and test changes against realistic infrastructure. But the same capabilities expand the security boundary. Docker daemons, nested workloads, and privileged operations introduce attack surfaces that require careful isolation and continuous hardening.
A conventional sandbox settles this tension by giving something up. It trades realism for safety, or speed for determinism. We built the workspace to refuse that trade. Kernel-level isolation contains the Docker daemon without shrinking what it can do, warm pools remove the latency cost of strong isolation, and database branching makes determinism effectively free. Each mechanism pushes the frontier outward rather than picking a compromised point along it. The challenge is preserving developer-level capability without ever allowing that capability to escape the sandbox boundary.
Building reliable agent infrastructure
A typical execution begins when the control plane receives a request and selects an execution environment. If a compatible warm sandbox is available, the agent attaches immediately. Otherwise, the control plane provisions a new sandbox from a versioned snapshot, initializes its services, performs health checks, and hands the environment off once it reaches a ready state.
From that point forward, execution occurs entirely inside the sandbox. The agent interacts with the filesystem, launches containers through Docker, queries reference databases, invokes MCP servers, and executes arbitrary code exactly as it would on a developer workstation. Throughout the run, outputs (including model responses, tool invocations, and execution events) are streamed back to the client while simultaneously being persisted as a complete execution trace.
That trace serves two purposes. Operationally, it provides the live stream that powers the interactive developer experience. Architecturally, it becomes a durable artifact of the run, enabling replay, debugging, auditing, and offline evaluation long after the sandbox itself has been destroyed.
This distinction matters because reproducibility requires more than recreating an environment. A deterministic starting state tells us where execution began; an execution trace tells us what happened afterward. Reliable agent infrastructure requires both: the ability to recreate the conditions of a run and the ability to understand the trajectory that emerged.
A sandbox run is therefore not simply a place where an agent executes. It is a controlled experiment with a known starting state, observable actions, and measurable outcomes.
This separation of responsibilities is intentional. The control plane manages the lifecycle, the execution plane provides a faithful runtime, the security plane enforces isolation and the data plane guarantees reproducibility. Each system remains narrowly focused, but together they produce an execution environment that is fast, scalable, realistic, reproducible, and safe.
The future of reliable agents will not come from limiting what they can do. It will come from building environments where they can do more safely.