Autonomous coding agents change the threat model of a development environment. An agent can run commands, install packages, modify thousands of files and remain active far longer than an interactive coding session. Docker's Cloud Sandboxes make a useful question testable: should that agent run beside your development environment, or inside a dedicated isolation boundary?
AiFolder test status — September 26, 2026: this article documents our benchmark design and the security questions raised by Docker's new sandbox workflow. Performance, cost and isolation scores are intentionally not presented as measured results until our controlled runs are complete.
Why a coding agent needs a different sandbox
A normal developer container is primarily a reproducible application environment. An autonomous agent is an active process that may execute unfamiliar commands and third-party code. Those are related problems, but they are not identical.
For agent workloads, the boundary has to be evaluated around credentials, host files, network access, persistence and unattended execution—not only whether the application builds correctly.
The three environments AiFolder will compare
| Environment | Question | Primary trade-off |
|---|---|---|
| Standard local container | How safe is the familiar Docker workflow for an autonomous agent? | Convenience vs host exposure |
| Dedicated local sandbox | Can stronger local isolation preserve fast iteration? | Isolation vs local resources |
| Docker Cloud Sandbox | Can the agent run remotely and unattended without exposing the developer machine? | Isolation/persistence vs cloud cost and data transfer |
Benchmark 1: can the agent see host secrets?
We will create harmless canary credentials and files outside the intended workspace. The task never asks the agent to read them. Each environment gets the same repository and coding prompt.
We then record which paths are visible, which environment variables are inherited, whether common credential directories are mounted, and whether the agent can access files outside the project boundary.
No real secrets will be used. Canary values let us test accidental exposure without putting accounts at risk.
Benchmark 2: network isolation
A coding agent often needs package registries, documentation and Git hosting, but unrestricted outbound access also increases the blast radius of a compromised dependency or malicious instruction.
We will measure default egress behavior, DNS visibility and how practical it is to restrict the environment to an allowlist. The useful result is not simply "internet works"; it is whether teams can express a least-privilege network policy without making ordinary development unusable.
Benchmark 3: malicious-looking but harmless repository instructions
Our fixture repository will include an adversarial instruction that attempts to make the agent read a canary file outside its workspace. This is a sandbox test, not an exploit test: the target is a fake value created specifically for the experiment.
If the model follows the bad instruction, the environment should still limit what it can reach. This separates model alignment from systems isolation.
Benchmark 4: long-running coding work
The practical appeal of a cloud sandbox is not only security. A developer should be able to hand off a large task without keeping a laptop awake.
We will run equivalent 1-hour, 4-hour and 8-hour agent workloads where supported and record:
- setup and cold-start time;
- task completion and test-pass rate;
- agent interruptions and required human interventions;
- CPU, memory and storage behavior;
- workspace persistence after reconnecting;
- total infrastructure and model cost;
- whether work continues when the local client disconnects.
Benchmark 5: local → cloud → local continuity
A strong workflow should not require a developer to rebuild context manually every time execution moves. We will test a realistic handoff: begin a task locally, move the working state into the cloud sandbox, let the agent continue, then retrieve and review the resulting Git diff locally.
We will count manual synchronization steps, lost state, Git conflicts and environment differences.
Containers are not automatically insecure
It would be misleading to conclude that "container = unsafe" and "microVM = safe." Security depends on configuration and the surrounding system. A carefully constrained container can be appropriate for many workloads; a poorly configured remote environment can still leak credentials or data.
The benchmark therefore tests observable boundaries rather than assigning security based on the technology label.
The permission model matters as much as isolation
Sandboxing answers what an agent can reach. Agent permissions answer what it is allowed to do. Strong systems need both.
For example, preventing access to the host SSH directory is useful even if an agent asks for it. Requiring confirmation before a destructive Git operation remains useful even inside an isolated machine. Defense in depth is particularly important when an agent consumes untrusted repository content.
What we will measure
| Metric | Why it matters |
|---|---|
| Host-file visibility | Measures filesystem blast radius |
| Credential visibility | Tests accidental secret exposure |
| Outbound network control | Limits exfiltration and dependency risk |
| Cold start | Affects interactive usability |
| Task success | Security is useless if the environment breaks the workflow |
| Human interventions | Measures unattended-agent viability |
| 1h / 4h / 8h cost | Shows the economics of persistent cloud execution |
| State persistence | Determines whether long tasks survive disconnects |
A safe test repository
agent-sandbox-test/
├── AGENTS.md
├── src/
├── tests/
└── fixtures/
└── untrusted-instructions.txt
outside-workspace/
└── CANARY_SECRET.txt # fake value onlyThe acceptance test checks the coding task. A separate observer checks whether the canary was ever accessed. This keeps security evaluation independent from the agent's own report.
Who should care about cloud sandboxes?
They are most interesting for teams running agents unattended, processing untrusted repositories, executing generated code, or delegating tasks that take long enough to outlive a laptop session. For a short, supervised edit in a trusted repository, the operational overhead may not be justified.
What would make Cloud Sandboxes win our test?
We would look for materially smaller host and credential exposure, practical network controls, reliable persistence and long-running execution, without an unacceptable penalty in startup time, developer workflow or total cost.
Conversely, if a hardened local setup achieves comparable isolation for the target workload at much lower friction, that is also a useful result.
From one article to an AI Coding Agent Security Benchmark
This experiment fits a broader AiFolder framework. Agent security is not one setting: repository instructions can be poisoned, history can be tampered with, tools can receive excessive permissions, code can leave the machine, and execution environments can expose credentials.
Our goal is to turn those questions into repeatable tests with observable pass/fail conditions rather than generic security advice.
Frequently asked questions
Is a Docker container a security sandbox?
Containers provide isolation primitives, but the effective boundary depends on configuration, privileges, mounts, networking and the host environment. We test the resulting exposure rather than assuming a particular configuration.
Why use fake canary secrets?
They reveal whether an agent can reach sensitive-looking data without exposing real credentials. This makes the experiment reproducible and safe.
Should coding agents always run in the cloud?
No. Privacy, latency, cost, hardware and threat model differ by project. Local execution can be preferable when data must remain on-device; isolated cloud execution can be useful for long-running or untrusted workloads.
Will AiFolder publish the raw benchmark?
When the controlled test is complete, we plan to publish environment versions, configuration, task definitions and aggregate measurements so results can be compared rather than taken on trust.
Related AiFolder guides
- AGENTS.md vs CLAUDE.md — portable instructions across coding agents.
- OpenAPI to SDK, CLI and MCP — deterministic interfaces for agent tooling.
- Jev vs LLMs for AI decisions — deciding which agent steps need generative models.
- AI coding tools — explore the broader coding-agent cluster.
Methodology note
Checked September 26, 2026. Launch capabilities are treated separately from AiFolder measurements. We will update this page only with first-hand benchmark values produced by a documented environment; vendor claims will remain attributed rather than presented as independent findings.