Short version for the impatient: before you compare a single Python sandbox library, run ls /dev/kvm on the machine that will actually execute the code. If that file isn’t there, half the options on your shortlist are already out, and you will find that out at runtime instead of at design time.
I learned this the annoying way. I had picked a sandbox, written the integration, tested it locally on a Linux box, and then watched it fail on the platform I was deploying to with an error message that took me an embarrassing amount of time to understand. The sandbox wanted hardware virtualisation. The host was itself a virtual machine with nested virtualisation switched off. Nothing about my code was wrong. The isolation model I’d chosen simply could not exist there.
This keeps happening to people, and the reason it keeps happening is that the discourse around running model generated code is entirely about libraries and not at all about substrate.
The three boundaries, and what each one actually stops
There are basically three isolation strengths available to you, and they are not close to equivalent.
A language level sandbox is the weakest. Restricted builtins, an AST allowlist, RestrictedPython, that whole family. These stop typos. They do not stop an adversary and they have never stopped an adversary. CPython has too many escape hatches through introspection, and every few years someone publishes another chain of __subclasses__ calls that gets you back to os.system. If a language model wrote the code, the code is at minimum untrusted input. Treat it accordingly.
A container is the middle. Namespaces, cgroups, seccomp, a dropped capability set. This is a real boundary and for most workloads it is enough. It is also a shared kernel boundary, which means a kernel bug is a full escape. gVisor sits slightly above plain containers by putting a user space kernel in the path, at a real syscall performance cost.
A microVM is the strongest of the three. Firecracker or libkrun or QEMU, hardware enforced, separate kernel, no shared syscall surface with your host. This is what AWS Lambda runs on, and it is what most of the newer AI sandbox products are built on underneath.
The microVM tier is the one that requires /dev/kvm. That is the whole catch.
The check that should be step zero
Put this in your setup script and your CI, not in a comment.
# Can this host do hardware virtualisation at all?
[ -e /dev/kvm ] && echo "kvm: yes" || echo "kvm: NO"
# And does the CPU even expose the flags?
grep -oE 'vmx|svm' /proc/cpuinfo | sort -u
Empty output on that second command means you are inside a guest without nested virtualisation. Plenty of managed environments are exactly that. Simon Willison ran this exact wall in August when he pointed Claude Fable at smolmachines to evaluate it as a sandbox. The agent’s own environment notes are worth quoting because they are so specific:
This Claude Code container: Linux 6.18.5-fc-v20 (itself a Firecracker guest), 4 vCPU, 15GB RAM. No /dev/kvm, no vmx/svm CPU flags, no nested virt.
A Firecracker guest that cannot start Firecracker guests. The workaround it found was to run the test battery on GitHub Actions ubuntu runners, which do expose /dev/kvm. That is a genuinely useful piece of operational trivia and I have written it on a sticky note.
The point generalises. Your laptop, a bare metal Hetzner box and a GitHub Actions runner will give you KVM. A container inside someone else’s managed platform very often will not. If your sandbox tier and your deployment target disagree, you have a design problem, not a bug.
What the smolvm testing actually found
I like numbers more than architecture diagrams, so here are the ones from that research writeup on smolvm 1.8.3.
Cold start landed between roughly 0.6 and 1.5 seconds. Warm execution came in around 50 milliseconds. Offline local images worked, no network execution worked, CPU and RAM limits held, guest enforced timeouts held, storage quotas held, read only input mounts and writable output mounts behaved, and --unprivileged did what it says.
That cold start number is the one to sit with. Sub second is fine for a data transformation job a user kicked off. It is not fine if you were imagining a sandbox per tool call inside a chat loop with a human waiting. If you want that, you pool warm VMs and eat the memory, which is a completely different cost model and worth deciding on before you write the integration rather than after.
smolvm itself presents a unified API over Firecracker, QEMU and libkrun, and there’s a smolmachines Python package if you want to embed the thing directly rather than shell out to a binary.
What the container tier looks like when you actually do it properly
Most people who say “I run it in Docker” are running docker run python:3.12 python script.py and calling it isolation. That container has network access, runs as root inside the namespace, can allocate until the host OOM killer intervenes, and has a writable root filesystem. It is not a sandbox. It is a convenient way to install Python.
The version that is defensible looks more like this, and the flags are the entire point:
docker run --rm \
--network=none \
--read-only \
--tmpfs /tmp:rw,noexec,nosuid,size=64m \
--memory=256m --memory-swap=256m \
--cpus=0.5 --pids-limit=64 \
--cap-drop=ALL --security-opt=no-new-privileges \
--user 65534:65534 \
-v "$PWD/in:/in:ro" -v "$PWD/out:/out" \
python:3.12-slim timeout -s KILL 5 python /in/task.py
Every one of those flags is closing a specific hole. --network=none kills exfiltration. --read-only plus a noexec tmpfs means the code can write scratch files but can’t drop and run a binary. --pids-limit stops fork bombs, which people forget about entirely and which will take down a host faster than a memory leak. --cap-drop=ALL and no-new-privileges remove the usual escalation paths. Running as nobody means a filesystem bug lands you somewhere useless.
The timeout wrapper is deliberately on the outside. If the limit lives inside the guest process, the guest process gets to decide about it.
That is a real boundary against opportunistic bad code. It is still one kernel bug away from a full escape, which is the honest reason the microVM tier exists at all. But it takes about six flags and ten minutes, and it is a very large improvement over what most teams are actually shipping.
The failure everyone forgets to test
Most sandbox writeups obsess over filesystem escape. In practice the thing that has actually bitten me is much dumber.
Resource exhaustion. A model writes a loop that never terminates, or allocates in a way that looks fine on the sample input and is quadratic on the real one. No escape, no exploit, just your host swapping itself to death while your queue backs up. Your sandbox needs a hard CPU time limit, a hard memory ceiling and a wall clock timeout enforced from outside the guest. Enforced inside the guest is not enough, because the code you are worried about is the code deciding whether to respect it.
Egress is the other one. A sandbox with network access is not a sandbox, it is a proxy with extra steps, and it turns any prompt injection into data exfiltration. Default to no network. Add an explicit allowlist only when a task genuinely needs one, and log every allowed destination.
That second failure mode is close cousin to something I wrote about in the post on agent tool schemas: the boundary you drew on the diagram and the boundary that exists at runtime are frequently different objects, and the gap is where the interesting incidents live.
Picking, without pretending there’s one answer
Here is roughly how I decide now.
If the code comes from a model but the inputs come from me, and it runs on my own hardware, a locked down container is proportionate. Read only root, no network, dropped capabilities, cgroup limits, a non root user. Cheap, fast, boring, sufficient.
If untrusted users can influence what gets executed, I want a microVM, which means I need a host with KVM, which usually means bare metal or a VM instance type that permits nested virtualisation. Hetzner dedicated boxes do. Plenty of managed container platforms don’t. Check before you architect, not after.
If I need it inside a platform that gives me neither, the honest options are to move the execution somewhere else, or to accept a hosted sandbox service and the data handling questions that come with it. Pretending a language level sandbox closes that gap is how people end up with incidents.
There’s a fourth answer that is unfashionable and often correct: don’t execute the code. A surprising number of “let the agent run Python” features are really “let the agent do arithmetic and reshape a table”, and a constrained expression evaluator over a fixed set of operations solves it without any of this. I’ve talked several clients out of a sandbox entirely and into a much smaller surface. Most of the work I do ends up being that kind of subtraction.
Try this today
Take whatever pipeline you have that executes model generated code, and run three tests against it.
First, while True: pass with a five second budget. Does anything actually kill it, and how long does it really take?
Second, a script that opens a socket to a host you control. Does the packet arrive? If yes, your sandbox has network egress and you probably didn’t mean it to.
Third, allocate memory in a loop. Watch what happens to the host, not the guest.
If all three behave, you have a sandbox. If any of them don’t, you have a container with optimistic naming. And run ls /dev/kvm on the production host while you’re in there, because the answer determines which of these problems you are even allowed to solve.
