local-vm.md

Local microVM runtime

berth dev, berth mcp and berth rpc run your app in a small virtual machine. The VM boots Berth's own pinned Linux kernel and a read-only base image, so the kernel that enforces your app's capabilities is the same on every machine, with Landlock always built in. No Docker daemon is involved, and no image is built.

Since 0.3.1 it is the default wherever it can run: Apple silicon Macs, and Linux on arm64 or x86_64 with KVM. Elsewhere (Intel Macs, Windows, Linux without /dev/kvm) the sandbox is a Docker container, and --runtime, BERTH_SANDBOX or "sandbox" in ~/.berth/config.json choose either one explicitly. The VM runs Node and Python apps that use the filesystem, /context, secrets, declared network hosts, the terminal, the browser and the GitHub broker. See Limits for what it doesn't cover yet. berth test and berth deploy build a container image, so they use Docker.

	--runtime vm (default where it can run)	--runtime docker
Kernel that enforces the policy	Berth's pinned kernel 6.12.109, Landlock built in	the Docker host's (Docker Desktop: no Landlock; Colima: yes)
What runs your app	your bundled app on a read-only share, in a pinned erofs rootfs	an image built from your project
Network	the egress broker in the guest, plus a host-side dialer that enforces the network:host: allowlist again on the host. The VM itself has no network device	the egress broker
First tools/call after berth mcp starts	about 0.7 s	about 1.6 s with a warm image
Reload after a code change	VM reboot, about 0.55 s	container restart

Context

You run the berth CLI on your own machine, and it boots a VM for each sandbox. The systems it touches are the host's hypervisor (Hypervisor.framework on macOS, KVM on Linux) through libkrun, the GitHub release that berth vm install downloads the pinned kernel, rootfs and berth-vmm from, and the hosts an app declares under network:host:. An MCP client reaches the app through berth mcp, as it does with Docker.

Requirements

• macOS on Apple silicon (Hypervisor.framework), or Linux with KVM (/dev/kvm readable and writable).
• libkrun 1.19.6, which berth vm install puts beside the published berth-vmm on every platform: built from its pinned tag by CI, checked against its sha256, and loaded first by berth-vmm (LC_RPATH @loader_path on macOS, RUNPATH $ORIGIN on Linux). Nothing to install yourself. Only a berth-vmm you build from source links a system libkrun instead: on macOS, Homebrew's (brew tap libkrun/krun && brew install libkrun, at 1.19.6).
• berth-vmm, the small launcher that runs one VM per process. berth vm install downloads it for macOS arm64, Linux arm64 and Linux x86_64; the CLI pins all three since 0.3.0. Otherwise build it from packages/vmm with cargo build --release. On macOS it has to be signed with the hypervisor entitlement. The published build is, and the build scripts sign a local one. berth doctor prints the codesign command if it isn't. The Linux builds need glibc 2.39 or later (Ubuntu 24.04, Debian 13, Fedora 40).
• The pinned kernel and rootfs, about 100 MB, installed with berth vm install.

Containers

berth dev (Run an app) brings these pieces up.

What happens:

• Bundling. The guest has Node but no node_modules, so the app is bundled with esbuild: dist/index.mjs (your app with @berthos/sdk, zod and your other dependencies inlined), runtime.mjs (the SDK's runtime), proto/context_bus.proto, and your berth.yml. esbuild and the SDK come from your project if it has them installed, or from the CLI's own dependencies if it doesn't. A berth init project works before npm install. Bundles are cached by content under ~/.berth/vm/apps.
• The VM. One berth-vmm process with 2 vCPUs and 512 MiB, no network device, and TSI off. Its run directory, ~/.berth/run/vm/<name>/, holds the sockets for the control, log and RPC ports, plus berth-vmm.pid, vm.json, vmm.log and console.log. berth-vmm runs detached, so berth mcp, berth rpc and berth vm commands find it again by its run directory. If its process is gone or no longer belongs to that run directory, the run directory is treated as stale and cleaned up.
• Inside. berth-init is PID 1. It compiles your capability policy, runs your app as its own uid in its own cgroup under agent-init (Landlock, seccomp), and starts the context bus. See the design notes.
• /workspace is on a per-app state disk, ~/.berth/vm/state/<app>.img. It is created sparse at 1 GiB and kept across reloads and sessions, like the Docker path's named volume.

Components

Inside the guest, berth-init does the work a container's entrypoint does. As described in the sections below, it:

• reads the secrets disk as root before anything starts, and gives each app only its own values (Secrets and other variables);
• compiles each app's capability policy, hashes it for berth attest, and runs the app as its own uid in its own cgroup under agent-init (Landlock, seccomp);
• starts the context bus and, as the apps' capabilities call for them, semantic-fs-daemon, the egress broker, the GitHub API broker, and the browser layer's Xvfb, x11vnc and noVNC (Limits);
• relays a published guest port, such as the terminal's 7681, to the host over vsock.

On the host, berth-vmm carries the kernel and rootfs pins it was built with and refuses to boot anything else (Install). It hashes each app share before the VM starts, confines itself under a Seatbelt profile on macOS, and runs the egress dialer that checks every connection against --egress-allow. See the design notes and the guest init design.

Code

Install

There's no separate step: the first berth dev or berth mcp that boots a VM installs everything below, the same way berth vm install does, and a later boot after a CLI upgrade replaces an older release's berth-vmm. To install ahead of time, for example before going offline or in a provisioning script:

berth vm install
berth doctor --sandbox vm

berth vm install puts the kernel and rootfs in ~/.berth/vm, where berth-vmm looks for them. It also puts the published berth-vmm in ~/.berth/vm/bin when none is found, and replaces the one there when it is an older release's (built for a different rootfs). A local build that boots this CLI's kernel and rootfs is kept; --force replaces it. Each file is checked against its sha256 pin before it is put in place, and a file that doesn't match is refused. The kernel and rootfs pins are the ones berth-vmm was built with. The CLI reads them out of the binary, because berth-vmm refuses to boot anything else. If there is no berth-vmm yet, it uses its own copy of the same pins.

What is downloaded, and from where

By default, from the GitHub release that .github/workflows/vm-artifacts.yml publishes for your architecture's kernel and rootfs pair, vm-artifacts-<first 8 hex of the kernel pin>-<first 8 of the rootfs pin>. Each guest architecture (aarch64 on Apple silicon and arm64 Linux, x86_64 on x86_64 Linux) has its own pins, in packages/vmm/{kernel,rootfs}/manifest-<arch>.toml, and so its own release. The aarch64 one:

File	Asset	sha256 (this CLI)	Size
~/.berth/vm/kernel/sha256/<sha>/Image	Image-<sha>	8f79e8dae97ebc0ab8fcdc4ad209bb025ec967be82c713503e0612cfdd340ec8	23,668,744 B
~/.berth/vm/rootfs/rootfs-<sha>.erofs	rootfs-<sha>.erofs	dfb5f6b030ca59884f2466783961169dd2f4d6fb71ff08d30028f67eff29331f	108,072,960 B
~/.berth/vm/bin/berth-vmm	berth-vmm-darwin-arm64-<sha>, berth-vmm-linux-arm64-<sha>	VMM_PINS in packages/cli/src/vm/pins.ts	about 700 KB
~/.berth/vm/bin/libkrun.1.19.6.dylib, libkrun.so.1.19.6	libkrun-darwin-arm64-<sha>.dylib, libkrun-linux-arm64-<sha>.so	LIBKRUN_PINS in the same file	about 4.7 MB

That is https://github.com/Ash20pk/BerthOS/releases/download/vm-artifacts-8f79e8da-dfb5f6b0/. The x86_64 one is vm-artifacts-10d4b54c-af4f3fc0: kernel 10d4b54c0896668a93f9d26d0a701aac4a14ce813f3ed4d3b4d26e50d7b7b8d1 (an ELF vmlinux, 27,928,992 B), rootfs af4f3fc0be1429cf324ffc77157c1c2359c641078675c1d6d0d82b746259ac69 (108,904,448 B), and berth-vmm-linux-x64-<sha> with libkrun-linux-x64-<sha>.so. CI rebuilds each architecture's kernel and rootfs from source on GitHub's Linux runners of that architecture, and publishes the release only if they match the pins bit for bit. The release also carries SHA256SUMS, each rootfs's input record, berth-vmm's build record, and the kernel's GPL sources (the exact linux and libkrunfw tarballs, the config delta, the resolved config and the build script; see SOURCES.md there).

berth-vmm and its libkrun are downloaded only when the CLI pins their sha256 for your platform (macOS arm64, Linux arm64 and Linux x86_64). Every platform is pinned since 0.3.0. A CLI built from a checkout whose pins have no published build yet says so; then build berth-vmm and pass --vmm, or download it by hand (below).

A download is checked against the pinned size as it streams, and against the sha256 before it's renamed into place, so nothing unverified is ever at the final path. A download that receives nothing for 30 seconds is aborted and retried, up to four attempts, resuming with an HTTP range request where the server supports one (GitHub's CDN does) and starting over where it doesn't. BERTH_VM_DOWNLOAD_STALL_MS changes the 30 seconds.

Sources, in order

1. A local directory: --from <dir>, or BERTH_VMM_ARTIFACTS. It can be the $BERTH_VMM_ARTIFACTS directory that packages/vmm's build scripts write (kernel/sha256/<sha>/Image, rootfs/rootfs-<sha>.erofs), a flat directory, one subdirectory per sha256, or a directory of downloaded release assets. On APFS the copy is a clone, so it takes no extra space.

2. A download, for anything the directory doesn't have. The URL is a template:

   Placeholder
   {asset}	the release asset name: Image-<sha>, rootfs-<sha>.erofs, berth-vmm-darwin-arm64-<sha>, libkrun-darwin-arm64-<sha>.dylib
   {kernel8}, {rootfs8}	the first 8 hex digits of the kernel and rootfs pins (the release tag); {kernel} and {rootfs} are the full pins
   {kind}, {sha256}, {file}	kernel, rootfs or vmm; the artifact's own pin; its file name in ~/.berth/vm (Image, rootfs-<sha>.erofs, berth-vmm)

   The default is https://github.com/Ash20pk/BerthOS/releases/download/vm-artifacts-{kernel8}-{rootfs8}/{asset}. Set a mirror with --url, BERTH_VM_ARTIFACTS_URL, or vm.artifactsUrl in ~/.berth/config.json. file:// works too, for example --url 'file:///Volumes/usb/vm-artifacts/{asset}'. A mirror needs no trust, because every file is checked against the pin.

--no-download copies from --from only. --vmm <path> copies that berth-vmm to ~/.berth/vm/bin/berth-vmm instead of downloading one. The signature is part of the binary, so it carries over. Otherwise the CLI looks for berth-vmm in BERTH_VMM, then ~/.berth/vm/bin, then PATH, then the checkout's packages/vmm/target/release.

If anything isn't installed, berth dev installs it the first time it runs, from the same sources.

Verifying by hand

tag=vm-artifacts-8f79e8da-dfb5f6b0
gh release download "$tag" -R Ash20pk/BerthOS -p 'Image-*' -p 'rootfs-*.erofs' -p 'berth-vmm-darwin-arm64-*' -p 'libkrun-darwin-arm64-*' -p SHA256SUMS
shasum -a 256 -c --ignore-missing SHA256SUMS     # each asset against the list
shasum -a 256 Image-* rootfs-*.erofs              # and against the pins above, which are also in
                                                  # packages/vmm/{kernel,rootfs}/manifest-<arch>.toml
berth vm install --from .                         # installs from this directory, checked again

SHA256SUMS comes from the same release, so it only catches a damaged download. The pins are what you trust: the table above, the manifests in the repository, and the copy compiled into berth-vmm (berth doctor --sandbox vm shows them).

berth-vmm is ad hoc signed, not notarized. A copy you download with a browser is quarantined, and macOS won't run it until you clear that, after checking its sha256:

shasum -a 256 berth-vmm-darwin-arm64-*            # must equal the name's hash and the CLI's pin
xattr -d com.apple.quarantine berth-vmm-darwin-arm64-*

berth vm install doesn't need this. A file it downloads isn't quarantined, and it clears the attribute only on a file whose sha256 matched the pin.

Building from source

The same scripts CI runs, from packages/vmm (a libkrun builder VM on macOS, a container on Linux):

cd packages/vmm
./scripts/build-kernel.sh       # checks the Image against kernel/manifest-<arch>.toml
./scripts/build-agent-init.sh   # agent-init + probe, checked against rootfs/manifest-<arch>.toml
./scripts/build-berth-init.sh   # berth-init + context-bus-daemon, checked likewise
./scripts/build-rootfs.sh       # the erofs image, checked likewise
cargo build --release           # berth-vmm, linked against the system libkrun 1.19.6 (the scripts also sign it)
berth vm install --from "$BERTH_VMM_ARTIFACTS" --vmm target/release/berth-vmm

Each script fails on a hash that isn't its pin and prints both. The builds are reproducible while Alpine 3.24 still serves the package versions in kernel/<arch>.apk.lock, guest/*.<arch>.apk.lock and rootfs/<arch>.apk.lock (<arch> is the host's: aarch64 or x86_64), which is how long the pins can be rebuilt bit for bit. See Distribution.

Run an app

cd apps/notes
berth dev

VM ready in 416 ms (bundle 9 ms, cached, boot 407 ms). boot 353f3798-…, berth-vmm pid 28633
[berth:dev] run dir ~/.berth/run/vm/berth-dev-notes; call it with `berth rpc notes --export <name>` …

From another terminal:

berth rpc notes --export add_note --input '{"text":"hello"}'
berth rpc notes --export list_notes
berth vm status                 # running VMs, their apps, pids, uids and cgroup limits
berth vm logs berth-dev-notes   # the guest's log: apps, berth-init, context-bus
berth vm stop berth-dev-notes

Python apps

A runtime: python app runs the same way. Nothing is bundled: the CLI copies the app's own files (not __pycache__, venv, node_modules or dot directories) into its share, and the guest starts it with the image's python3 and berth_sdk, loading src/app.py, as a container does.

Only the standard library and berth_sdk's own dependencies (pyyaml, pydantic, protobuf, from Alpine) are in the image. Nothing is pip-installed, so a package your app imports beyond those fails at boot with Python's ModuleNotFoundError in berth vm logs. Use --runtime docker for such an app. A share holds at most 2,000 files and 32 MiB.

Secrets and other variables

export GITHUB_TOKEN=...            # or keep it in a .env file
berth dev --env GITHUB_TOKEN --env GITHUB_REPO=owner/name
berth dev --env-file .env

--env and --env-file work as they do on berth os up and with Docker, and on berth mcp when it boots the VM itself. A name an app declares under secrets: in its berth.yml reaches that app alone. Any other name reaches every app in the sandbox, with a warning. A declared name with no value prints a warning naming it, and the app boots without it.

None of it goes on the guest's kernel command line, which every process in the guest can read in /proc/cmdline. The CLI writes the values to secrets.img in the run directory (0600), berth-vmm attaches it as a read-only disk, and berth-init reads it as root before anything starts, removes the device node, and puts each app's values into that app's environment only. The apps run as different uids, so one can't read another's /proc/<pid>/environ. The CLI deletes secrets.img as soon as the sandbox is ready. Values are never logged; berth-init reports the names it delivered. See the secrets reference.

Reloading

Saving a file in src/ or berth.yml rebundles. If the bundle comes out the same (you changed only a comment, say), the VM keeps running. Otherwise the VM is stopped and a fresh one boots on the new bundle:

Change detected, rebooting the VM with the new bundle...
Reloaded in 537 ms (bundle 127 ms, stop 36 ms, boot 374 ms). boot 0820e707-…

The CLI reboots the whole VM rather than restarting only the app. The app share is read-only, and berth-init has no way to restart a single app yet. A boot also recompiles the capability policy, so a berth.yml change takes effect, which an app restart wouldn't do. Your notes stay on the state disk.

MCP

berth mcp --app notes --app-dir apps/notes

The bridge works as it does with Docker (see the MCP bridge reference). If berth dev is already running the app, the bridge attaches to that VM and leaves it running when the client goes. Otherwise it boots its own VM and stops it at the end of the session. Tool calls are audited the same way, and the session records the VM's boot evidence. berth attest <runId> then reports the session with boot.isolation (see the attestation reference).

Time from spawning berth mcp to the first answered tools/call, measured by packages/cli/test/vm-e2e.mjs on an Apple M4 under load: 625 to 851 ms when the bridge boots its own VM (about 300 ms of that is the CLI starting), and about 260 ms when it attaches to a running one. The Docker path takes about 1.6 s with a warm image.

Choosing the runtime

The first of these that is set wins:

per command	--runtime vm or --runtime docker on berth dev, berth mcp, berth rpc (--sandbox on berth doctor)
per shell	export BERTH_SANDBOX=docker
per user	{"sandbox": "docker"} in ~/.berth/config.json
the host	vm on an Apple silicon Mac, or on Linux with a /dev/kvm you can open read-write, when the CLI pins a published berth-vmm for the platform; docker everywhere else

berth doctor --fix sets up Colima for the Docker path, so it checks Docker unless --sandbox vm is passed.

The variable is BERTH_SANDBOX, not BERTH_RUNTIME. BERTH_RUNTIME already selects the container runtime Docker uses, such as runsc for gVisor.

~/.berth/config.json also takes vm.artifactsUrl, vm.artifactsDir and vm.vmm.

Limits

An app whose berth.yml needs something the VM doesn't have yet is refused before boot, with the reason and a pointer back to --runtime docker:

• Network. The VM has no network device, and TSI is off. An app that declares network:host: or browser:navigate: reaches those hosts through the egress broker inside the guest, which dials out over vsock to a dialer in berth-vmm. The CLI passes your apps' scopes to berth-vmm as --egress-allow, and the dialer enforces that allowlist again on the host: it resolves names itself and refuses private, loopback, link-local and metadata addresses. One app per sandbox may declare egress, as with containers. See the egress design.
• /context persists. berth-init runs semantic-fs-daemon in the guest, so an app that declares /context works as in a container, except that /context is kept on the state disk with /workspace: it survives berth vm stop and the next boot (a container's goes when the container does, unless you snapshot it). query_context ranks by meaning, through one embeddings daemon per sandbox rather than a model in every app: the model takes about 250 MB once loaded, so a single-app sandbox that declares /context gets 768 MiB of RAM instead of 512. Needs a berth-vmm whose image has the daemon and the embeddings kit; with an older one the app is refused, or ranks by keyword. See the design.
• terminal:* works as in a container: apps/terminal's shared tmux shell (run_command, send_keys, read_screen) runs as the app's uid, in the tty group, under its Landlock rules, and berth dev prints its web view's loopback URL and a login generated for the session. The port reaches the host through berth-vmm run --publish 7681: berth-init relays vsock 2000 to 127.0.0.1:7681 in the guest, and the CLI serves the run directory's publish-7681.sock on 127.0.0.1. The login travels on the secrets disk, to that app alone. ttyd starts with the terminal's first call, as in a container.
• github:* works as in a container: berth-init starts the GitHub API broker for the one app that declares it, as its own uid under agent-init, and that app alone is told to use it and to trust its CA. The broker reaches api.github.com through the egress dialer, so the CLI adds api.github.com:443 to --egress-allow. Needs a berth-vmm whose image has the broker. See GitHub API scoping.
• browser:navigate: is a host pattern for the egress broker, like network:host:.
• The other browser: capabilities (apps/browser-native) work through the browser layer, an optional 456 MiB image that isn't part of berth vm install. The first sandbox that needs it downloads it (the browser layer (456 MB) is needed by browser-native; installing it once), checked against its pin like the rest; berth vm install --layer browser fetches it ahead of time. berth-init lays it over the base read-only and runs Xvfb, x11vnc and noVNC as their own confined uid; Chromium comes from the layer and goes out through the egress broker. berth dev prints noVNC's loopback URL and a password made for the session. A browser sandbox gets 2 GiB of RAM. See the layers design.
• Other <service>:* capabilities have no broker in the VM yet.
• Native addons (.node files) can't run in the guest, because they were built for your host, not for Linux on arm64. Bundling stops with that error.
• Files your app reads from its own directory at run time, beyond berth.yml, aren't in the share. Only the bundle is.
• Architecture. A guest is its host's architecture: aarch64 on Apple silicon and arm64 Linux, x86_64 on x86_64 Linux, each with its own pinned kernel and rootfs. Intel Macs aren't supported (libkrun has no backend there). CI boots the x86_64 guest under KVM on every artifacts build; GitHub's arm64 runners have no KVM, so Linux arm64 is built the same way but booted only on machines that have it.
• berth-vmm is confined too. libkrun runs the VMM and the guest as one security context, so berth-vmm run confines itself to this sandbox's files (and the network only for the egress dialer): a Seatbelt profile on macOS; on Linux, Landlock (filesystem, TCP from Linux 6.7, abstract sockets and signals from 6.12) and a seccomp filter that refuses exec, ptrace, mounts, namespaces, module loading, bpf, keyrings and io_uring. A Linux kernel without Landlock can't run it confined, and berth-vmm stops rather than run unconfined. berth attest records it as boot.isolation.hostSandbox; --no-host-sandbox turns it off for debugging.
• /app is measured at boot, and it stays live. berth-vmm hashes each app share before the VM starts and records it as boot.isolation.apps (the sha256 of its sorted file listing). The share is virtio-fs, so a change after boot isn't in that digest; berth dev reboots on a new bundle, which measures it again.
• Policy digests are berth-init's: it hashes each app's compiled policy, the bytes agent-init reads, and berth attest records them under policies, as for a container.

Troubleshooting

berth doctor --sandbox vm checks everything the VM runtime needs, without contacting Docker, and prints a fix for each failure:

  ✔ Hypervisor (HVF)
  ✔ berth-vmm                          ~/.berth/vm/bin/berth-vmm
  ✔ berth-vmm hypervisor entitlement   signed with com.apple.security.hypervisor
  ✔ berth-vmm's pinned kernel and rootfs
  ✔ libkrun                            1.19.6
  ✔ Pinned kernel and rootfs in ~/.berth/vm

• berth-vmm hypervisor entitlement fails: run the codesign --sign - --force --entitlements ~/.berth/vm/berth-vmm.entitlements <berth-vmm> command doctor prints. berth vm install writes the entitlements file.
• The boot fails: the error includes berth-vmm's last lines. ~/.berth/run/vm/<name>/console.log has the guest console, and vmm.log has berth-vmm's own output, including the measurement line.
• "<name>" is already running: another berth dev owns that VM. Stop it there, or run berth vm stop <name>.