Run microsandbox in Kubernetes

​

Prerequisites

• A Linux worker with usable /dev/kvm; virtual workers need nested virtualization.
• A namespace that permits privileged pods and host device mounts.
• An application image matching the worker’s architecture: amd64 or arm64.
• Docker, kubectl access, and a registry the worker can pull from.
• Pod egress for downloading the guest image.

ls -l /dev/kvm

​

Build the application

import asyncio
import os
from pathlib import Path

from microsandbox import Sandbox, Volume


async def main():
    workspace = Path(os.environ.get("WORKSPACE_DIR", "/workspace")).resolve()
    workspace.mkdir(parents=True, exist_ok=True)
    (workspace / "input.txt").write_text("hello from the pod\n")

    async with await Sandbox.create(
        "example",
        image="python:3.12-slim",
        cpus=1,
        memory=512,
        volumes={"/workspace": Volume.bind(str(workspace))},
    ) as sandbox:
        result = await sandbox.exec(
            "sh", ["-c", "tr a-z A-Z < /workspace/input.txt > /workspace/output.txt"]
        )
        if result.exit_code != 0:
            raise RuntimeError(result.stderr_text)

    print((workspace / "output.txt").read_text(), end="")


asyncio.run(main())

FROM python:3.12-slim-bookworm

RUN pip install --no-cache-dir --only-binary=:all: microsandbox==0.7.6

ENV PYTHONUNBUFFERED=1 \
    MSB_HOME=/msb \
    WORKSPACE_DIR=/workspace

WORKDIR /app
COPY agent.py .

CMD ["python", "agent.py"]

docker buildx build --platform linux/arm64 \
  -t REGISTRY/PROJECT/msb-agent:0.7.6 --push .

​

Run the pod

kubectl label node YOUR_KVM_NODE microsandbox.dev/kvm=true
kubectl create namespace msb-example

apiVersion: v1
kind: Pod
metadata:
  name: msb-agent
  namespace: msb-example
spec:
  restartPolicy: Never
  automountServiceAccountToken: false
  terminationGracePeriodSeconds: 30
  nodeSelector:
    kubernetes.io/os: linux
    kubernetes.io/arch: arm64
    microsandbox.dev/kvm: "true"
  containers:
    - name: agent
      image: REGISTRY/PROJECT/msb-agent:0.7.6
      securityContext:
        privileged: true
        runAsUser: 0
      resources:
        requests:
          cpu: "1"
          memory: 1Gi
          ephemeral-storage: 2Gi
        limits:
          cpu: "2"
          memory: 2Gi
          ephemeral-storage: 10Gi
      volumeMounts:
        - name: kvm
          mountPath: /dev/kvm
        - name: runtime
          mountPath: /msb
        - name: workspace
          mountPath: /workspace
  volumes:
    - name: kvm
      hostPath:
        path: /dev/kvm
        type: CharDevice
    - name: runtime
      emptyDir: {}
    - name: workspace
      emptyDir: {}

kubectl apply --dry-run=server -f pod.yaml
kubectl apply -f pod.yaml
kubectl -n msb-example wait --for=jsonpath='{.status.phase}'=Succeeded \
  pod/msb-agent --timeout=300s
kubectl -n msb-example logs msb-agent

HELLO FROM THE POD

kubectl delete namespace msb-example
kubectl label node YOUR_KVM_NODE microsandbox.dev/kvm-

​

Device plugins

1. Find its resource name and capacity with kubectl describe node YOUR_KVM_NODE.
2. Request the resource in container limits as the plugin documents. Names such as devices.kubevirt.io/kvm are plugin-specific.
3. Remove the kvm volume and mount; the plugin supplies the device.
4. Remove privileged: true and verify sandbox boot, device permissions, and seccomp/AppArmor/SELinux compatibility. Device access alone does not prove VM startup works.

​

Operational notes

• Files: emptyDir survives container restarts but not pod deletion. Use a PVC for durable output and Volume.bind(path, readonly=True) for protected inputs.
• Runtime state: keep /msb private to each pod. It contains the database, image cache, and sandbox files. Persisting it does not preserve running VMs.
• Resources: budget for the application, all guests, and runtime overhead. This example allows 2 GiB for one 512 MiB guest. Measure concurrent workloads and disk usage; keep runtime volumes disk-backed.
• Networking: both cluster egress rules and microsandbox policies apply. Guest images download through the application container.
• Shutdown: the context manager kills and removes the sandbox on exit. For graceful shutdown, call await sandbox.stop() before leaving the context. Long-running applications need a SIGTERM handler that stops accepting work, finishes or cancels tasks, and awaits shutdown within the pod’s grace period. Default signal handling does not unwind Python contexts. No sandbox survives pod deletion.

​

Troubleshooting

kubectl -n msb-example get pod msb-agent -o wide
kubectl -n msb-example describe pod msb-agent
kubectl -n msb-example logs msb-agent

command: ["msb", "doctor"]

kubectl -n msb-example delete pod msb-agent
kubectl apply -f pod.yaml
kubectl -n msb-example logs -f msb-agent --pod-running-timeout=300s

​

Validation status