Container Runtime Interface implementation for running Box as a Kubernetes RuntimeClass

Kubernetes CRI

A3S Box implements the Kubernetes Container Runtime Interface (CRI) via gRPC, allowing it to be used as a Kubernetes RuntimeClass. Each pod runs in its own MicroVM with full VM-level isolation.

Architecture

┌──────────────────────────────────┐
│          kubelet                 │
│  RuntimeService · ImageService   │
└──────────┬───────────────────────┘
           │ gRPC (CRI v1)
┌──────────▼───────────────────────┐
│       a3s-box-cri server         │
│  ┌──────────────────────────┐    │
│  │    RuntimeService        │    │
│  │  Pod Sandbox → VmManager │    │
│  │  Container → Session     │    │
│  ├──────────────────────────┤    │
│  │    ImageService          │    │
│  │  Pull → ImagePuller      │    │
│  │  Status → ImageStore     │    │
│  ├──────────────────────────┤    │
│  │    Streaming Server      │    │
│  │  Exec · Attach · PF      │    │
│  │  HTTP :18800             │    │
│  ├──────────────────────────┤    │
│  │    State Stores          │    │
│  │  SandboxStore (in-memory)│    │
│  │  ContainerStore          │    │
│  └──────────────────────────┘    │
└──────────────────────────────────┘
           │ vsock
┌──────────▼───────────────────────┐
│       MicroVM (per pod)          │
│  guest-init · Exec · PTY         │
└──────────────────────────────────┘

CRI to Box Mapping

Prop

Type

RuntimeService

Maps CRI pod and container lifecycle to Box VM management:

Lifecycle

Prop

Type

Exec & Attach

Prop

Type

Port Forward & Resources

Prop

Type

Streaming Server

CRI uses a two-phase protocol for interactive operations (kubectl exec, kubectl attach, kubectl port-forward):

Kubelet calls the gRPC method (e.g., Exec) → returns a streaming URL
Kubelet connects to the URL via HTTP → bidirectional I/O

A3S Box runs an HTTP streaming server (default 127.0.0.1:18800) that bridges these connections to the existing vsock exec/PTY infrastructure inside the MicroVM.

kubectl exec -it pod -- /bin/sh
    │
    ▼
kubelet → gRPC Exec() → returns http://127.0.0.1:18800/exec/<token>
    │
    ▼
kubelet → HTTP connect to streaming URL
    │
    ▼
Streaming Server → PTY Unix socket → vsock → Guest PTY Server
    │                                              │
    └──────── bidirectional terminal I/O ──────────┘

Exec Modes

Prop

Type

Attach

Attach connects to a shell (/bin/sh) in the container via the PTY server. This provides the same experience as kubectl attach -it.

Port Forward

Port forwarding proxies TCP connections to guest ports. The streaming server establishes a socat bridge inside the guest VM:

kubectl port-forward pod 8080:80
    │
    ▼
Streaming Server → exec "socat STDIO TCP:127.0.0.1:80" in guest
    │
    └── bidirectional TCP proxy ──→ guest port 80

Configuration

The streaming server address can be configured when creating the CRI server:

let server = CriServer::new(socket_path, image_store, auth)
    .with_streaming_addr("0.0.0.0:18800".parse().unwrap());

ImageService

Prop

Type

Config Mapping

The config_mapper module translates CRI pod configuration into BoxConfig:

CRI PodSandboxConfig
    ├── metadata (name, namespace, uid) → BoxConfig.name
    ├── dns_config → BoxConfig.dns
    ├── port_mappings → BoxConfig.port_map
    ├── linux.resources → BoxConfig.resources
    │   ├── cpu_period / cpu_quota → ResourceLimits
    │   ├── memory_limit → ResourceConfig.memory_mb
    │   └── pids_limit → ResourceLimits.pids_limit
    ├── annotations → feature flags
    │   ├── "a3s.box/tee" → TeeConfig
    │   ├── "a3s.box/network-mode" → NetworkMode
    │   └── "a3s.box/pool" → PoolConfig
    └── labels → metadata tracking

State Stores

SandboxStore

pub struct PodSandbox {
    pub id: String,
    pub metadata: PodSandboxMetadata,
    pub state: SandboxState,       // "ready" or "notready"
    pub created_at: i64,
    pub vm_manager: VmManager,
}

pub struct SandboxStore {
    sandboxes: RwLock<HashMap<String, PodSandbox>>,
}

ContainerStore

pub struct Container {
    pub id: String,
    pub sandbox_id: String,
    pub metadata: ContainerMetadata,
    pub state: ContainerState,     // "running" or "exited"
    pub exit_code: Option<i32>,
}

pub struct ContainerStore {
    containers: RwLock<HashMap<String, Container>>,
}

Both stores are currently in-memory. State is lost on CRI server restart, which may orphan running VMs.

Kubernetes Integration

RuntimeClass

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: a3s-box
handler: a3s-box

Pod Spec

apiVersion: v1
kind: Pod
metadata:
  name: secure-workload
  annotations:
    a3s.box/tee: "sev-snp"
    a3s.box/network-mode: "bridge"
spec:
  runtimeClassName: a3s-box
  containers:
    - name: app
      image: my-app:latest
      resources:
        limits:
          cpu: "2"
          memory: "1Gi"

TEE via Annotations

Enable confidential computing for specific pods:

metadata:
  annotations:
    a3s.box/tee: "sev-snp"
    a3s.box/tee-generation: "genoa"
    a3s.box/workload-id: "secure-inference"

Limitations

Prop

Type

Kubernetes CRI

On this page