Container Runtimes

containerd and CRI-O: lightweight, production-grade container runtimes designed for Kubernetes and edge environments.

Overview

Container runtimes are the fundamental layer that executes containers. Unlike Docker, which is a complete platform with extensive tooling, containerd and CRI-O are minimal runtimes optimised for Kubernetes integration. They implement the Container Runtime Interface (CRI) specification, enabling direct integration with kubelet without intermediary layers.

Both runtimes offer:

• Lower resource overhead than Docker
• Direct CRI protocol support for Kubernetes
• OCI image and runtime compliance
• Efficient image management
• Production-grade stability

⤢ zoom

---

containerd

The container runtime promoted from Docker and now a CNCF graduated project. It provides core functionality for managing container lifecycle.

Architecture

containerd uses a modular architecture:

• containerd daemon: gRPC server handling CRI requests
• containerd-shim: Lightweight wrapper per container, decoupling runtime from daemon
• Runtime engines: OCI-compliant executors (runc, kata, gVisor)
• Image service: Pull, store, and manage container images
• Snapshot service: Efficient image layer management

Key Concepts

Container namespace: Logical grouping for containers. Default is "k8s.io".

Snapshots: Copy-on-write image layer management. containerd supports overlayfs, btrfs, and native snapshotter.

Tasks: Active running instances of containers with lifecycle management.

Leases: Reference counting mechanism to prevent premature garbage collection.

Common Commands

Basic Container Management

# List containers
ctr container list

# Create container from image
ctr run -d docker.io/library/alpine:latest mycontainer sh

# Delete container
ctr container delete mycontainer

# View container info
ctr container info mycontainer

Image Management

# Pull image
ctr image pull docker.io/library/nginx:latest

# List images
ctr image list

# Remove image
ctr image delete docker.io/library/nginx:latest

# Tag image
ctr image tag docker.io/library/nginx:latest myregistry.com/nginx:v1

# Push image
ctr image push myregistry.com/nginx:v1

# Show image details
ctr image info docker.io/library/nginx:latest

Task Execution

# Create and run task (non-detached)
ctr task start -d mycontainer

# List running tasks
ctr task list

# Execute command in running task
ctr task exec -t mycontainer id

# Pause task
ctr task pause mycontainer

# Resume task
ctr task resume mycontainer

# Kill task
ctr task kill mycontainer

# Delete task
ctr task delete mycontainer

Snapshots and Layers

# List snapshots
ctr snapshot list

# View snapshot info
ctr snapshot info <snapshot-id>

# Use specific snapshotter
ctr --snapshotter=overlayfs snapshot list

Configuration

Primary config file: /etc/containerd/config.toml

Config versions: containerd 2.x defaults to config version = 3, which splits the CRI plugin into io.containerd.cri.v1.runtime (runtime/CNI) and io.containerd.cri.v1.images (images/registry). The older version = 2 form with a single io.containerd.grpc.v1.cri plugin is still parsed by 2.x but is the 1.x layout. Registry mirrors are now configured via a certs.d hosts directory (config_path) rather than inline registry.mirrors tables — see "Image Registry Authentication" below. Generate a current default with containerd config default.

Basic Configuration Example (containerd 2.x, config v3)

version = 3

[plugins.'io.containerd.cri.v1.runtime']
  [plugins.'io.containerd.cri.v1.runtime'.containerd]
    default_runtime_name = "runc"

    [plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runc]
      runtime_type = "io.containerd.runc.v2"

      [plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runc.options]
        SystemdCgroup = true

[plugins.'io.containerd.cri.v1.images']
  snapshotter = "overlayfs"

  [plugins.'io.containerd.cri.v1.images'.registry]
    config_path = "/etc/containerd/certs.d"

[grpc]
  address = "/run/containerd/containerd.sock"

[metrics]
  address = "127.0.0.1:1338"

Registry Mirror Configuration

In containerd 2.x the inline registry.mirrors tables are removed; mirrors live in a certs.d hosts directory pointed at by config_path (see the basic config above):

# /etc/containerd/certs.d/docker.io/hosts.toml
server = "https://registry-1.docker.io"

[host."https://mirror.aliyun.com"]
  capabilities = ["pull", "resolve"]

Custom Runtime Configuration (e.g., kata-containers)

[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.kata]
  runtime_type = "io.containerd.kata.v2"

Image Management

# Load image from tar
ctr image import image.tar

# Export image to tar
ctr image export archive.tar docker.io/library/nginx:latest

# Show filesystem usage
ctr image du

# Clean up unused images
ctr image prune

# Check image digest
ctr image list --quiet | grep nginx

Examples

Running a Simple Container

# Pull busybox
ctr image pull docker.io/library/busybox:latest

# Create and run container
ctr run -d docker.io/library/busybox:latest mybox sleep 1000

# List running tasks
ctr task list

# Execute command inside
ctr task exec -t mybox wget -O - http://example.com

Container with Volume Mount

# Create container with volume bind mount
ctr run -d \
  -v /tmp/data:/data:rw \
  docker.io/library/alpine:latest \
  myapp \
  sleep 3600

# Verify mount
ctr task exec -t myapp mount | grep data

Multi-Platform Image Pull

# Pull specific platform variant
ctr image pull --all-platforms docker.io/library/ubuntu:latest

# Run ARM64 image on amd64 system (with appropriate runtime)
ctr run --runtimes=qemu docker.io/library/ubuntu:latest arm-container

---

CRI-O

The Kubernetes-native container runtime maintained by the CRI-O project. Designed from the ground up as a slim, efficient Kubernetes runtime without unnecessary features.

Architecture

CRI-O implements the Kubernetes CRI specification directly:

• CRI server: Implements CRI protocol for kubelet communication
• OCI runtime handler: Pluggable OCI runtime support
• Conmon: Container monitoring utility, separate process per container
• Image service: Pulls and manages images via containers/image library
• Network service: CNI plugin integration for pod networking

Key Concepts

Pods and Containers: Native pod concept; sandbox container provides network namespace.

Runtime handlers: Named OCI runtime configurations for specialised workloads (e.g., GPU, secure containers).

Image streams: References to base images in Kubernetes; automatic pulls and garbage collection.

Hooks: Lifecycle hooks integrated with systemd and custom handlers.

Common Commands

Container and Pod Management

# List pods
crictl pods

# Inspect pod
crictl inspectp <pod-id>

# Create pod
crictl runp pod-config.json

# Delete pod
crictl rmp <pod-id>

# List containers
crictl ps

# Inspect container
crictl inspect <container-id>

# Create container
crictl create <pod-id> container-config.json pod-config.json

# Start container
crictl start <container-id>

# Stop container
crictl stop <container-id>

# Remove container
crictl rm <container-id>

Image Management

# List images
crictl images

# Pull image
crictl pull docker.io/library/nginx:latest

# Remove image
crictl rmi docker.io/library/nginx:latest

# Show image info
crictl inspect --type=image docker.io/library/nginx:latest

# Image filesystem usage
crictl images --verbose

Execution and Debugging

# Execute command in container
crictl exec -i -t <container-id> /bin/bash

# View container logs (logs is per-container; takes a CONTAINER-ID, not a pod ID)
crictl logs <container-id>

# Follow logs
crictl logs -f <container-id>

# Stats
crictl stats <container-id>

# Version info
crictl version

Configuration

Primary config file: /etc/crio/crio.conf.d/

Basic Configuration Example

[crio]
  storage_driver = "overlay"
  storage_option = [
    "overlay.override_kernel_check=1"
  ]

[crio.runtime]
  default_runtime = "runc"

  [crio.runtime.runtimes.runc]
    runtime_path = "/usr/bin/runc"
    runtime_type = "oci"
    runtime_root = "/run/runc"

[crio.image]
  default_runtime = "runc"
  global_auth_file = "/var/lib/crio/auth.json"

  pause_image = "registry.k8s.io/pause:3.8"
  pause_image_auth_file = ""

  [[crio.image.registries]]
  prefix = "docker.io"
  insecure = false
  blocked = false

  [crio.image.registries.docker]
    endpoint = ["https://registry-1.docker.io"]
    auth_file = "/var/lib/crio/auth.json"

[crio.network]
  network_dir = "/etc/cni/net.d/"
  plugin_dir = "/opt/cni/bin/"
  default_network = "crio"

[crio.monitoring]
  enable_metrics = true
  metrics_port = 9090

Runtime Handler Configuration

[crio.runtime.runtimes.kata]
  runtime_path = "/usr/bin/kata-runtime"
  runtime_type = "oci"
  runtime_root = "/run/kata"

Image Registry Mirror

[[crio.image.registries]]
prefix = "docker.io"

[crio.image.registries.docker]
endpoint = [
  "https://mirror.aliyun.com",
  "https://registry-1.docker.io"
]

Image Management

# Prune unused images and containers
crictl rmi --prune

# Show storage status
crictl stats

# Show configured registries
crictl info | grep registry

# Pull with specific auth
REGISTRY_AUTH=<base64-auth> crictl pull myregistry.com/image:tag

# Pull multiple images in parallel (scripted)
for image in nginx:latest postgres:13 redis:latest; do
  crictl pull "docker.io/library/$image" &
done
wait

Examples

Running a Pod with Containers

# Create pod configuration
cat > pod.json <<EOF
{
  "metadata": {
    "name": "web-server",
    "namespace": "default",
    "uid": "abc123"
  },
  "linux": {
    "namespaces": {
      "path": ""
    }
  }
}
EOF

# Create container configuration
cat > container.json <<EOF
{
  "metadata": {
    "name": "nginx"
  },
  "image": {
    "image": "docker.io/library/nginx:latest"
  },
  "linux": {}
}
EOF

# Create pod
POD_ID=$(crictl runp pod.json)

# Create and start container
CONTAINER_ID=$(crictl create $POD_ID container.json pod.json)
crictl start $CONTAINER_ID

# Verify
crictl ps

Pod Networking Verification

# Get pod IP
crictl inspectp $POD_ID | jq '.info.runtimeSpec.linux.namespaces'

# Execute network commands
crictl exec -i -t $CONTAINER_ID ip addr show
crictl exec -i -t $CONTAINER_ID ping 8.8.8.8

---

Kubernetes Integration

Installation and Configuration

containerd in Kubernetes

# Configure containerd socket for kubelet
cat > /etc/kubernetes/kubelet.conf.d/00-cri.conf <<EOF
KUBELET_KUBECONFIG_ARGS=--kubeconfig=/etc/kubernetes/kubelet.conf
KUBELET_CONTAINER_RUNTIME=remote
KUBELET_CONTAINER_RUNTIME_ENDPOINT=unix:///run/containerd/containerd.sock
EOF

# Reload kubelet
systemctl restart kubelet

CRI-O in Kubernetes

# Configure CRI-O socket for kubelet
cat > /etc/kubernetes/kubelet.conf.d/00-crio.conf <<EOF
KUBELET_KUBECONFIG_ARGS=--kubeconfig=/etc/kubernetes/kubelet.conf
KUBELET_CONTAINER_RUNTIME=remote
KUBELET_CONTAINER_RUNTIME_ENDPOINT=unix:///var/run/crio/crio.sock
EOF

# Reload kubelet
systemctl restart kubelet

Pod Lifecycle

⤢ zoom

Runtime Selection per Node

# Label node for specific runtime
kubectl label nodes node-1 container-runtime=cri-o

# Run pod on CRI-O nodes
apiVersion: v1
kind: Pod
metadata:
  name: crio-pod
spec:
  nodeSelector:
    container-runtime: cri-o
  containers:
  - name: app
    image: nginx:latest

---

Differences from Docker

Aspect	Docker	containerd	CRI-O
Scope	Complete platform	Container runtime only	Kubernetes-only runtime
Architecture	Monolithic daemon	Modular, containerd + shims	Single daemon (conmon helpers)
Resource Usage	High (full features)	Minimal	Minimal
Kubernetes Integration	dockershim removed in k8s 1.24 (May 2022)	Native CRI support	Native CRI implementation
CLI Tool	docker	ctr	crictl
Image Format	OCI + docker-specific	OCI standard	OCI standard
Configuration	Complex daemon.json	TOML based	INI based
Use Case	Development, production	Kubernetes, edge	Kubernetes production

Migration from Docker

# Export Docker images
docker save myimage:tag | ctr -a /path/to/content.db image import -

# Convert Docker image archives
skopeo copy docker-archive:image.tar oci-archive:image.oci

# Migrate container configs
# Docker run command -> containerd TOML + task creation

---

When to Use Each Runtime

Use containerd if:

• Running Kubernetes on production clusters
• Need a balance between features and simplicity
• Want CNCF-backed project with stable releases
• Require custom runtime handlers (kata, gVisor)
• Using as embedded runtime in applications

Use CRI-O if:

• Kubernetes is your only use case
• Minimal overhead is critical
• Want tight Kubernetes-only integration
• Need RedHat/CentOS ecosystem support
• Operating in constrained environments

Use Docker if:

• Developing locally with full Docker ecosystem
• Need docker-compose for multi-container development
• Require advanced networking and orchestration features
• Not running Kubernetes (traditional deployments)
• Team familiarity is high priority

---

Quick Reference

containerd CLI (ctr)

# Images
ctr image pull <image>
ctr image list
ctr image delete <image>
ctr image tag <source> <target>
ctr image push <image>

# Containers
ctr run -d <image> <name>          # 'ctr container' has no 'run' subcommand
ctr container list
ctr container delete <name>

# Tasks
ctr task start <name>
ctr task list
ctr task exec -t <name> <command>
ctr task kill <name>

# Snapshots
ctr snapshot list
ctr snapshot --snapshotter=<type> list

# Config
/etc/containerd/config.toml
containerd --version
ctr version

CRI-O CLI (crictl)

# Pods
crictl runp <pod-config.json>
crictl pods
crictl inspectp <pod-id>
crictl rmp <pod-id>

# Containers
crictl create <pod-id> <container-config.json> <pod-config.json>
crictl ps
crictl start <container-id>
crictl stop <container-id>
crictl rm <container-id>

# Images
crictl pull <image>
crictl images
crictl rmi <image>

# Execution
crictl exec -i -t <container-id> <command>
crictl logs <container-id>
crictl stats

# Config
/etc/crio/crio.conf.d/
crio config

Common Port Numbers

Service	Port	Protocol
containerd gRPC	1337 (default unix socket)	Unix domain socket
containerd metrics	1338	TCP
CRI-O CRI socket	/var/run/crio/crio.sock	Unix domain socket
CRI-O metrics	9090	TCP

---

Common Issues and Solutions

containerd Issues

Issue	Cause	Solution
Image pull fails	Registry unreachable or auth fail	Check registry config in config.toml; verify credentials in /etc/containerd/certs.d/
Container exits immediately	Entrypoint/command invalid	Verify image layers with ctr image info; check logs
High memory usage	Image snapshots not cleaned	Run ctr image prune; adjust snapshotter configuration
Permission denied	Wrong socket permissions	Check /run/containerd/containerd.sock owned by root:root

CRI-O Issues

Issue	Cause	Solution
kubelet can't reach CRI-O	Socket not readable	Verify /var/run/crio/crio.sock permissions; check crio daemon status
Pod network not configured	CNI plugins missing	Install CNI plugins in /opt/cni/bin/; verify network config in /etc/cni/net.d/
Image not found after pull	Storage location changed	Check storage_path in crio.conf; verify storage driver
High CPU on crio daemon	Excessive garbage collection	Adjust image GC settings in crio.conf.d/

Shared Issues

# Debug runtime connectivity
sudo ctr --address /run/containerd/containerd.sock version
sudo crictl version

# Check CRI socket exists
ls -la /run/containerd/containerd.sock
ls -la /var/run/crio/crio.sock

# Monitor daemon logs
journalctl -u containerd -f
journalctl -u crio -f

# Check resource usage
ps aux | grep containerd
ps aux | grep crio

# Verify OCI runtime availability
which runc
runc --version

# Force garbage collection
ctr --namespace k8s.io content prune
crictl rmi --prune

# Check and cleanup stale data
du -sh /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/
du -sh /var/lib/crio/overlay-layers/

---

Advanced Configuration

containerd with gVisor Sandboxing

[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"

# Use for specific pods:
# apiVersion: v1
# kind: Pod
# spec:
#   runtimeClassName: gvisor

CRI-O with Custom NetworkPolicy

[crio.network]
network_dir = "/etc/cni/net.d/"
plugin_dir = "/opt/cni/bin/"

# Deploy Calico or Flannel CNI plugins
# to /opt/cni/bin/ directory

Image Registry Authentication

containerd:

# Create auth config
mkdir -p /etc/containerd/certs.d/myregistry.com
cat > /etc/containerd/certs.d/myregistry.com/hosts.toml <<EOF
server = "https://myregistry.com"
[host."https://myregistry.com"]
  capabilities = ["pull", "resolve"]
  auth = "base64-encoded-credentials"
EOF

CRI-O:

# Use global auth file
cat > /var/lib/crio/auth.json <<EOF
{
  "auths": {
    "myregistry.com": {
      "auth": "base64-encoded-credentials"
    }
  }
}
EOF
chmod 600 /var/lib/crio/auth.json