Docker

Essential commands and patterns for containerising applications with Docker.

Overview

Docker is a platform for developing, shipping, and running applications in containers. Containers package an application with its dependencies, ensuring consistent behaviour across environments.

⤢ zoom

---

Image Management

Docker images are read-only templates used to create containers.

Building Images

# Build image from Dockerfile in current directory
docker build -t myapp:1.0 .

# Build with custom Dockerfile location
docker build -f Dockerfile.prod -t myapp:prod .

# Build with build arguments
docker build --build-arg NODE_ENV=production -t myapp:1.0 .

# Build without using cache
docker build --no-cache -t myapp:1.0 .

# Build for specific platform
docker build --platform linux/amd64 -t myapp:1.0 .

Tagging and Pushing

# Tag an existing image
docker tag myapp:1.0 registry.example.com/myapp:1.0

# Push to registry
docker push registry.example.com/myapp:1.0

# Push all tags for an image
docker push --all-tags registry.example.com/myapp

Pulling Images

# Pull latest version
docker pull nginx

# Pull specific tag
docker pull nginx:1.25-alpine

# Pull by digest (immutable)
docker pull nginx@sha256:abc123...

Listing and Inspecting Images

# List all images
docker images

# List with filtering
docker images --filter "dangling=true"

# Show image history (layers)
docker history myapp:1.0

# Inspect image metadata
docker inspect myapp:1.0

Cleaning Up Images

# Remove specific image
docker rmi myapp:1.0

# Remove all unused images
docker image prune

# Remove all images without at least one container
docker image prune -a

# Remove images older than 24 hours
docker image prune -a --filter "until=24h"

---

Container Lifecycle

⤢ zoom

Creating and Running Containers

# Run container in foreground
docker run nginx

# Run in detached mode (background)
docker run -d --name webserver nginx

# Run with port mapping (host:container)
docker run -d -p 8080:80 nginx

# Run with environment variables
docker run -d -e MYSQL_ROOT_PASSWORD=secret mysql:8

# Run with automatic removal on exit
docker run --rm -it ubuntu bash

# Run with custom hostname
docker run -d --hostname myhost nginx

# Run with specific user
docker run -d --user 1000:1000 nginx

Managing Running Containers

# List running containers
docker ps

# List all containers (including stopped)
docker ps -a

# Stop a container gracefully
docker stop webserver

# Stop with custom timeout (default 10s)
docker stop -t 30 webserver

# Kill a container immediately
docker kill webserver

# Restart a container
docker restart webserver

# Pause/unpause container processes
docker pause webserver
docker unpause webserver

Executing Commands in Containers

# Execute command in running container
docker exec webserver ls /usr/share/nginx/html

# Interactive shell session
docker exec -it webserver /bin/bash

# Execute as specific user
docker exec -u root webserver whoami

# Set environment variable for command
docker exec -e DEBUG=true webserver printenv

Removing Containers

# Remove stopped container
docker rm webserver

# Force remove running container
docker rm -f webserver

# Remove all stopped containers
docker container prune

# Remove multiple containers
docker rm container1 container2 container3

---

Volume Management

Volumes provide persistent storage for containers that survives container lifecycle.

⤢ zoom

Named Volumes

# Create a named volume
docker volume create mydata

# List volumes
docker volume ls

# Inspect volume details
docker volume inspect mydata

# Run container with named volume
docker run -d -v mydata:/app/data myapp:1.0

# Remove unused volumes
docker volume prune

# Remove specific volume
docker volume rm mydata

Bind Mounts

# Mount host directory to container
docker run -d -v /host/path:/container/path nginx

# Mount with read-only access
docker run -d -v /host/config:/etc/nginx/conf.d:ro nginx

# Mount current directory
docker run -d -v $(pwd):/app myapp:1.0

# Using --mount syntax (more explicit)
docker run -d \
    --mount type=bind,source=/host/path,target=/container/path \
    nginx

Volume with Docker Compose

services:
  database:
    image: postgres:15
    volumes:
      # Named volume for data persistence
      - postgres_data:/var/lib/postgresql/data
      # Bind mount for initialisation scripts
      - ./init-scripts:/docker-entrypoint-initdb.d:ro

volumes:
  postgres_data:  # Declare named volume

---

Network Management

Docker networking enables communication between containers and external systems.

⤢ zoom

Network Commands

# List networks
docker network ls

# Create custom bridge network
docker network create mynetwork

# Create network with custom subnet
docker network create \
    --driver bridge \
    --subnet 172.20.0.0/16 \
    --gateway 172.20.0.1 \
    mynetwork

# Inspect network
docker network inspect mynetwork

# Remove network
docker network rm mynetwork

# Remove all unused networks
docker network prune

Connecting Containers to Networks

# Run container on specific network
docker run -d --network mynetwork --name app1 myapp:1.0

# Connect running container to network
docker network connect mynetwork app1

# Disconnect from network
docker network disconnect mynetwork app1

# Run with host networking
docker run -d --network host nginx

# Run with no networking
docker run -d --network none myapp:1.0

Container DNS and Communication

# Containers on the same custom network can resolve each other by name
docker network create appnet
docker run -d --network appnet --name database postgres:15
docker run -d --network appnet --name app myapp:1.0

# In the app container, connect to database using hostname 'database'
# Example: postgresql://database:5432/mydb

# Set custom DNS servers
docker run -d --dns 8.8.8.8 --dns 8.8.4.4 myapp:1.0

---

Dockerfile Best Practices

Multi-Stage Builds

Multi-stage builds reduce final image size by separating build and runtime environments.

⤢ zoom

# Build stage
FROM golang:1.21-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o /app/server

# Runtime stage
FROM alpine:3.18
RUN apk --no-cache add ca-certificates
WORKDIR /app
COPY --from=builder /app/server .
EXPOSE 8080
USER 1000:1000
ENTRYPOINT ["./server"]

Layer Optimisation

# BAD: Each RUN creates a new layer
RUN apt-get update
RUN apt-get install -y curl
RUN apt-get install -y git
RUN apt-get clean

# GOOD: Combine commands and clean up in same layer
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        curl \
        git && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

Caching Dependencies

# GOOD: Copy dependency files first (better cache utilisation)
FROM node:20-alpine
WORKDIR /app

# Dependencies change less frequently than source code
COPY package.json package-lock.json ./
RUN npm ci --omit=dev

# Source code changes frequently
COPY . .
RUN npm run build

EXPOSE 3000
CMD ["node", "dist/index.js"]

Security Best Practices

FROM node:20-alpine

# Create non-root user
RUN addgroup -g 1001 appgroup && \
    adduser -u 1001 -G appgroup -D appuser

WORKDIR /app
COPY --chown=appuser:appgroup . .

# Install dependencies
RUN npm ci --omit=dev

# Switch to non-root user
USER appuser

EXPOSE 3000
CMD ["node", "index.js"]

Complete Dockerfile Example

# syntax=docker/dockerfile:1

FROM python:3.11-slim AS base

# Set environment variables
ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    PIP_NO_CACHE_DIR=1

WORKDIR /app

# Install system dependencies
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        curl \
        libpq-dev && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Create non-root user
RUN useradd -m -u 1000 appuser

# Install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY --chown=appuser:appuser . .

# Switch to non-root user
USER appuser

EXPOSE 8000

# Health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
    CMD curl -f http://localhost:8000/health || exit 1

CMD ["gunicorn", "--bind", "0.0.0.0:8000", "app:app"]

---

.dockerignore Usage

The .dockerignore file excludes files from the build context, reducing build time and image size.

# Version control
.git
.gitignore
.svn

# Dependencies (will be installed in container)
node_modules
__pycache__
*.pyc
venv/
.venv/

# Build outputs
dist/
build/
*.egg-info/

# Development files
.env
.env.local
*.log
.coverage
htmlcov/

# IDE and editor files
.idea/
.vscode/
*.swp
*.swo
*~

# Docker files (not needed in image)
Dockerfile*
docker-compose*.yml
.dockerignore

# Documentation
README.md
docs/

# Test files
tests/
test/
*.test.js
*_test.go

---

Docker Compose Essentials

Docker Compose defines and runs multi-container applications.

Example docker-compose.yml

services:
  web:
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
      - DATABASE_URL=postgresql://db:5432/myapp
    depends_on:
      db:
        condition: service_healthy
    networks:
      - appnet

  db:
    image: postgres:15-alpine
    volumes:
      - postgres_data:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: myapp
      POSTGRES_USER: user
      POSTGRES_PASSWORD: secret
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user -d myapp"]
      interval: 10s
      timeout: 5s
      retries: 5
    networks:
      - appnet

  redis:
    image: redis:7-alpine
    command: redis-server --appendonly yes
    volumes:
      - redis_data:/data
    networks:
      - appnet

volumes:
  postgres_data:
  redis_data:

networks:
  appnet:
    driver: bridge

Common Compose Commands

# Start all services in detached mode
docker compose up -d

# Start with build
docker compose up -d --build

# Stop and remove containers
docker compose down

# Stop and remove containers, volumes, and images
docker compose down -v --rmi all

# View logs
docker compose logs

# Follow logs for specific service
docker compose logs -f web

# View logs with timestamps
docker compose logs -t web

# Execute command in service
docker compose exec web bash

# Run one-off command
docker compose run --rm web npm test

# Scale service
docker compose up -d --scale worker=3

# List running services
docker compose ps

# Restart specific service
docker compose restart web

# Pull latest images
docker compose pull

---

Container Inspection and Debugging

Viewing Logs

# View container logs
docker logs webserver

# Follow log output
docker logs -f webserver

# Show last 100 lines
docker logs --tail 100 webserver

# Show logs since timestamp
docker logs --since 2023-01-01T00:00:00 webserver

# Show logs with timestamps
docker logs -t webserver

Inspecting Containers

# Full container inspection (JSON)
docker inspect webserver

# Get specific field
docker inspect --format='{{.State.Status}}' webserver

# Get container IP address
docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' webserver

# Get environment variables
docker inspect --format='{{range .Config.Env}}{{println .}}{{end}}' webserver

# Get mounted volumes
docker inspect --format='{{range .Mounts}}{{.Source}} -> {{.Destination}}{{"\n"}}{{end}}' webserver

Resource Usage

# Real-time resource statistics
docker stats

# Stats for specific containers
docker stats webserver database

# One-time stats output
docker stats --no-stream

# View running processes in container
docker top webserver

Debugging Techniques

# Copy files from container
docker cp webserver:/var/log/nginx/error.log ./error.log

# Copy files to container
docker cp ./config.json webserver:/app/config.json

# View filesystem changes
docker diff webserver

# Export container filesystem
docker export webserver > webserver.tar

# Attach to running container
docker attach webserver

# Check container events
docker events --filter container=webserver

---

Resource Limits and Constraints

Control container resource usage to ensure stability and fairness.

Memory Limits

# Set memory limit
docker run -d --memory=512m nginx

# Set memory with swap limit
docker run -d --memory=512m --memory-swap=1g nginx

# Set soft limit (reservation)
docker run -d --memory=512m --memory-reservation=256m nginx

CPU Limits

# Limit to specific CPU cores
docker run -d --cpuset-cpus="0,1" nginx

# Set CPU shares (relative weight, default 1024)
docker run -d --cpu-shares=512 nginx

# Limit CPU usage (0.5 = 50% of one core)
docker run -d --cpus=0.5 nginx

# Set CPU quota and period
docker run -d --cpu-quota=50000 --cpu-period=100000 nginx

Combined Resource Limits

# Production container with resource limits
docker run -d \
    --name production-app \
    --memory=1g \
    --memory-reservation=512m \
    --cpus=2 \
    --restart=unless-stopped \
    --health-cmd="curl -f http://localhost:8080/health || exit 1" \
    --health-interval=30s \
    --health-timeout=10s \
    --health-retries=3 \
    myapp:1.0

Docker Compose Resource Limits

services:
  web:
    image: myapp:1.0
    deploy:
      resources:
        limits:
          cpus: '2'
          memory: 1G
        reservations:
          cpus: '0.5'
          memory: 256M
    restart: unless-stopped

---

Quick Reference

Most Common Commands

Command	Description
docker run -d -p 8080:80 nginx	Run container in background with port mapping
docker ps -a	List all containers
docker logs -f <container>	Follow container logs
docker exec -it <container> bash	Interactive shell in container
docker stop <container>	Gracefully stop container
docker rm -f <container>	Force remove container
docker build -t myapp:1.0 .	Build image from Dockerfile
docker images	List all images
docker rmi <image>	Remove image
docker volume ls	List volumes
docker network ls	List networks
docker system prune -a	Remove all unused data

Docker Compose Quick Reference

Command	Description
docker compose up -d	Start services in background
docker compose down	Stop and remove containers
docker compose logs -f	Follow all service logs
docker compose exec web bash	Shell into service
docker compose ps	List services
docker compose build	Build/rebuild services
docker compose pull	Pull service images

---

Common Issues and Solutions

Issue	Cause	Solution
Container exits immediately	No foreground process	Add CMD that keeps running, or use -d with appropriate entrypoint
Cannot connect to Docker daemon	Docker service not running	Run sudo systemctl start docker
Port already in use	Another process using the port	Use different port mapping or stop conflicting process
Permission denied on volume	UID/GID mismatch	Match container user to host permissions or use --user flag
Image build fails at COPY	File in .dockerignore or wrong context	Check .dockerignore and build context path
Container cannot resolve hostname	Not on custom network	Use user-defined bridge network for DNS resolution
Out of disk space	Unused images/containers	Run docker system prune -a --volumes
Slow build times	No layer caching	Order Dockerfile commands by change frequency (least to most)
Health check failing	Wrong endpoint or timing	Adjust interval/timeout/retries or check application startup time
Cannot pull image	Authentication required	Run docker login <registry>

Debugging Steps

# Check container status and exit code
docker inspect --format='{{.State.Status}} (exit: {{.State.ExitCode}})' <container>

# View container logs for errors
docker logs --tail 50 <container>

# Check if port is listening inside container
docker exec <container> netstat -tlnp

# Verify environment variables
docker exec <container> env

# Check filesystem permissions
docker exec <container> ls -la /app

# Test network connectivity
docker exec <container> ping -c 3 database

---

Related Topics

The following topics complement Docker and are commonly used together:

• Kubernetes - Container orchestration for production deployments at scale
• Podman - Daemonless container engine with rootless container support
• Container Security - Image scanning, runtime security, and best practices for secure containers
• Helm - Package manager for Kubernetes applications
• CI/CD Patterns - Integrating Docker builds into continuous integration pipelines
• Prometheus/Grafana - Monitoring and observability for containerised applications