Docker Networking Deep Dive

Comprehensive guide to Docker networking drivers, DNS, IPAM, and troubleshooting techniques.

Overview

Docker networking enables containerised applications to communicate with each other, host systems, and external networks. Understanding network drivers, DNS resolution, and troubleshooting tools is essential for building robust containerised architectures. Docker provides multiple network drivers, each optimised for different use cases from single-host development to multi-host production clusters.

⤢ zoom

---

Bridge Networks

Bridge networking creates an isolated software bridge on the host, connecting containers within a virtual network.

⤢ zoom

Default Bridge Network

The default bridge network is created automatically. Containers can communicate via IP but not by name.

# Run container on default bridge
docker run -d --name app1 nginx

# Inspect default bridge
docker network inspect bridge

# Container uses default docker0 bridge (172.17.0.0/16 typically)
docker inspect app1 --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}'

# Containers on default bridge must use IP addresses
docker exec app1 ping 172.17.0.3  # Works if another container exists
docker exec app1 ping app2         # Fails - no DNS on default bridge

User-Defined Bridge Networks

Custom bridge networks provide automatic DNS resolution, better isolation, and dynamic attachment.

# Create custom bridge with default settings
docker network create mynetwork

# Create bridge with custom subnet and gateway
docker network create \
    --driver bridge \
    --subnet 172.20.0.0/16 \
    --gateway 172.20.0.1 \
    --ip-range 172.20.5.0/24 \
    mynetwork

# Create bridge with multiple subnets (dual-stack)
docker network create \
    --driver bridge \
    --subnet 172.20.0.0/16 \
    --subnet 2001:db8::/64 \
    dualstack-net

# Run containers on custom network with DNS
docker run -d --network mynetwork --name app nginx
docker run -d --network mynetwork --name db postgres:15

# DNS resolution works on custom networks
docker exec app ping db          # Resolves via embedded DNS server
docker exec app curl http://db:5432

Bridge Network Configuration Options

# Create bridge with advanced options
docker network create \
    --driver bridge \
    --subnet 10.0.0.0/24 \
    --gateway 10.0.0.1 \
    --opt "com.docker.network.bridge.name=br-custom" \
    --opt "com.docker.network.bridge.enable_icc=true" \
    --opt "com.docker.network.bridge.enable_ip_masquerade=true" \
    --opt "com.docker.network.driver.mtu=1500" \
    --label environment=production \
    prod-network

# Disable inter-container communication
docker network create \
    --driver bridge \
    --opt "com.docker.network.bridge.enable_icc=false" \
    isolated-net

Bridge Network Use Cases

Scenario	Best Practice
Development environment	User-defined bridge for DNS and isolation
Microservices on single host	Separate bridge per logical grouping
Frontend/backend separation	Two bridges with selective connectivity
Legacy applications	Default bridge if no DNS needed

---

Host Networking

Host network mode disables network isolation, sharing the host's network namespace directly.

⤢ zoom

Using Host Network Mode

macOS/Docker Desktop note: Docker runs inside a Linux VM, so host networking behaves differently from Linux. Docker Desktop 4.34+ does support it, but it is opt-in (enable under Settings → Resources → Network → Enable host networking) and operates at layer 4 only (TCP/UDP — no lower-level protocols). With it enabled, a container listener is reachable from the Mac via localhost. When it is disabled (the default), --network host places the container in the VM's network namespace, not the Mac's, and the container's ports sit on the VM loopback rather than the Mac's localhost — use -p port publishing instead. On Linux, host networking is unrestricted.

# Run container with host networking
docker run -d --network host --name webapp myapp:1.0

# No port mapping needed - app binds directly to host ports
# If app listens on :8080, it's accessible on host:8080

# Check network mode
docker inspect webapp --format='{{.HostConfig.NetworkMode}}'
# Output: host

# View which ports application is using on host
docker exec webapp netstat -tlnp

Host Network with Docker Compose

version: '3.8'
services:
  monitoring:
    image: prometheus:latest
    network_mode: host
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    # No ports section needed - uses host network directly

Host Network Characteristics

Advantages:

• Maximum performance - no network translation overhead
• Low latency - no bridging or NAT
• Direct access - to all host network interfaces
• Simplified configuration - for network-intensive applications

Disadvantages:

• No isolation - containers can conflict on ports
• Security concerns - full access to host network stack
• No portability - configurations tied to host network setup
• Port conflicts - multiple containers cannot bind same port

When to Use Host Network

# Use case 1: Network monitoring tools (need raw socket access)
docker run -d \
    --network host \
    --cap-add NET_RAW \
    --cap-add NET_ADMIN \
    nicolaka/netshoot

# Use case 2: High-performance network applications
docker run -d \
    --network host \
    --name high-throughput-proxy \
    nginx-performance-optimised:1.0

# Use case 3: SNMP monitoring that needs to bind to specific interfaces
docker run -d \
    --network host \
    snmp-exporter:latest

---

Macvlan Networks

Macvlan assigns a MAC address to each container, making it appear as a physical device on the network.

⤢ zoom

Creating Macvlan Networks

# Create macvlan network on parent interface eth0
docker network create -d macvlan \
    --subnet=192.168.1.0/24 \
    --gateway=192.168.1.1 \
    --ip-range=192.168.1.100/28 \
    -o parent=eth0 \
    macvlan-net

# Run container with specific IP from macvlan range
docker run -d \
    --network macvlan-net \
    --ip=192.168.1.101 \
    --name legacy-app \
    legacy-server:1.0

# Container gets real MAC address on physical network
docker exec legacy-app ip addr show eth0

Macvlan with VLAN Tagging (802.1Q)

# Create macvlan on VLAN 100 tagged interface
docker network create -d macvlan \
    --subnet=10.100.0.0/24 \
    --gateway=10.100.0.1 \
    -o parent=eth0.100 \
    macvlan-vlan100

# Requires VLAN interface on host first
# ip link add link eth0 name eth0.100 type vlan id 100
# ip link set eth0.100 up

# Run container on VLAN-tagged macvlan
docker run -d \
    --network macvlan-vlan100 \
    --ip=10.100.0.50 \
    vlan-app:1.0

Macvlan Communication Limitations

# ⚠️ WARNING: Host cannot directly communicate with macvlan containers
# This is due to Linux kernel isolation

# WORKAROUND: Create macvlan interface on host
ip link add macvlan-shim link eth0 type macvlan mode bridge
ip addr add 192.168.1.99/32 dev macvlan-shim
ip link set macvlan-shim up
ip route add 192.168.1.100/28 dev macvlan-shim

# Now host can reach macvlan containers
ping 192.168.1.101

Macvlan Modes

# Bridge mode (default) - containers can communicate with each other
docker network create -d macvlan \
    -o parent=eth0 \
    -o macvlan_mode=bridge \
    macvlan-bridge

# Private mode - containers isolated from each other
docker network create -d macvlan \
    -o parent=eth0 \
    -o macvlan_mode=private \
    macvlan-private

# VEPA mode - requires external switch support
docker network create -d macvlan \
    -o parent=eth0 \
    -o macvlan_mode=vepa \
    macvlan-vepa

# Passthru mode - single container gets entire physical interface
docker network create -d macvlan \
    -o parent=eth0 \
    -o macvlan_mode=passthru \
    macvlan-passthru

Macvlan Use Cases

Use Case	Rationale
Legacy applications expecting physical network	Direct L2 connectivity
Network appliances (DHCP, DNS)	Need to appear as physical hosts
VLAN segmentation requirements	802.1Q VLAN tagging support
IP address preservation	Maintain specific IPs from external IPAM
Multi-tenant isolation	Hardware-level network separation

---

User-Defined Networks and DNS

Docker's embedded DNS server provides automatic service discovery on user-defined networks.

⤢ zoom

Embedded DNS Server

# Create network and containers
docker network create appnet
docker run -d --network appnet --name database postgres:15
docker run -d --network appnet --name cache redis:7
docker run -d --network appnet --name api myapi:1.0

# DNS resolution within network
docker exec api nslookup database
# Output:
# Server:    127.0.0.11
# Address:   127.0.0.11:53
# Name:      database
# Address:   172.18.0.2

# DNS works for container names
docker exec api ping database
docker exec api ping cache

# View container's DNS configuration
docker exec api cat /etc/resolv.conf
# Output:
# nameserver 127.0.0.11
# options ndots:0

Network Aliases

# Assign multiple DNS names to a container
docker run -d \
    --network appnet \
    --network-alias db \
    --network-alias postgres \
    --network-alias primary-db \
    --name database \
    postgres:15

# All aliases resolve to same container
docker exec api ping db
docker exec api ping postgres
docker exec api ping primary-db
docker exec api ping database  # Container name also works

DNS Round-Robin Load Balancing

# Create multiple containers with same network alias
docker run -d --network appnet --network-alias web nginx:1.0
docker run -d --network appnet --network-alias web nginx:1.0
docker run -d --network appnet --network-alias web nginx:1.0

# DNS returns all IPs in round-robin fashion
docker run --rm --network appnet nicolaka/netshoot \
    nslookup web

# Output shows multiple A records:
# Name:   web
# Address: 172.18.0.4
# Address: 172.18.0.5
# Address: 172.18.0.6

# Applications get different IPs on each DNS lookup
docker run --rm --network appnet nicolaka/netshoot \
    sh -c 'for i in $(seq 1 5); do curl -s http://web:80 | grep hostname; done'

Multi-Network Containers

# Create multiple networks
docker network create frontend
docker network create backend

# Database only on backend network
docker run -d --network backend --name database postgres:15

# API on both networks
docker run -d --name api myapi:1.0
docker network connect frontend api
docker network connect backend api

# Web only on frontend network
docker run -d --network frontend --name web nginx

# Connectivity test
docker exec web ping api         # Works (same frontend network)
docker exec web ping database    # Fails (different networks)
docker exec api ping database    # Works (both on backend network)

# View container's network memberships
docker inspect api --format='{{range $k, $v := .NetworkSettings.Networks}}{{$k}} {{end}}'
# Output: backend frontend

Custom DNS Configuration

# Set custom DNS servers for container
docker run -d \
    --dns 8.8.8.8 \
    --dns 8.8.4.4 \
    --dns-search example.com \
    --dns-opt ndots:2 \
    --name custom-dns \
    nginx

# Verify DNS configuration
docker exec custom-dns cat /etc/resolv.conf
# Output:
# search example.com
# nameserver 8.8.8.8
# nameserver 8.8.4.4
# options ndots:2

# Disable embedded DNS (use custom DNS only)
docker run -d \
    --dns 1.1.1.1 \
    --dns-option use-vc \
    nginx

DNS with Docker Compose

version: '3.8'

services:
  web:
    image: nginx
    networks:
      frontend:
        aliases:
          - www
          - webserver
      backend:
        aliases:
          - api-gateway

  api:
    image: myapi:1.0
    networks:
      backend:
        aliases:
          - api
          - service
    dns:
      - 8.8.8.8
      - 1.1.1.1
    dns_search:
      - example.com

  database:
    image: postgres:15
    networks:
      backend:
    hostname: primary-db

networks:
  frontend:
  backend:

---

Port Publishing and IPAM

Port publishing exposes container services to external networks, while IPAM controls IP address allocation.

⤢ zoom

Port Publishing Syntax

# Publish single port (random host port)
docker run -d -P nginx  # Maps to random port like 32768

# Publish specific port mapping (host:container)
docker run -d -p 8080:80 nginx

# Publish multiple ports
docker run -d \
    -p 8080:80 \
    -p 8443:443 \
    nginx

# Publish to specific interface
docker run -d -p 127.0.0.1:8080:80 nginx  # Only localhost
docker run -d -p 192.168.1.10:8080:80 nginx  # Specific IP

# Publish with protocol specification
docker run -d -p 8080:80/tcp -p 53:53/udp nginx

# Publish range of ports
docker run -d -p 9000-9010:9000-9010 myapp:1.0

# View published ports
docker port webserver
# Output:
# 80/tcp -> 0.0.0.0:8080
# 80/tcp -> [::]:8080

Port Publishing with Docker Compose

version: '3.8'

services:
  web:
    image: nginx
    ports:
      # Short syntax
      - "8080:80"
      - "8443:443"

  api:
    image: myapi:1.0
    ports:
      # Long syntax with protocol
      - target: 8000
        published: 8000
        protocol: tcp
        mode: host

  admin:
    image: admin-panel:1.0
    ports:
      # Bind to specific interface
      - "127.0.0.1:9000:80"

  metrics:
    image: prometheus:latest
    ports:
      # Publish to IPv6
      - "[::1]:9090:9090"

Understanding iptables NAT Rules

# View Docker's iptables rules
iptables -t nat -L -n -v | grep DOCKER

# Example output for published port 8080:80
# DNAT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:8080 to:172.17.0.2:80

# View specific chain
iptables -t nat -L DOCKER -n --line-numbers

# Trace packet flow for debugging
iptables -t raw -A PREROUTING -p tcp --dport 8080 -j TRACE
iptables -t raw -A OUTPUT -p tcp --dport 8080 -j TRACE
# View in: dmesg | grep TRACE

IPAM (IP Address Management)

Docker uses IPAM drivers to allocate IP addresses to containers.

# View default IPAM configuration
docker network inspect bridge | jq '.[0].IPAM'
# Output:
# {
#   "Driver": "default",
#   "Options": null,
#   "Config": [
#     {
#       "Subnet": "172.17.0.0/16",
#       "Gateway": "172.17.0.1"
#     }
#   ]
# }

# Create network with custom IPAM
docker network create \
    --driver bridge \
    --subnet 10.1.0.0/16 \
    --ip-range 10.1.5.0/24 \
    --gateway 10.1.0.1 \
    --aux-address "host1=10.1.0.2" \
    --aux-address "host2=10.1.0.3" \
    custom-ipam-net

# Reserve IP addresses with aux-address (won't be assigned to containers)
docker network create \
    --subnet 192.168.100.0/24 \
    --gateway 192.168.100.1 \
    --aux-address "reserved1=192.168.100.10" \
    --aux-address "reserved2=192.168.100.11" \
    reserved-net

Static IP Assignment

# Assign static IP to container
docker run -d \
    --network custom-ipam-net \
    --ip 10.1.5.100 \
    --name static-web \
    nginx

# Verify IP assignment
docker inspect static-web \
    --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}'
# Output: 10.1.5.100

# Static IP in Docker Compose
version: '3.8'

services:
  database:
    image: postgres:15
    networks:
      backend:
        ipv4_address: 10.5.0.100

networks:
  backend:
    driver: bridge
    ipam:
      driver: default
      config:
        - subnet: 10.5.0.0/24
          gateway: 10.5.0.1
          ip_range: 10.5.0.128/25

Dual-Stack IPv4/IPv6 IPAM

# Create dual-stack network
docker network create \
    --ipv6 \
    --subnet 172.20.0.0/16 \
    --subnet 2001:db8:1::/64 \
    --gateway 172.20.0.1 \
    --gateway 2001:db8:1::1 \
    dualstack

# Run container with dual-stack
docker run -d \
    --network dualstack \
    --name ipv6-test \
    nginx

# Verify both IPv4 and IPv6 addresses
docker exec ipv6-test ip addr show eth0
# Output shows both:
# inet 172.20.0.2/16 brd 172.20.255.255 scope global eth0
# inet6 2001:db8:1::2/64 scope global

# Enable IPv6 in daemon.json for default networks
# /etc/docker/daemon.json
{
  "ipv6": true,
  "fixed-cidr-v6": "2001:db8::/64"
}

IPAM Driver Options

# Custom IPAM driver (dhcp, external, custom)
docker network create \
    --driver bridge \
    --ipam-driver dhcp \
    --ipam-opt dhcp_option=option1 \
    dhcp-network

# Check available IPAM space
docker network inspect custom-ipam-net | jq '.[0].IPAM.Config'

# Find next available IP
docker network inspect custom-ipam-net \
    --format='{{range .Containers}}{{.IPv4Address}} {{end}}'

---

Network Troubleshooting

Diagnosing network issues requires inspection tools, packet capture, and connectivity testing.

⤢ zoom

Network Inspection Commands

# List all networks with details
docker network ls

# Inspect specific network
docker network inspect mynetwork

# View network connectivity of container
docker inspect webserver --format='{{json .NetworkSettings.Networks}}' | jq

# Get container IP address
docker inspect webserver \
    --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}'

# Get all IPs for all containers
docker ps -q | xargs -I {} docker inspect {} \
    --format='{{.Name}}: {{range .NetworkSettings.Networks}}{{.IPAddress}} {{end}}'

# View port mappings
docker port webserver

# List all containers on specific network
docker network inspect mynetwork \
    --format='{{range .Containers}}{{.Name}} {{.IPv4Address}}{{"\n"}}{{end}}'

Container Connectivity Testing

# Ping between containers
docker exec web ping -c 3 database

# Test specific port connectivity
docker exec web nc -zv database 5432  # TCP connection test
docker exec web curl -v http://api:8000/health

# Telnet to check port
docker exec web telnet database 5432

# Test UDP connectivity
docker exec web nc -u -zv logserver 514

# HTTP request with timing
docker exec web curl -w "@curl-format.txt" -o /dev/null -s http://api:8000
# Where curl-format.txt contains:
#   time_namelookup: %{time_namelookup}\n
#   time_connect: %{time_connect}\n
#   time_total: %{time_total}\n

DNS Troubleshooting

# Check DNS resolution
docker exec web nslookup database

# Detailed DNS query with dig
docker exec web dig database

# Query specific DNS server
docker exec web nslookup database 8.8.8.8

# Check container's DNS configuration
docker exec web cat /etc/resolv.conf

# Check /etc/hosts for static entries
docker exec web cat /etc/hosts

# Test external DNS
docker exec web nslookup google.com
docker exec web dig @8.8.8.8 example.com

# Debug DNS with verbose output
docker exec web dig +trace database

Packet Capture with tcpdump

# Install tcpdump in container (if not present)
docker exec -it web apt-get update && apt-get install -y tcpdump

# Capture packets on container's interface
docker exec web tcpdump -i eth0 -nn -A

# Capture specific port traffic
docker exec web tcpdump -i eth0 port 80 -nn -v

# Capture HTTP traffic
docker exec web tcpdump -i eth0 -A -s 0 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'

# Write capture to file
docker exec web tcpdump -i eth0 -w /tmp/capture.pcap
docker cp web:/tmp/capture.pcap ./capture.pcap
wireshark capture.pcap  # Analyse locally

# Capture on host for container traffic
# Find container's veth interface on host
container_id=$(docker inspect web --format='{{.Id}}')
ifindex=$(docker exec web cat /sys/class/net/eth0/iflink)
veth=$(ip link | grep "^${ifindex}:" | cut -d: -f2 | xargs)

# Capture on veth pair
tcpdump -i $veth -nn -v

Advanced tcpdump Filters

# Capture only SYN packets (connection attempts)
docker exec web tcpdump -i eth0 'tcp[tcpflags] & tcp-syn != 0' -nn

# Capture packets larger than 1000 bytes
docker exec web tcpdump -i eth0 'greater 1000' -nn

# Capture traffic between two specific IPs
docker exec web tcpdump -i eth0 'host 172.20.0.2 and host 172.20.0.3' -nn

# Capture non-ICMP traffic
docker exec web tcpdump -i eth0 'not icmp' -nn

# Capture HTTP POST requests
docker exec web tcpdump -i eth0 -A -s 0 'tcp dst port 80 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354)'

# Monitor specific container's traffic from host
container_pid=$(docker inspect web --format='{{.State.Pid}}')
nsenter -t $container_pid -n tcpdump -i eth0 -nn

Using netshoot for Troubleshooting

# Run netshoot container with network troubleshooting tools
docker run --rm -it --network mynetwork nicolaka/netshoot

# Tools available in netshoot:
# - tcpdump, ngrep, tshark
# - iperf, netperf
# - nmap, ncat, socat
# - curl, wget, httpie
# - drill, nslookup, dig
# - netstat, ss, ip, iftop
# - mtr, traceroute, ping

# Test from netshoot container
docker run --rm --network mynetwork nicolaka/netshoot \
    bash -c "ping -c 3 database && curl http://api:8000/health"

# Network scanning
docker run --rm --network mynetwork nicolaka/netshoot \
    nmap -sn 172.20.0.0/24  # Network sweep

# Bandwidth testing between containers
# On container 1 (server)
docker exec web iperf3 -s

# On container 2 (client)
docker run --rm --network mynetwork nicolaka/netshoot \
    iperf3 -c web -t 10

Bridge and iptables Inspection

# View Docker bridges on host
ip link show type bridge
brctl show  # If bridge-utils installed

# View bridge ports
ip link show master docker0

# Inspect bridge configuration
ip addr show docker0

# View iptables rules for Docker
iptables -t nat -L DOCKER -n -v
iptables -t filter -L DOCKER -n -v
iptables -t filter -L DOCKER-ISOLATION-STAGE-1 -n -v

# Check for blocked traffic
iptables -L -n -v | grep DROP
iptables -L -n -v | grep REJECT

# Monitor iptables counters (run twice to see changes)
iptables -t nat -L DOCKER -n -v -Z  # Zero counters
# Generate traffic
iptables -t nat -L DOCKER -n -v     # View counters

# Debug connection tracking
conntrack -L | grep 172.20.0  # Show connections in conntrack table

Performance Testing

# Test latency between containers
docker exec web ping -c 100 database | tail -1

# TCP throughput test
# Server
docker exec database iperf3 -s

# Client
docker exec web iperf3 -c database -t 30 -i 1

# UDP throughput test
docker exec web iperf3 -c database -u -b 1G

# HTTP request benchmarking
docker exec web ab -n 1000 -c 10 http://api:8000/

# Connection rate testing
docker run --rm --network mynetwork williamyeh/wrk \
    wrk -t4 -c100 -d30s http://api:8000/

Common Network Issues

Issue	Symptoms	Diagnosis	Solution
DNS resolution fails	ping: unknown host	Check /etc/resolv.conf, test with nslookup	Use user-defined network, verify DNS config
Port not accessible	Connection refused/timeout	Check with netstat -tlnp, verify port mapping	Ensure app binds to 0.0.0.0, check firewall
Inter-container communication fails	Ping works but app fails	Use tcpdump to see traffic, check app logs	Verify app configuration, check network policies
Slow network performance	High latency, low throughput	Use iperf3, check MTU settings	Tune MTU, check bridge options
IP address conflict	Container fails to start	docker network inspect, check IPAM	Remove conflicting container, adjust ip-range
Cannot reach external network	External connections timeout	Check NAT rules with iptables -t nat -L	Verify ip_forward enabled, check MASQUERADE rules

---

Overlay Networks and Swarm

Overlay networks enable multi-host container communication using VXLAN encapsulation.

⤢ zoom

Initialising Docker Swarm

# Initialise Swarm on manager node
docker swarm init --advertise-addr 192.168.1.10

# Output provides join command:
# docker swarm join --token SWMTKN-1-xxx... 192.168.1.10:2377

# Join worker nodes (run on worker hosts)
docker swarm join --token SWMTKN-1-xxx... 192.168.1.10:2377

# View Swarm nodes
docker node ls

# Promote worker to manager
docker node promote worker-node-1

# Remove node from Swarm
docker node rm worker-node-2

Creating Overlay Networks

# Create overlay network (requires Swarm mode)
docker network create \
    --driver overlay \
    --subnet 10.0.9.0/24 \
    --gateway 10.0.9.1 \
    myoverlay

# Create overlay with encryption
docker network create \
    --driver overlay \
    --opt encrypted \
    secure-overlay

# Create attachable overlay (for standalone containers)
docker network create \
    --driver overlay \
    --attachable \
    app-overlay

# View overlay networks
docker network ls --filter driver=overlay

Overlay Network Configuration

# Create overlay with custom MTU
docker network create \
    --driver overlay \
    --opt com.docker.network.driver.mtu=1450 \
    overlay-custom-mtu

# Create overlay with specific VXLAN ID
docker network create \
    --driver overlay \
    --opt com.docker.network.driver.overlay.vxlanid_list=4097 \
    overlay-custom-vxlan

# Multiple subnets on overlay
docker network create \
    --driver overlay \
    --subnet 10.10.0.0/24 \
    --subnet 10.11.0.0/24 \
    --gateway 10.10.0.1 \
    --gateway 10.11.0.1 \
    multi-subnet-overlay

Deploying Services on Overlay Networks

# Create service on overlay network
docker service create \
    --name web \
    --network myoverlay \
    --replicas 3 \
    -p 8080:80 \
    nginx

# View service details
docker service ls
docker service ps web

# Service automatically load-balances across replicas
docker service inspect web --format='{{json .Endpoint}}' | jq

# Scale service
docker service scale web=5

# Update service
docker service update --image nginx:alpine web

Swarm Service Discovery

# Services can resolve each other by name across hosts
docker service create --name api --network myoverlay api:1.0
docker service create --name db --network myoverlay postgres:15

# DNS works across all Swarm nodes
docker service create \
    --name client \
    --network myoverlay \
    alpine sleep 3600

# Find container for client service
client_id=$(docker ps --filter "name=client" --format "{{.ID}}" | head -1)

# Test DNS resolution across hosts
docker exec $client_id ping api
docker exec $client_id nslookup db

# VIP (Virtual IP) mode - single IP for service (default)
docker service inspect api --format='{{.Endpoint.VirtualIPs}}'

# DNS round-robin mode - multiple IPs
docker service create \
    --name api-dnsrr \
    --network myoverlay \
    --endpoint-mode dnsrr \
    api:1.0

Ingress Network

Swarm's ingress network provides routing mesh for published ports.

# Ingress network created automatically with Swarm
docker network inspect ingress

# Publishing port uses ingress routing mesh
docker service create \
    --name web \
    --replicas 3 \
    --publish published=8080,target=80 \
    nginx

# Any Swarm node can receive traffic on port 8080
# Traffic is automatically routed to available container
curl http://node1:8080  # Routes to any of 3 replicas
curl http://node2:8080  # Also works on different node

# Bypass ingress with host mode (binds to node's port directly)
docker service create \
    --name web-host \
    --publish published=8080,target=80,mode=host \
    nginx

Overlay Network Encryption

# Create encrypted overlay network
docker network create \
    --driver overlay \
    --opt encrypted \
    secure-net

# Encryption details:
# - Uses IPsec (AES-GCM)
# - Automatic key rotation
# - Control plane encryption by default (TLS)
# - Data plane encryption with --opt encrypted

# Deploy service on encrypted network
docker service create \
    --name secure-api \
    --network secure-net \
    --replicas 3 \
    secure-api:1.0

# View encryption status
docker network inspect secure-net | jq '.[0].Options'

Overlay Network Troubleshooting

# Check overlay network on specific node
docker network ls --filter driver=overlay

# Inspect overlay on manager
docker network inspect myoverlay

# View VXLAN interfaces on host
ip -d link show | grep vxlan

# Check if VXLAN traffic is flowing (UDP 4789)
tcpdump -i eth0 udp port 4789 -nn

# Check Swarm gossip protocol (TCP/UDP 7946)
tcpdump -i eth0 port 7946 -nn

# View overlay network peers
docker network inspect myoverlay \
    --format='{{range .Peers}}{{.Name}} {{.IP}}{{"\n"}}{{end}}'

# Diagnose connectivity between Swarm nodes
# On manager
docker run --rm --network myoverlay nicolaka/netshoot \
    ping -c 3 <worker-node-container-ip>

# Check service connectivity across nodes
docker service logs web

# View service endpoint information
docker service inspect web --format='{{json .Endpoint}}' | jq

Docker Stack with Overlay Networks

version: '3.8'

services:
  web:
    image: nginx:alpine
    deploy:
      replicas: 3
      placement:
        max_replicas_per_node: 1
    networks:
      - frontend
      - backend
    ports:
      - "8080:80"

  api:
    image: myapi:1.0
    deploy:
      replicas: 5
      resources:
        limits:
          cpus: '0.5'
          memory: 512M
    networks:
      - backend
      - database_net
    environment:
      - DB_HOST=database

  database:
    image: postgres:15
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.role == manager
    networks:
      - database_net
    volumes:
      - db-data:/var/lib/postgresql/data

networks:
  frontend:
    driver: overlay
    attachable: true
  backend:
    driver: overlay
    driver_opts:
      encrypted: "true"
  database_net:
    driver: overlay
    internal: true  # No external connectivity

volumes:
  db-data:

# Deploy stack
docker stack deploy -c docker-compose.yml myapp

# View stack services
docker stack services myapp

# View tasks across nodes
docker stack ps myapp

# View stack networks
docker network ls --filter label=com.docker.stack.namespace=myapp

# Remove stack
docker stack rm myapp

Overlay Network Limitations and Considerations

Consideration	Details
MTU	Default 1450 bytes (to accommodate VXLAN overhead). May need tuning for jumbo frames.
Encryption overhead	IPsec adds ~10-20% CPU overhead. Use only when necessary.
Port requirements	TCP 2377 (management), TCP/UDP 7946 (gossip), UDP 4789 (VXLAN) must be open.
Network scale	Tested up to ~1000 nodes. Beyond that, consider Kubernetes.
Windows support	Limited overlay support on Windows nodes.
Attachable networks	Standalone containers cannot join overlay without --attachable flag.
External connectivity	Internal networks block external access even with port publishing.

Overlay Network Best Practices

# 1. Use encryption only for sensitive data
docker network create --driver overlay --opt encrypted secure-net

# 2. Segment services with multiple overlay networks
docker network create --driver overlay frontend
docker network create --driver overlay backend
docker network create --driver overlay --internal database

# 3. Monitor VXLAN health
tcpdump -i eth0 udp port 4789 -c 100 | wc -l  # Should see traffic

# 4. Use DNS for service discovery
# Avoid hardcoding IPs - use service names

# 5. Set proper resource limits
docker service create \
    --network myoverlay \
    --reserve-memory 256M \
    --limit-memory 512M \
    myapp:1.0

# 6. Use placement constraints
docker service create \
    --network myoverlay \
    --constraint 'node.labels.type==compute' \
    myapp:1.0

# 7. Monitor network performance
# Use iperf3 to test bandwidth between nodes
docker service create --name iperf-server --network myoverlay \
    --mode global networkstatic/iperf3 -s

---

Quick Reference

Network Drivers Comparison

Driver	Use Case	Isolation	DNS	Multi-host	Performance
bridge	Single-host apps	High	Yes (user-defined)	No	Good
host	Maximum performance	None	N/A	No	Excellent
macvlan	Legacy apps, VLANs	Medium	No	Yes (L2)	Excellent
overlay	Swarm, multi-host	High	Yes	Yes	Good
none	Complete isolation	Complete	No	No	N/A

Common Network Commands

Command	Description
docker network create mynet	Create user-defined bridge network
docker network ls	List all networks
docker network inspect mynet	View network configuration
docker network connect mynet container1	Connect container to network
docker network disconnect mynet container1	Disconnect container from network
docker run --network mynet --name app nginx	Run container on specific network
docker run --network host nginx	Run with host networking
docker run -p 8080:80 nginx	Publish port (host:container)
docker port container1	View port mappings
docker exec container1 ping container2	Test connectivity

DNS Resolution Examples

# User-defined bridge - DNS works
docker network create appnet
docker run -d --network appnet --name db postgres:15
docker run -d --network appnet --name app myapp:1.0
docker exec app ping db  # ✓ Works

# Default bridge - DNS does not work
docker run -d --name db postgres:15
docker run -d --name app myapp:1.0
docker exec app ping db  # ✗ Fails
docker exec app ping 172.17.0.2  # ✓ Works with IP

Troubleshooting Checklist

# 1. Verify network exists and is connected
docker network inspect mynetwork

# 2. Check container's network configuration
docker inspect container --format='{{json .NetworkSettings}}' | jq

# 3. Test basic connectivity
docker exec container ping -c 3 target

# 4. Verify DNS resolution
docker exec container nslookup target

# 5. Check port connectivity
docker exec container nc -zv target 8080

# 6. Capture packets for analysis
docker exec container tcpdump -i eth0 -nn -c 100

# 7. Check firewall rules
iptables -L DOCKER -n -v
iptables -t nat -L DOCKER -n -v

# 8. Verify routing
docker exec container ip route show

# 9. Check for IP conflicts
docker network inspect mynetwork | jq '.[0].Containers'

# 10. Test from clean container
docker run --rm --network mynetwork nicolaka/netshoot ping target

Network Performance Tuning

# Increase MTU for better performance
docker network create \
    --driver bridge \
    --opt com.docker.network.driver.mtu=9000 \
    jumbo-net

# Disable inter-container communication (security)
docker network create \
    --driver bridge \
    --opt com.docker.network.bridge.enable_icc=false \
    isolated-net

# Enable IPv6
docker network create \
    --ipv6 \
    --subnet 2001:db8::/64 \
    ipv6-net

# Optimise for high connection rate
# /etc/docker/daemon.json
{
  "userland-proxy": false,  # Use iptables only
  "iptables": true
}

---

Common Issues and Solutions

Issue	Symptoms	Diagnosis	Solution
DNS resolution failure	ping: unknown host database	docker exec app nslookup database	Use user-defined bridge network, not default bridge
Port binding conflict	Error: address already in use	netstat -tlnp | grep 8080	Change host port, stop conflicting process
No external connectivity	Cannot reach internet	docker exec app ping 8.8.8.8	Check IP forwarding: sysctl net.ipv4.ip_forward
Container-to-container fails	Connection refused between containers	tcpdump, check network membership	Verify both containers on same network
Slow overlay performance	High latency, packet loss	iperf3 test, check MTU	Reduce MTU to 1450, disable encryption if not needed
Macvlan host isolation	Host cannot reach containers	ip addr on host	Create macvlan shim interface on host
Swarm node unreachable	Node shows as down	Check ports 2377, 7946, 4789	Open firewall ports, verify connectivity
IP address exhausted	Cannot create container	docker network inspect IPAM	Increase subnet size or create new network
Docker0 conflicts	Network 172.17.0.0 conflicts with corporate	Edit /etc/docker/daemon.json	Set custom bridge CIDR with bip option
Published port unreachable	Port open but no response	iptables -t nat -L, docker port	Verify app binds 0.0.0.0, check iptables rules

Advanced Troubleshooting

# Debug iptables rules
iptables-save | grep DOCKER

# Check Docker daemon logs
journalctl -u docker -f

# Verify network namespace
ip netns list
docker inspect container --format='{{.NetworkSettings.SandboxKey}}'

# Enter container's network namespace
container_pid=$(docker inspect container --format='{{.State.Pid}}')
nsenter -t $container_pid -n ip addr

# Check for kernel network issues
dmesg | grep -i network
dmesg | grep -i vxlan

# Monitor real-time network events
docker events --filter type=network

# Test with minimal container
docker run --rm --network mynetwork alpine:latest sh -c "ping -c 3 target"

---

Related Topics

The following topics complement Docker Networking and are commonly used together:

• Docker Compose Production Patterns - Multi-container networking with environment-specific configurations
• Kubernetes Networking - CNI plugins, Services, Ingress, and NetworkPolicies for production orchestration
• Container Security - Network isolation, segmentation, and secure communication patterns
• Nginx/HAProxy - Reverse proxies for container traffic management and load balancing
• Prometheus/Grafana - Monitoring container network metrics and performance
• Istio/Service Mesh - Advanced traffic management, security, and observability for microservices