RabbitMQ Administration

Advanced message broker administration covering exchange patterns, clustering, policies, and security hardening.

Overview

RabbitMQ is a robust, enterprise-grade message broker implementing AMQP 0-9-1 (with support for AMQP 1.0, MQTT, and STOMP via plugins). As an administrator, you'll manage exchanges, queues, bindings, clustering, policies, and security configurations to ensure reliable, high-performance message delivery across distributed systems.

graph TB
    subgraph "RabbitMQ Core Components"
        Producer[Producer Application] -->|Publish| Exchange[Exchange]
        Exchange -->|Routing Key| Binding[Bindings]
        Binding --> Queue1[Queue 1]
        Binding --> Queue2[Queue 2]
        Binding --> Queue3[Queue 3]
        Queue1 -->|Consume| Consumer1[Consumer 1]
        Queue2 -->|Consume| Consumer2[Consumer 2]
        Queue3 -->|Consume| Consumer3[Consumer 3]

        Queue1 -.->|Dead Letter| DLX[Dead Letter Exchange]
        DLX --> DLQ[Dead Letter Queue]
    end

    style Exchange fill:#e1f5fe
    style DLX fill:#ffebee
    style DLQ fill:#ffebee

Exchange Types and Bindings

Key Concepts

Exchange: Receives messages from publishers and routes them to queues based on bindings and routing rules
Binding: Link between an exchange and a queue with optional routing key and arguments
Routing Key: Message attribute used by exchanges to determine routing
Binding Key: Pattern used by exchanges to match against routing keys
Headers: Key-value pairs in messages used by headers exchanges for routing

Exchange Types

Direct Exchange - Routes messages to queues where the binding key exactly matches the routing key:

graph LR
    P[Publisher] -->|routing_key: error| E[Direct Exchange<br/>logs]
    E -->|binding: error| Q1[Error Queue]
    E -->|binding: warning| Q2[Warning Queue]
    E -->|binding: info| Q3[Info Queue]

    style E fill:#e1f5fe
    style Q1 fill:#ffcdd2

# Note: examples use the classic Python rabbitmqadmin (v1), shipped by the
# management plugin. RabbitMQ also ships a newer Rust rabbitmqadmin v2 with
# different syntax (subcommands like `declare queue --name ...`); check
# `rabbitmqadmin --version` if a command here is unrecognised.

# Create direct exchange
rabbitmqadmin declare exchange name=logs type=direct durable=true

# Bind queues with specific routing keys
rabbitmqadmin declare binding source=logs destination=error_queue \
    routing_key=error

rabbitmqadmin declare binding source=logs destination=warning_queue \
    routing_key=warning

# Publish message with routing key
rabbitmqadmin publish exchange=logs routing_key=error \
    payload="Database connection failed"

Fanout Exchange - Routes messages to all bound queues, ignoring routing keys:

graph LR
    P[Publisher] --> E[Fanout Exchange<br/>broadcast]
    E --> Q1[Queue 1]
    E --> Q2[Queue 2]
    E --> Q3[Queue 3]

    style E fill:#e1f5fe

# Create fanout exchange
rabbitmqadmin declare exchange name=broadcast type=fanout durable=true

# Bind multiple queues (routing_key ignored for fanout)
rabbitmqadmin declare binding source=broadcast destination=analytics_queue
rabbitmqadmin declare binding source=broadcast destination=logging_queue
rabbitmqadmin declare binding source=broadcast destination=audit_queue

# All bound queues receive the message
rabbitmqadmin publish exchange=broadcast payload="System notification"

Topic Exchange - Routes messages based on pattern matching between routing key and binding pattern:

graph LR
    P1[Publisher] -->|uk.weather.rain| E[Topic Exchange<br/>events]
    P2[Publisher] -->|uk.news.politics| E
    P3[Publisher] -->|us.weather.snow| E

    E -->|uk.#| Q1[UK All Events]
    E -->|*.weather.*| Q2[All Weather]
    E -->|uk.weather.#| Q3[UK Weather]

    style E fill:#e1f5fe

Wildcard patterns:

* matches exactly one word
# matches zero or more words

# Create topic exchange
rabbitmqadmin declare exchange name=events type=topic durable=true

# Bind with wildcard patterns
rabbitmqadmin declare binding source=events destination=uk_all \
    routing_key="uk.#"

rabbitmqadmin declare binding source=events destination=all_weather \
    routing_key="*.weather.*"

rabbitmqadmin declare binding source=events destination=uk_weather \
    routing_key="uk.weather.#"

# Publish with hierarchical routing key
rabbitmqadmin publish exchange=events routing_key="uk.weather.rain" \
    payload='{"temp":15,"conditions":"rainy"}'

Headers Exchange - Routes based on message header attributes instead of routing keys:

# Create headers exchange
rabbitmqadmin declare exchange name=tasks type=headers durable=true

# Bind with header matching (x-match: all = AND, any = OR)
rabbitmqadmin declare binding source=tasks destination=urgent_tasks \
    arguments='{"x-match":"all","priority":"high","department":"ops"}'

rabbitmqadmin declare binding source=tasks destination=dev_tasks \
    arguments='{"x-match":"any","department":"dev","department":"qa"}'

# Publish with headers (requires client library)
# Python example:
# channel.basic_publish(
#     exchange='tasks',
#     routing_key='',
#     body='Task payload',
#     properties=pika.BasicProperties(
#         headers={'priority': 'high', 'department': 'ops'}
#     )
# )

Managing Bindings

# List all bindings
rabbitmqctl list_bindings

# List bindings for specific vhost
rabbitmqctl list_bindings -p /production

# Detailed binding information
rabbitmqadmin list bindings

# Remove binding
rabbitmqadmin delete binding source=logs destination=error_queue \
    properties_key=error

# View exchange details and bindings
rabbitmqctl list_exchanges name type durable auto_delete arguments
rabbitmqadmin list exchanges name type durable
rabbitmqadmin list bindings source destination routing_key

Common Patterns

Multi-tenant routing with topic exchanges:

# tenant.service.action pattern
# Examples: acme.orders.created, globex.users.updated

rabbitmqadmin declare exchange name=multi_tenant type=topic

# Per-tenant queue
rabbitmqadmin declare binding source=multi_tenant destination=acme_events \
    routing_key="acme.#"

# Cross-tenant service queue
rabbitmqadmin declare binding source=multi_tenant destination=all_orders \
    routing_key="*.orders.*"

Priority routing with direct exchanges:

# Separate queues by priority level
rabbitmqadmin declare queue name=critical_tasks durable=true
rabbitmqadmin declare queue name=normal_tasks durable=true

rabbitmqadmin declare binding source=task_router destination=critical_tasks \
    routing_key=critical

rabbitmqadmin declare binding source=task_router destination=normal_tasks \
    routing_key=normal

Queue Policies and TTL/DLX

Key Concepts

Policy: Set of rules applied to queues or exchanges matching a pattern
TTL (Time To Live): Maximum time a message can remain in a queue before expiry
DLX (Dead Letter Exchange): Exchange where rejected, expired, or maxed-out messages are republished
DLQ (Dead Letter Queue): Queue bound to a DLX for capturing dead-lettered messages
Message TTL: Per-message expiration time
Queue TTL: Automatic queue deletion after period of inactivity

Queue Policies

Classic queue mirroring was removed in RabbitMQ 4.0. The ha-mode, ha-params, and ha-sync-mode keys are no longer recognised — set_policy rejects them with "are not recognised policy settings". For replicated HA, declare quorum queues (see below); high availability is now a property of the queue type, not a policy applied to classic queues.

# Set policy using rabbitmqctl (message-ttl shown; any valid key works)
rabbitmqctl set_policy ttl-1h "^temp\." \
    '{"message-ttl":3600000}' \
    --priority 10 \
    --apply-to queues

# Set policy using rabbitmqadmin
rabbitmqadmin declare policy name=max-len pattern="^bounded\." \
    definition='{"max-length":10000,"overflow":"reject-publish"}' \
    priority=10 \
    apply-to=queues

# List all policies
rabbitmqctl list_policies
rabbitmqadmin list policies

# Clear policy
rabbitmqctl clear_policy ttl-1h

Common Policy Patterns:

# Force a queue type for matching names (operator policy; overrides client args)
rabbitmqctl set_operator_policy qq-default ".*" \
    '{"max-length":1000000}' --apply-to queues

# Queue length limit
rabbitmqctl set_policy max-length "^limited\." \
    '{"max-length":10000,"overflow":"reject-publish"}'

# Message TTL (milliseconds)
rabbitmqctl set_policy ttl-policy "^temp\." \
    '{"message-ttl":3600000}'

# Queue expiry (milliseconds of inactivity)
rabbitmqctl set_policy queue-ttl "^ephemeral\." \
    '{"expires":1800000}'

# Max priority
rabbitmqctl set_policy priority-policy "^priority\." \
    '{"max-priority":10}'

Dead Letter Exchanges (DLX)

Setting up DLX:

# Create dead letter exchange
rabbitmqadmin declare exchange name=dlx type=direct durable=true

# Create dead letter queue
rabbitmqadmin declare queue name=dead_letter_queue durable=true

# Bind DLQ to DLX
rabbitmqadmin declare binding source=dlx destination=dead_letter_queue \
    routing_key=dead

# Create main queue with DLX configuration
rabbitmqadmin declare queue name=main_queue durable=true \
    arguments='{"x-dead-letter-exchange":"dlx","x-dead-letter-routing-key":"dead"}'

DLX via Policy:

# Apply DLX to all queues matching pattern
rabbitmqctl set_policy dlx-policy "^monitored\." \
    '{"dead-letter-exchange":"dlx","dead-letter-routing-key":"dead"}'

DLX Triggers:

flowchart TD
    A[Message in Queue] --> B{Rejection Scenarios}
    B -->|basic.reject/nack<br/>requeue=false| DLX[Dead Letter Exchange]
    B -->|Message TTL expired| DLX
    B -->|Queue length limit<br/>exceeded| DLX
    B -->|Consumer cannot<br/>process| DLX

    DLX --> DLQ[Dead Letter Queue]
    DLQ --> Analysis[Manual Analysis<br/>or Replay]

    style DLX fill:#ffebee
    style DLQ fill:#ffcdd2

Messages are dead-lettered when:

Consumer rejects with requeue=false (basic.reject/basic.nack)
Message TTL expires
Queue length limit exceeded (overflow: drop-head or reject-publish)

# Complete DLX setup with TTL
rabbitmqadmin declare exchange name=work_exchange type=direct
rabbitmqadmin declare exchange name=work_dlx type=direct

rabbitmqadmin declare queue name=work_queue durable=true \
    arguments='{"x-message-ttl":60000,"x-dead-letter-exchange":"work_dlx"}'

rabbitmqadmin declare queue name=work_dlq durable=true

rabbitmqadmin declare binding source=work_exchange destination=work_queue \
    routing_key=work

rabbitmqadmin declare binding source=work_dlx destination=work_dlq \
    routing_key=work

TTL Configuration

Message TTL (per-message):

# Using Python pika library
import pika

channel.basic_publish(
    exchange='work',
    routing_key='task',
    body='Task data',
    properties=pika.BasicProperties(
        expiration='60000'  # 60 seconds in milliseconds
    )
)

Queue TTL (all messages in queue):

# Set at queue creation
rabbitmqadmin declare queue name=temp_queue \
    arguments='{"x-message-ttl":30000}'

# Or via policy
rabbitmqctl set_policy ttl-30s "^temp\." \
    '{"message-ttl":30000}'

Queue Expiry (delete idle queue):

# Queue deleted after 1 hour of no consumers
rabbitmqadmin declare queue name=auto_expire \
    arguments='{"x-expires":3600000}'

Overflow Behaviour

# Drop oldest messages when queue full
rabbitmqctl set_policy drop-head "^limited\." \
    '{"max-length":1000,"overflow":"drop-head"}'

# Reject new publishes when queue full
rabbitmqctl set_policy reject-publish "^strict\." \
    '{"max-length":1000,"overflow":"reject-publish"}'

# Reject new publishes and dead-letter
rabbitmqctl set_policy reject-publish-dlx "^strict\." \
    '{"max-length":1000,"overflow":"reject-publish-dlx"}'

Clustering and Quorum Queues

Key Concepts

Cluster: Multiple RabbitMQ nodes joined together for high availability and load distribution
Node: Single RabbitMQ server instance in a cluster
Quorum Queue: Replicated queue type using Raft consensus algorithm
Classic Queue: Traditional non-replicated queue type (CQv2 since 4.0; mirroring removed — use quorum queues for HA)
Stream: Append-only log data structure for high-throughput scenarios

Cluster Architecture

graph TB
    subgraph "RabbitMQ Cluster"
        subgraph "Node 1 (Disc)"
            N1["rabbit@node1<br/>Disc Node"]
            Q1[Queue: orders<br/>Leader]
            Q2[Queue: payments<br/>Follower]
        end

        subgraph "Node 2 (Disc)"
            N2["rabbit@node2<br/>Disc Node"]
            Q3[Queue: orders<br/>Follower]
            Q4[Queue: payments<br/>Leader]
        end

        subgraph "Node 3 (RAM)"
            N3["rabbit@node3<br/>RAM Node"]
            Q5[Queue: orders<br/>Follower]
        end
    end

    N1 <--> N2
    N2 <--> N3
    N1 <--> N3

    LB[Load Balancer] --> N1
    LB --> N2
    LB --> N3

    style N1 fill:#e3f2fd
    style N2 fill:#e3f2fd
    style N3 fill:#fff3e0
    style Q1 fill:#c8e6c9
    style Q4 fill:#c8e6c9

Cluster Setup

Prerequisites:

# Ensure Erlang cookie is identical on all nodes
# /var/lib/rabbitmq/.erlang.cookie or $HOME/.erlang.cookie
cat /var/lib/rabbitmq/.erlang.cookie
# Must be same on all nodes

# Ensure hostname resolution
echo "192.168.1.10 rabbit1" >> /etc/hosts
echo "192.168.1.11 rabbit2" >> /etc/hosts
echo "192.168.1.12 rabbit3" >> /etc/hosts

# Ensure firewall allows ports
# 4369: epmd (Erlang Port Mapper Daemon)
# 5672: AMQP
# 15672: Management plugin HTTP
# 25672: Inter-node communication
ufw allow 4369,5672,15672,25672/tcp

Creating a Cluster:

# On node1 (already running)
rabbitmqctl cluster_status

# On node2
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl join_cluster rabbit@node1
rabbitmqctl start_app

# On node3
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl join_cluster rabbit@node1
rabbitmqctl start_app

# Verify cluster
rabbitmqctl cluster_status

# Output shows:
# Cluster status of node rabbit@node2
# Nodes: [rabbit@node1, rabbit@node2, rabbit@node3]
# Running nodes: [rabbit@node1, rabbit@node2, rabbit@node3]

RAM vs Disc Nodes:

# Join as RAM node (faster, metadata only in RAM)
rabbitmqctl join_cluster rabbit@node1 --ram

# Change node type
rabbitmqctl stop_app
rabbitmqctl change_cluster_node_type disc
rabbitmqctl start_app

# Best practice: At least 2 disc nodes for metadata persistence

Cluster Management

# View cluster status
rabbitmqctl cluster_status
rabbitmqctl list_nodes

# Remove node from cluster (run on node to be removed)
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl start_app

# Force remove node (run on remaining node)
rabbitmqctl forget_cluster_node rabbit@node3

# Rename node (update rabbitmq-env.conf)
NODENAME=rabbit@newname
rabbitmqctl rename_cluster_node rabbit@oldname rabbit@newname

# Note: sync_queue / cancel_sync_queue were removed in 4.0 along with classic
# queue mirroring. Quorum queues replicate via Raft and need no manual sync;
# grow/shrink their membership with rabbitmq-queues add_member / delete_member.
rabbitmq-queues grow rabbit@node3 all   # add a member on a new node
rabbitmq-queues shrink rabbit@node3     # remove members before decommissioning

Quorum Queues

Quorum queues provide high availability and data safety using Raft consensus.

Creating Quorum Queues:

# Declare quorum queue
rabbitmqadmin declare queue name=orders.quorum durable=true \
    arguments='{"x-queue-type":"quorum"}'

# Queue type can't be set by a policy — it's fixed at declaration via the
# x-queue-type argument (or the vhost default_queue_type). A policy may still
# tune quorum settings, e.g. the poison-message delivery limit:
rabbitmqctl set_policy quorum-limit "^qq\." \
    '{"delivery-limit":3}' \
    --apply-to queues

# Set initial replication group size (default: 3, capped at the cluster size)
rabbitmqadmin declare queue name=regional.quorum \
    arguments='{"x-queue-type":"quorum","x-quorum-initial-group-size":3}'

Quorum Queue Features:

stateDiagram-v2
    [*] --> Leader: Election
    Leader --> Follower1: Replicate
    Leader --> Follower2: Replicate

    Follower1 --> Leader: Leader fails
    Follower2 --> Leader: Election

    Leader --> [*]: Majority lost

    note right of Leader
        Accepts writes
        Coordinates replication
        Serves reads
    end note

    note right of Follower1
        Replicates data
        Participates in elections
        Can become leader
    end note

# Quorum queue characteristics:
# - Automatic replication to majority of nodes
# - Poison message handling (delivery-limit)
# - At-least-once delivery guarantee
# - No message priorities (use separate queues)
# - No lazy mode (all messages on disk)

# Set delivery limit for poison messages
rabbitmqadmin declare queue name=resilient.quorum \
    arguments='{"x-queue-type":"quorum","x-delivery-limit":5}'

# View quorum queue status
rabbitmqctl list_queues name type leader members

# View detailed quorum queue info
rabbitmqadmin list queues name type messages consumers

Quorum Queue vs Classic Mirrored:

Classic mirrored queues were removed in RabbitMQ 4.0. The column below is retained only to map old deployments onto their quorum-queue replacement — there is no supported way to create a mirrored classic queue on 3.13+/4.x.

Feature	Quorum Queue	Classic Mirrored (removed in 4.0)
Replication	Raft consensus	Active-passive mirrors
Data safety	Strong guarantees	Could lose data on failover
Performance	Higher write latency	Lower write latency
Poison messages	Built-in delivery-limit	Manual DLX setup required
Memory	Always on disk	Could be lazy
Priorities	Not supported	Supported
Status	Current HA mechanism	Gone — migrate to quorum queues

Load Balancing

# HAProxy configuration for RabbitMQ cluster
# /etc/haproxy/haproxy.cfg

# Frontend for AMQP
frontend rabbitmq_amqp
    bind *:5672
    mode tcp
    default_backend rabbitmq_amqp_backend

backend rabbitmq_amqp_backend
    mode tcp
    balance roundrobin
    option tcplog
    option tcp-check
    server rabbit1 192.168.1.10:5672 check inter 5000 rise 2 fall 3
    server rabbit2 192.168.1.11:5672 check inter 5000 rise 2 fall 3
    server rabbit3 192.168.1.12:5672 check inter 5000 rise 2 fall 3

# Frontend for Management UI
frontend rabbitmq_management
    bind *:15672
    mode http
    default_backend rabbitmq_management_backend

backend rabbitmq_management_backend
    mode http
    balance roundrobin
    option httpchk GET /api/healthchecks/node
    server rabbit1 192.168.1.10:15672 check inter 5000
    server rabbit2 192.168.1.11:15672 check inter 5000
    server rabbit3 192.168.1.12:15672 check inter 5000

Management Plugin Usage

Key Concepts

Management Plugin: HTTP API and web UI for RabbitMQ administration
rabbitmqadmin: CLI tool that wraps the HTTP API
Monitoring: Real-time metrics, rates, and health checks
REST API: Programmatic access to management operations

Enabling Management Plugin

# Enable plugin
rabbitmq-plugins enable rabbitmq_management

# List enabled plugins
rabbitmq-plugins list

# Access web UI
# http://localhost:15672
# Default credentials: guest/guest (localhost only)

# Create admin user
rabbitmqctl add_user admin secure_password
rabbitmqctl set_user_tags admin administrator
rabbitmqctl set_permissions -p / admin ".*" ".*" ".*"

# Delete guest user (production)
rabbitmqctl delete_user guest

rabbitmqadmin CLI

# Install rabbitmqadmin
wget http://localhost:15672/cli/rabbitmqadmin
chmod +x rabbitmqadmin
mv rabbitmqadmin /usr/local/bin/

# Configure connection
rabbitmqadmin --help
rabbitmqadmin -H localhost -u admin -p password list queues

# Create config file
cat > ~/.rabbitmqadmin.conf << EOF
[default]
hostname = localhost
port = 15672
username = admin
password = secure_password
vhost = /
EOF

# List resources
rabbitmqadmin list users
rabbitmqadmin list vhosts
rabbitmqadmin list exchanges
rabbitmqadmin list queues
rabbitmqadmin list bindings
rabbitmqadmin list connections
rabbitmqadmin list channels
rabbitmqadmin list consumers
rabbitmqadmin list policies
rabbitmqadmin list parameters

# Detailed output
rabbitmqadmin list queues vhost name messages consumers memory

# Export/Import configuration
rabbitmqadmin export backup.json
rabbitmqadmin import backup.json

HTTP API Usage

Authentication:

# Basic auth
curl -u admin:password http://localhost:15672/api/overview

# Using jq for JSON parsing
curl -s -u admin:password http://localhost:15672/api/queues | jq .

Common API Endpoints:

# Cluster overview
curl -u admin:password http://localhost:15672/api/overview

# Node information
curl -u admin:password http://localhost:15672/api/nodes

# List vhosts
curl -u admin:password http://localhost:15672/api/vhosts

# List queues
curl -u admin:password http://localhost:15672/api/queues

# Queue details
curl -u admin:password http://localhost:15672/api/queues/%2f/orders

# List connections
curl -u admin:password http://localhost:15672/api/connections

# List channels
curl -u admin:password http://localhost:15672/api/channels

# Health checks (deep health-check tree under /api/health/checks/...)
curl -u admin:password http://localhost:15672/api/health/checks/alarms
curl -u admin:password http://localhost:15672/api/health/checks/port-listener/5672

# Create queue via API
curl -u admin:password -X PUT \
    -H "content-type: application/json" \
    -d '{"durable":true,"arguments":{"x-queue-type":"quorum"}}' \
    http://localhost:15672/api/queues/%2f/myqueue

# Delete queue
curl -u admin:password -X DELETE \
    http://localhost:15672/api/queues/%2f/myqueue

# Purge queue
curl -u admin:password -X DELETE \
    http://localhost:15672/api/queues/%2f/myqueue/contents

# Publish message
curl -u admin:password -X POST \
    -H "content-type: application/json" \
    -d '{"properties":{},"routing_key":"test","payload":"hello","payload_encoding":"string"}' \
    http://localhost:15672/api/exchanges/%2f/amq.default/publish

# Get messages (destructive)
curl -u admin:password -X POST \
    -H "content-type: application/json" \
    -d '{"count":5,"ackmode":"ack_requeue_false","encoding":"auto"}' \
    http://localhost:15672/api/queues/%2f/myqueue/get

Monitoring and Metrics

# View queue metrics
rabbitmqadmin list queues name messages consumers \
    message_bytes messages_ready messages_unacknowledged \
    message_stats.publish message_stats.deliver

# View connection metrics
rabbitmqadmin list connections name state channels \
    recv_oct sent_oct recv_cnt sent_cnt

# View channel metrics
rabbitmqadmin list channels connection name number \
    consumer_count messages_unacknowledged messages_uncommitted

# Memory usage
rabbitmqctl status | grep memory
rabbitmqadmin list nodes name mem_used mem_limit mem_alarm

# Disk space
rabbitmqadmin list nodes name disk_free disk_free_limit disk_free_alarm

# File descriptors
rabbitmqadmin list nodes name fd_used fd_total

Prometheus Metrics:

# Enable Prometheus plugin
rabbitmq-plugins enable rabbitmq_prometheus

# Metrics endpoint
curl http://localhost:15692/metrics

# Common metrics to monitor:
# - rabbitmq_queue_messages
# - rabbitmq_queue_messages_ready
# - rabbitmq_queue_messages_unacknowledged
# - rabbitmq_queue_consumers
# - rabbitmq_connections
# - rabbitmq_channel_count
# - rabbitmq_erlang_processes_used
# - rabbitmq_io_read_bytes_total
# - rabbitmq_io_write_bytes_total

Web UI Features

Dashboard Overview:

Total connections, channels, queues
Message rates (publish, deliver, ack)
Node health and alarms
Global message counts

Key Sections:

Connections: Active client connections, channels per connection
Channels: Message throughput per channel
Exchanges: List all exchanges and bindings
Queues: Queue details, get messages, purge, delete
Admin: User management, vhosts, policies, limits

Tracing:

# Enable via Web UI: Admin -> Tracing
# Or via CLI:
rabbitmq-plugins enable rabbitmq_tracing

# Create trace
rabbitmqctl trace_on

# Pattern matching (all messages to/from vhost)
# Log file: /var/tmp/rabbitmq-tracing/

# Disable trace
rabbitmqctl trace_off

TLS and Authentication Hardening

Key Concepts

TLS: Transport Layer Security for encrypted client connections
mTLS: Mutual TLS requiring client certificates
x509 Authentication: Using client certificates for authentication
SASL Mechanisms: PLAIN, AMQPLAIN, EXTERNAL (for x509)
Virtual Host Permissions: Read, write, configure access control

TLS Configuration

Generate Certificates:

# Using easy-rsa or openssl
# Create CA certificate
openssl genrsa -out ca_key.pem 2048
openssl req -new -x509 -days 3650 -key ca_key.pem -out ca_certificate.pem \
    -subj "/CN=MyCA"

# Create server certificate
openssl genrsa -out server_key.pem 2048
openssl req -new -key server_key.pem -out server.csr \
    -subj "/CN=rabbit1.example.com"

openssl x509 -req -in server.csr -CA ca_certificate.pem \
    -CAkey ca_key.pem -CAcreateserial -out server_certificate.pem \
    -days 365

# Create client certificate
openssl genrsa -out client_key.pem 2048
openssl req -new -key client_key.pem -out client.csr \
    -subj "/CN=client"

openssl x509 -req -in client.csr -CA ca_certificate.pem \
    -CAkey ca_key.pem -CAcreateserial -out client_certificate.pem \
    -days 365

# Set permissions
chmod 600 *_key.pem
chown rabbitmq:rabbitmq *.pem

Configure RabbitMQ for TLS:

% /etc/rabbitmq/rabbitmq.conf

% Standard AMQP port (disable if TLS-only)
listeners.tcp.default = 5672

% TLS listener
listeners.ssl.default = 5671

% Paths to certificates
ssl_options.cacertfile = /etc/rabbitmq/certs/ca_certificate.pem
ssl_options.certfile   = /etc/rabbitmq/certs/server_certificate.pem
ssl_options.keyfile    = /etc/rabbitmq/certs/server_key.pem

% Require valid client certificate (mTLS)
ssl_options.verify     = verify_peer
ssl_options.fail_if_no_peer_cert = true

% TLS versions
ssl_options.versions.1 = tlsv1.3
ssl_options.versions.2 = tlsv1.2

% Cipher suites (modern, secure)
ssl_options.ciphers.1 = TLS_AES_256_GCM_SHA384
ssl_options.ciphers.2 = TLS_AES_128_GCM_SHA256
ssl_options.ciphers.3 = TLS_CHACHA20_POLY1305_SHA256
ssl_options.ciphers.4 = TLS_AES_128_CCM_SHA256

% Honour server cipher order
ssl_options.honor_cipher_order = true

% Session caching
ssl_options.reuse_sessions = true

% Depth of certificate chain verification
ssl_options.depth = 2

% Management UI over HTTPS
management.ssl.port       = 15671
management.ssl.cacertfile = /etc/rabbitmq/certs/ca_certificate.pem
management.ssl.certfile   = /etc/rabbitmq/certs/server_certificate.pem
management.ssl.keyfile    = /etc/rabbitmq/certs/server_key.pem

Client Connection with TLS:

# Python pika example
import ssl
import pika

context = ssl.create_default_context(cafile="/path/to/ca_certificate.pem")
context.load_cert_chain(
    certfile="/path/to/client_certificate.pem",
    keyfile="/path/to/client_key.pem"
)

ssl_options = pika.SSLOptions(context, "rabbit1.example.com")

parameters = pika.ConnectionParameters(
    host='rabbit1.example.com',
    port=5671,
    ssl_options=ssl_options,
    credentials=pika.PlainCredentials('username', 'password')
)

connection = pika.BlockingConnection(parameters)

x509 Certificate Authentication

Enable x509 Plugin:

# Enable authentication plugin
rabbitmq-plugins enable rabbitmq_auth_mechanism_ssl

# Configure to extract username from certificate DN

Configuration:

% /etc/rabbitmq/rabbitmq.conf

% Enable x509 authentication
auth_mechanisms.1 = PLAIN
auth_mechanisms.2 = AMQPLAIN
auth_mechanisms.3 = EXTERNAL

% Extract username from certificate Common Name
ssl_cert_login_from = common_name

% Or from Distinguished Name
% ssl_cert_login_from = distinguished_name

Create User from Certificate:

# Extract CN from client certificate
openssl x509 -in client_certificate.pem -noout -subject
# subject=CN = client_app

# Create user matching CN
rabbitmqctl add_user client_app ""
rabbitmqctl set_permissions -p / client_app ".*" ".*" ".*"

# User authenticates via certificate only (no password)

Client Connection with x509:

# Python pika with certificate authentication
context = ssl.create_default_context(cafile="/path/to/ca_certificate.pem")
context.load_cert_chain(
    certfile="/path/to/client_certificate.pem",
    keyfile="/path/to/client_key.pem"
)

ssl_options = pika.SSLOptions(context, "rabbit1.example.com")

# No credentials needed - authenticated via certificate
parameters = pika.ConnectionParameters(
    host='rabbit1.example.com',
    port=5671,
    ssl_options=ssl_options
)

connection = pika.BlockingConnection(parameters)

Authentication and Authorisation

User Management:

# Create user
rabbitmqctl add_user username password

# Set tags (administrator, monitoring, management, policymaker)
rabbitmqctl set_user_tags username administrator

# Change password
rabbitmqctl change_password username newpassword

# List users
rabbitmqctl list_users

# Delete user
rabbitmqctl delete_user username

# Clear tags
rabbitmqctl set_user_tags username

Permissions (per vhost):

# Format: rabbitmqctl set_permissions [-p vhost] user configure write read

# Full access
rabbitmqctl set_permissions -p / username ".*" ".*" ".*"

# Read-only access
rabbitmqctl set_permissions -p / readonly "" "" ".*"

# Specific queue patterns
rabbitmqctl set_permissions -p / app_user "^app-.*" "^app-.*" "^app-.*"

# List permissions
rabbitmqctl list_permissions -p /
rabbitmqctl list_user_permissions username

# Clear permissions
rabbitmqctl clear_permissions -p / username

Permission Patterns:

Configure: Create/delete exchanges, queues, bindings
Write: Publish messages to exchanges
Read: Consume from queues, get messages, purge

# Examples:
# Publisher only (no queue creation)
rabbitmqctl set_permissions -p / publisher "" ".*" ""

# Consumer only (no publishing)
rabbitmqctl set_permissions -p / consumer "" "" ".*"

# Specific exchange/queue namespace
rabbitmqctl set_permissions -p / service1 "^service1\\..*" "^service1\\..*" "^service1\\..*"

Virtual Hosts

# Create vhost
rabbitmqctl add_vhost /production

# List vhosts
rabbitmqctl list_vhosts

# Delete vhost (deletes all queues/exchanges)
rabbitmqctl delete_vhost /production

# Grant user access to vhost
rabbitmqctl set_permissions -p /production username ".*" ".*" ".*"

# Set resource limits per vhost
rabbitmqctl set_vhost_limits -p /production '{"max-connections": 500}'
rabbitmqctl set_vhost_limits -p /production '{"max-queues": 1000}'

Security Hardening Checklist

Network Security:

# Disable guest user
rabbitmqctl delete_user guest

# Bind management to localhost only (if using reverse proxy)
# rabbitmq.conf:
# management.tcp.ip = 127.0.0.1

# Use TLS for all client connections
# Disable non-TLS listener:
# listeners.tcp = none

# Enable firewall rules
ufw allow from 10.0.0.0/8 to any port 5671 proto tcp
ufw allow from 10.0.0.0/8 to any port 15671 proto tcp

Access Control:

# Principle of least privilege - specific patterns per user
rabbitmqctl set_permissions -p / app1 "^app1-.*" "^app1-.*" "^app1-.*"

# Separate vhosts for different applications
rabbitmqctl add_vhost /app1
rabbitmqctl add_vhost /app2

# Use strong passwords or certificate authentication
rabbitmqctl add_user admin "$(openssl rand -base64 32)"

# Regular audit of users and permissions
rabbitmqctl list_users
rabbitmqctl list_permissions -p /

Monitoring and Logging:

# Enable audit logging
# rabbitmq.conf:
# log.file.level = info
# log.connection.level = info
# log.channel.level = info

# Monitor failed authentication attempts
journalctl -u rabbitmq-server | grep -i "failed"

# Set up alarms
rabbitmqctl set_vm_memory_high_watermark 0.5
rabbitmqctl set_disk_free_limit 2GB

Rate Limiting:

# Connection limits
rabbitmqctl set_vhost_limits -p / '{"max-connections": 1000}'

# Per-user connection limits
rabbitmqctl set_user_limits username '{"max-connections": 10}'

# Per-channel limits
rabbitmqctl set_vhost_limits -p / '{"max-channels": 5000}'

Quick Reference

Essential rabbitmqctl Commands

# Cluster
rabbitmqctl cluster_status
rabbitmqctl join_cluster rabbit@node1
rabbitmqctl forget_cluster_node rabbit@node2

# Queues
rabbitmqctl list_queues name messages consumers memory
rabbitmqctl purge_queue queue_name
rabbitmqctl delete_queue queue_name

# Exchanges
rabbitmqctl list_exchanges name type durable

# Bindings
rabbitmqctl list_bindings

# Users & Permissions
rabbitmqctl add_user username password
rabbitmqctl set_user_tags username administrator
rabbitmqctl set_permissions -p / username ".*" ".*" ".*"
rabbitmqctl list_users
rabbitmqctl list_permissions -p /

# Policies (ha-mode removed in 4.0 — set a valid key such as message-ttl)
rabbitmqctl set_policy ttl-1h "^temp\." '{"message-ttl":3600000}'
rabbitmqctl list_policies
rabbitmqctl clear_policy ttl-1h

# Virtual Hosts
rabbitmqctl add_vhost /production
rabbitmqctl list_vhosts
rabbitmqctl delete_vhost /staging

# Monitoring
rabbitmqctl status
rabbitmqctl environment
rabbitmqctl list_connections
rabbitmqctl list_channels

# Node Management
rabbitmqctl stop
rabbitmqctl stop_app
rabbitmqctl start_app
rabbitmqctl reset
rabbitmqctl shutdown

rabbitmqadmin Common Commands

# Declare resources
rabbitmqadmin declare exchange name=myexch type=direct
rabbitmqadmin declare queue name=myqueue durable=true
rabbitmqadmin declare binding source=myexch destination=myqueue routing_key=test

# List resources
rabbitmqadmin list exchanges
rabbitmqadmin list queues
rabbitmqadmin list bindings

# Publish/Consume
rabbitmqadmin publish exchange=myexch routing_key=test payload="test message"
rabbitmqadmin get queue=myqueue ackmode=ack_requeue_false

# Export/Import
rabbitmqadmin export config.json
rabbitmqadmin import config.json

# Delete resources
rabbitmqadmin delete queue name=myqueue
rabbitmqadmin delete exchange name=myexch

Configuration File Locations

# Main config
/etc/rabbitmq/rabbitmq.conf          # Modern format
/etc/rabbitmq/advanced.config        # Erlang format

# Environment
/etc/rabbitmq/rabbitmq-env.conf

# Enabled plugins
/etc/rabbitmq/enabled_plugins

# Erlang cookie
/var/lib/rabbitmq/.erlang.cookie

# Logs
/var/log/rabbitmq/

# Data directory
/var/lib/rabbitmq/mnesia/

Common Issues and Solutions

Queue Performance Issues

Problem: Slow message consumption

# Check queue backlogs
rabbitmqctl list_queues name messages messages_ready messages_unacknowledged consumers

# Check consumer utilisation
rabbitmqadmin list channels name consumer_count prefetch_count

# Solution: Increase consumers or adjust prefetch
# In consumer application:
# channel.basic_qos(prefetch_count=100)  # Process multiple messages

# Check for memory alarms
rabbitmqctl status | grep mem_alarm

# Solution: Increase the memory high-watermark, or move large backlogs to
# quorum queues (always on disk). Note: the classic "lazy" queue-mode policy
# is ignored from 3.12 onwards — CQv2 already keeps messages on disk by
# default, so there is nothing to enable.
rabbitmqctl set_vm_memory_high_watermark 0.6

Problem: Messages accumulating (dead consumer)

# Identify queue with dead consumers
rabbitmqadmin list queues name consumers messages

# Check consumer connection status
rabbitmqadmin list consumers queue_name channel consumer_tag

# Solution: Implement TTL and DLX
rabbitmqctl set_policy ttl-dlx "^monitored\." \
    '{"message-ttl":3600000,"dead-letter-exchange":"dlx"}'

Cluster Issues

Problem: Node won't join cluster

# Check Erlang cookie matches
cat /var/lib/rabbitmq/.erlang.cookie  # Must be identical

# Check network connectivity
telnet node1 4369  # epmd
telnet node1 25672  # inter-node

# Check hostnames resolve
ping rabbit@node1

# Solution: Reset and rejoin
rabbitmqctl stop_app
rabbitmqctl reset
rabbitmqctl join_cluster rabbit@node1
rabbitmqctl start_app

Problem: Cluster partition detected

# Check partition status
rabbitmqctl cluster_status

# Solution: Restart minority nodes
rabbitmqctl stop_app
rabbitmqctl start_app

# For persistent partitions, manually recover
rabbitmqctl forget_cluster_node rabbit@problematic_node

# Configure partition handling
# rabbitmq.conf:
# cluster_partition_handling = autoheal
# or
# cluster_partition_handling = pause_minority

Memory Issues

Problem: Memory alarm triggered

# Check memory usage
rabbitmqctl status | grep memory

# Check per-node memory
rabbitmqadmin list nodes name mem_used mem_limit mem_alarm

# Identify memory-heavy queues
rabbitmqadmin list queues name memory messages

# Solution: Purge unnecessary messages
rabbitmqctl purge_queue queue_name

# Solution: Move large/durable backlogs to quorum queues, which are always on
# disk. (The old '{"queue-mode":"lazy"}' policy is a no-op from 3.12 — CQv2
# classic queues already page to disk.)

# Solution: Increase memory limit (50% of RAM by default)
# rabbitmq.conf:
# vm_memory_high_watermark.relative = 0.6

Problem: Disk space alarm

# Check disk space
df -h /var/lib/rabbitmq

# Check alarm status
rabbitmqctl status | grep disk_free_alarm

# Solution: Increase disk space or clear old logs
rm -f /var/log/rabbitmq/*.log.gz

# Set disk free limit
# rabbitmq.conf:
# disk_free_limit.absolute = 10GB

Connection Issues

Problem: Connection refused

# Check RabbitMQ is running
systemctl status rabbitmq-server

# Check listening ports
netstat -tlnp | grep beam.smp

# Check firewall
ufw status
iptables -L -n | grep 5672

# Check logs
journalctl -u rabbitmq-server -n 100
tail -f /var/log/rabbitmq/rabbit@hostname.log

# Solution: Ensure service running and ports accessible
systemctl start rabbitmq-server
ufw allow 5672/tcp

Problem: TLS handshake failures

# Test TLS connection
openssl s_client -connect localhost:5671 -CAfile ca_certificate.pem

# Check certificate validity
openssl x509 -in server_certificate.pem -noout -dates -subject

# Verify certificate chain
openssl verify -CAfile ca_certificate.pem server_certificate.pem

# Check logs for TLS errors
grep -i ssl /var/log/rabbitmq/*.log

# Common issues:
# - Hostname mismatch (CN doesn't match server hostname)
# - Expired certificate
# - Incomplete certificate chain
# - Client certificate not trusted

Performance Tuning

High throughput publishing:

# Enable publisher confirms for reliability
# Use batching in application code
# Increase TCP buffer sizes in rabbitmq.conf:
# tcp_listen_options.sndbuf = 196608
# tcp_listen_options.recvbuf = 196608
# tcp_listen_options.backlog = 128

# Use quorum queues for durability
rabbitmqadmin declare queue name=high_throughput \
    arguments='{"x-queue-type":"quorum"}'

# Disable disk sync for non-critical data (increases risk)
# rabbitmq.conf:
# queue_index_embed_msgs_below = 4096

Low latency:

# Use classic queues (not quorum)
# Minimise durability overhead
# Use non-durable queues for transient data
rabbitmqadmin declare queue name=transient durable=false

# Increase prefetch for consumers
# channel.basic_qos(prefetch_count=1000)

# Use direct exchange (fastest routing)

Large messages:

# Increase max message size
# rabbitmq.conf:
# max_message_size = 134217728  # 128MB

# Consider chunking messages in application
# Use external storage (S3) with message references
# (HiPE was removed in Erlang/OTP 24 — no longer an option on modern brokers)

Debugging Techniques

# Enable verbose logging
rabbitmqctl trace_on

# View real-time connection activity
rabbitmqctl list_connections name peer_host peer_port state channels

# Monitor channel activity
rabbitmqctl list_channels connection name number consumer_count \
    messages_unacknowledged

# Capture network traffic
tcpdump -i any -w rabbitmq.pcap port 5672

# Query management API for detailed stats
curl -s -u admin:password http://localhost:15672/api/queues/%2f/myqueue | jq .

# Check Erlang VM statistics
rabbitmqctl status | grep -A 20 statistics

# View queue internals (look the queue record up first, then ask for fields)
rabbitmqctl eval '{ok, Q} = rabbit_amqqueue:lookup(rabbit_misc:r(<<"/">>, queue, <<"myqueue">>)), rabbit_amqqueue:info(Q, [name, type, state]).'

This comprehensive guide covers the essential aspects of RabbitMQ administration. For production deployments, always test configurations in staging environments, monitor cluster health continuously, implement proper backup procedures for metadata and message stores, and maintain documentation of your topology and policies.