Cert-Manager
A Kubernetes add-on for automating the management and issuance of TLS certificates from various sources.
Cert-Manager Cheatsheet
A Kubernetes add-on for automating the management and issuance of TLS certificates from various sources.
Overview
Cert-Manager is a native Kubernetes certificate management controller that automates the creation, renewal, and management of TLS certificates. It supports multiple certificate authorities including Let's Encrypt, HashiCorp Vault, Venafi, and self-signed certificates, making it essential for securing Kubernetes workloads.
graph TB
subgraph "Cert-Manager Architecture"
CM[Cert-Manager Controller]
subgraph "Custom Resources"
Cert[Certificate]
Issuer[Issuer]
ClusterIssuer[ClusterIssuer]
CR[CertificateRequest]
Order[Order]
Challenge[Challenge]
end
subgraph "Certificate Authorities"
LE["Let's Encrypt"]
Vault[HashiCorp Vault]
Self[Self-Signed]
CA[CA Issuer]
end
subgraph "Challenge Solvers"
HTTP[HTTP01]
DNS[DNS01]
end
end
Cert --> CM
CM --> CR
CR --> Issuer
CR --> ClusterIssuer
Issuer --> LE
Issuer --> Vault
Issuer --> Self
Issuer --> CA
LE --> Order
Order --> Challenge
Challenge --> HTTP
Challenge --> DNS
Installation
Key Concepts
- CRDs: Custom Resource Definitions that extend Kubernetes API
- Namespace: Cert-Manager typically runs in
cert-managernamespace - Webhook: Validates and mutates cert-manager resources
- Controller: Watches for Certificate resources and manages issuance
Installation Methods
# Install using kubectl
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.20.2/cert-manager.yaml
# Install using Helm
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v1.20.2 \
--set crds.enabled=true # installCRDs is deprecated since v1.15
# Verify installation
kubectl get pods -n cert-manager
kubectl get crd | grep cert-manager
Verify Installation
# Check cert-manager pods are running
kubectl get pods -n cert-manager
# Expected output:
# cert-manager-xxxxxxx-xxxxx 1/1 Running
# cert-manager-cainjector-xxxxx 1/1 Running
# cert-manager-webhook-xxxxx 1/1 Running
# Check CRDs are installed
kubectl get crd | grep cert-manager
# Test with a self-signed issuer
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
name: cert-manager-test
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: test-selfsigned
namespace: cert-manager-test
spec:
selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: selfsigned-cert
namespace: cert-manager-test
spec:
dnsNames:
- example.com
secretName: selfsigned-cert-tls
issuerRef:
name: test-selfsigned
EOF
# Check certificate status
kubectl get certificate -n cert-manager-test
# Clean up test
kubectl delete namespace cert-manager-test
Certificate Resources
Key Concepts
- Certificate: Defines desired certificate properties and triggers issuance
- CertificateRequest: Internal resource representing a single certificate request
- Secret: Kubernetes Secret containing the issued certificate and private key
- Renewal: Automatic re-issuance before certificate expiry
flowchart LR
A[Certificate] -->|Creates| B[CertificateRequest]
B -->|Processed by| C[Issuer/ClusterIssuer]
C -->|Issues| D[TLS Secret]
D -->|Contains| E[tls.crt + tls.key]
style A fill:#e1f5fe
style D fill:#c8e6c9
Certificate Resource
# Full Certificate specification
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: example-com
namespace: default
spec:
# Secret to store the certificate
secretName: example-com-tls
# Duration and renewal settings
duration: 2160h # 90 days
renewBefore: 360h # Renew 15 days before expiry
# Subject fields
subject:
organizations:
- My Organisation
# Common name (legacy, prefer dnsNames)
commonName: example.com
# Private key settings
privateKey:
algorithm: RSA
encoding: PKCS1
size: 2048
rotationPolicy: Always # or Never
# Usages define what the certificate can be used for
usages:
- server auth
- client auth
# DNS names for the certificate (SAN)
dnsNames:
- example.com
- www.example.com
- "*.example.com" # Wildcard
# IP addresses (SAN)
ipAddresses:
- 192.168.1.1
# URI SANs
uris:
- spiffe://cluster.local/ns/default/sa/example
# Email addresses (SAN)
emailAddresses:
- admin@example.com
# Reference to Issuer
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer # or Issuer
group: cert-manager.io
Issuer Resource
# Namespace-scoped Issuer
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: letsencrypt-staging
namespace: default
spec:
acme:
server: https://acme-staging-v02.api.letsencrypt.org/directory
email: admin@example.com
privateKeySecretRef:
name: letsencrypt-staging-key
solvers:
- http01:
ingress:
class: nginx
ClusterIssuer Resource
# Cluster-wide Issuer (available in all namespaces)
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: admin@example.com
privateKeySecretRef:
name: letsencrypt-prod-key
solvers:
- http01:
ingress:
class: nginx
Common Commands
# List certificates
kubectl get certificates -A
kubectl get cert -A # Short form
# Describe certificate details
kubectl describe certificate example-com
# Check certificate status
kubectl get certificate example-com -o jsonpath='{.status.conditions}'
# View the generated secret
kubectl get secret example-com-tls -o yaml
# Decode certificate from secret
kubectl get secret example-com-tls -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -text -noout
# Check certificate expiry
kubectl get secret example-com-tls -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -enddate -noout
# List issuers
kubectl get issuers -A
kubectl get clusterissuers
# Force certificate renewal (recommended)
cmctl renew example-com
# Deleting the secret also triggers re-issuance, but is NOT recommended
# (it does not rotate the key cleanly): kubectl delete secret example-com-tls
# View certificate requests
kubectl get certificaterequests -A
kubectl get cr -A # Short form
Common Issuers
Key Concepts
- Self-Signed: Generates certificates signed by its own private key
- CA Issuer: Uses a provided CA certificate and key to sign certificates
- ACME: Automated Certificate Management Environment (Let's Encrypt)
- Vault: Integrates with HashiCorp Vault PKI backend
- Venafi: Enterprise certificate management platform
flowchart TB
subgraph "Issuer Types"
direction LR
SS[Self-Signed]
CA[CA Issuer]
ACME["ACME/Let's Encrypt"]
VLT[Vault PKI]
end
SS -->|Use for| Dev[Development/Testing]
CA -->|Use for| Int[Internal Services]
ACME -->|Use for| Pub[Public-Facing Services]
VLT -->|Use for| Ent[Enterprise/PKI Integration]
Self-Signed Issuer
# Self-signed issuer (for development/testing)
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: selfsigned-issuer
spec:
selfSigned: {}
---
# Certificate using self-signed issuer
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: my-selfsigned-cert
namespace: default
spec:
secretName: my-selfsigned-cert-tls
duration: 8760h # 1 year
renewBefore: 720h
commonName: my-app.local
dnsNames:
- my-app.local
- localhost
issuerRef:
name: selfsigned-issuer
kind: ClusterIssuer
CA Issuer
# First, create a self-signed CA
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: my-ca
namespace: cert-manager
spec:
isCA: true
commonName: My Organisation CA
secretName: my-ca-secret
duration: 87600h # 10 years
renewBefore: 8760h
privateKey:
algorithm: ECDSA
size: 256
issuerRef:
name: selfsigned-issuer
kind: ClusterIssuer
---
# ClusterIssuer using the CA
# NOTE: a ClusterIssuer's ca.secretName is read from the cert-manager
# controller's own namespace (default: cert-manager), NOT from the namespace
# of the Certificate. Put my-ca-secret there, or it fails silently.
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: my-ca-issuer
spec:
ca:
secretName: my-ca-secret
---
# Certificate signed by CA
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: internal-app-cert
namespace: default
spec:
secretName: internal-app-tls
duration: 2160h
renewBefore: 360h
dnsNames:
- internal-app.company.local
issuerRef:
name: my-ca-issuer
kind: ClusterIssuer
ACME Issuer (Let's Encrypt)
# Staging issuer (for testing - higher rate limits)
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
email: admin@example.com
server: https://acme-staging-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-staging-account-key
solvers:
- http01:
ingress:
class: nginx
---
# Production issuer
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
email: admin@example.com
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-prod-account-key
solvers:
- http01:
ingress:
class: nginx
Vault PKI Issuer
# Vault issuer for enterprise PKI
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: vault-issuer
namespace: default
spec:
vault:
server: https://vault.example.com
path: pki_int/sign/example-dot-com
caBundle: <base64-encoded-ca-cert>
auth:
kubernetes:
role: cert-manager
mountPath: /v1/auth/kubernetes
secretRef:
name: vault-token
key: token
---
# Alternative: AppRole authentication
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
name: vault-approle-issuer
namespace: default
spec:
vault:
server: https://vault.example.com
path: pki_int/sign/example-dot-com
auth:
appRole:
path: approle
roleId: "db02de05-fa39-4855-059b-67221c5c2f63"
secretRef:
name: vault-approle-secret
key: secretId
ACME/Let's Encrypt Integration
Key Concepts
- ACME Protocol: Standard protocol for automated certificate issuance
- Account: Registration with the ACME server
- Order: Request for a certificate
- Challenge: Proof of domain ownership
- Authorisation: Permission to issue certificates for a domain
sequenceDiagram
participant CM as Cert-Manager
participant LE as Let's Encrypt
participant DNS as DNS Provider
participant Web as Web Server
CM->>LE: Create Account
LE-->>CM: Account Created
CM->>LE: Create Order (domains)
LE-->>CM: Challenges Required
alt HTTP01 Challenge
CM->>Web: Create /.well-known/acme-challenge/token
LE->>Web: Verify Challenge
Web-->>LE: Token + Thumbprint
else DNS01 Challenge
CM->>DNS: Create _acme-challenge TXT record
LE->>DNS: Verify TXT Record
DNS-->>LE: Challenge Token
end
LE-->>CM: Authorisation Complete
CM->>LE: Finalise Order
LE-->>CM: Certificate Issued
Let's Encrypt Endpoints
# Staging (testing) - not trusted by browsers
# Higher rate limits for testing
server: https://acme-staging-v02.api.letsencrypt.org/directory
# Production - trusted certificates
# Strict rate limits apply
server: https://acme-v02.api.letsencrypt.org/directory
Rate Limits
# Let's Encrypt Production Rate Limits:
# - 50 certificates per registered domain per week
# - 5 duplicate certificates per week
# - 300 new orders per account per 3 hours
# - 10 accounts per IP per 3 hours
# - 5 failed validations per account per hour
# Always test with staging first!
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
server: https://acme-staging-v02.api.letsencrypt.org/directory
email: admin@example.com
privateKeySecretRef:
name: letsencrypt-staging-key
solvers:
- http01:
ingress:
class: nginx
Multiple Domains and Wildcards
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: wildcard-example-com
namespace: default
spec:
secretName: wildcard-example-com-tls
dnsNames:
- example.com
- "*.example.com" # Wildcard requires DNS01 challenge
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
DNS01 and HTTP01 Challenges
Key Concepts
- HTTP01: Proves domain ownership via HTTP endpoint on port 80
- DNS01: Proves domain ownership via DNS TXT record
- Solver: Configuration for how to complete challenges
- Selector: Matches certificates to specific solvers
HTTP01 Challenge
flowchart LR
A[Cert-Manager] -->|Creates| B["Challenge Pod/Ingress"]
B -->|Serves| C["/.well-known/acme-challenge/token"]
D["Let's Encrypt"] -->|Requests| C
C -->|Returns| D
D -->|Validates| A
# HTTP01 with Ingress controller
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-http01
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: admin@example.com
privateKeySecretRef:
name: letsencrypt-http01-key
solvers:
- http01:
ingress:
class: nginx
---
# HTTP01 with Gateway API
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-gateway
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: admin@example.com
privateKeySecretRef:
name: letsencrypt-gateway-key
solvers:
- http01:
gatewayHTTPRoute:
parentRefs:
- name: my-gateway
namespace: default
kind: Gateway
---
# HTTP01 with specific service type
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-http01-nodeport
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: admin@example.com
privateKeySecretRef:
name: letsencrypt-http01-key
solvers:
- http01:
ingress:
class: nginx
serviceType: NodePort
podTemplate:
spec:
nodeSelector:
kubernetes.io/os: linux
DNS01 Challenge
# DNS01 with Route53
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-dns01-route53
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: admin@example.com
privateKeySecretRef:
name: letsencrypt-dns01-key
solvers:
- dns01:
route53:
region: eu-west-1
hostedZoneID: Z1234567890ABC # Optional
accessKeyIDSecretRef:
name: route53-credentials
key: access-key-id
secretAccessKeySecretRef:
name: route53-credentials
key: secret-access-key
---
# DNS01 with CloudFlare
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-dns01-cloudflare
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: admin@example.com
privateKeySecretRef:
name: letsencrypt-dns01-key
solvers:
- dns01:
cloudflare:
email: cloudflare@example.com
apiTokenSecretRef:
name: cloudflare-api-token
key: api-token
---
# DNS01 with Google Cloud DNS
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-dns01-clouddns
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: admin@example.com
privateKeySecretRef:
name: letsencrypt-dns01-key
solvers:
- dns01:
cloudDNS:
project: my-gcp-project
serviceAccountSecretRef:
name: clouddns-service-account
key: key.json
---
# DNS01 with Azure DNS
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-dns01-azuredns
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: admin@example.com
privateKeySecretRef:
name: letsencrypt-dns01-key
solvers:
- dns01:
azureDNS:
subscriptionID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
resourceGroupName: my-resource-group
hostedZoneName: example.com
environment: AzurePublicCloud
managedIdentity:
clientID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Multiple Solvers with Selectors
# Different solvers for different domains
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-multi
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: admin@example.com
privateKeySecretRef:
name: letsencrypt-multi-key
solvers:
# Wildcard certificates must use DNS01
- selector:
dnsNames:
- "*.example.com"
dns01:
cloudflare:
apiTokenSecretRef:
name: cloudflare-api-token
key: api-token
# Specific zone uses Route53
- selector:
dnsZones:
- "internal.example.com"
dns01:
route53:
region: eu-west-1
# Default to HTTP01 for everything else
- http01:
ingress:
class: nginx
Creating DNS Provider Secrets
# Route53 credentials
kubectl create secret generic route53-credentials \
--namespace cert-manager \
--from-literal=access-key-id=AKIAIOSFODNN7EXAMPLE \
--from-literal=secret-access-key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
# CloudFlare API token
kubectl create secret generic cloudflare-api-token \
--namespace cert-manager \
--from-literal=api-token=your-cloudflare-api-token
# Google Cloud DNS service account
kubectl create secret generic clouddns-service-account \
--namespace cert-manager \
--from-file=key.json=./service-account.json
Certificate Renewal
Key Concepts
- Automatic Renewal: Cert-Manager automatically renews before expiry
- renewBefore: Time before expiry to trigger renewal
- rotationPolicy: Controls private key rotation on renewal
- Renewal Window: Period when renewal is triggered
stateDiagram-v2
[*] --> Issued
Issued --> Valid : Certificate Active
Valid --> RenewalPending : renewBefore threshold reached
RenewalPending --> Renewing : Trigger Renewal
Renewing --> Issued : New Certificate
Renewing --> Failed : Error
Failed --> RenewalPending : Retry
Renewal Configuration
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: auto-renew-cert
namespace: default
spec:
secretName: auto-renew-tls
# Certificate valid for 90 days
duration: 2160h
# Renew 30 days before expiry
renewBefore: 720h
# Rotate private key on each renewal
privateKey:
rotationPolicy: Always # or Never
dnsNames:
- example.com
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
Monitoring Renewal Status
# Check certificate status
kubectl get certificate -A
# View certificate conditions
kubectl get certificate example-com -o jsonpath='{.status.conditions}' | jq
# Check renewal time
kubectl get certificate example-com -o jsonpath='{.status.renewalTime}'
# Check not after (expiry)
kubectl get certificate example-com -o jsonpath='{.status.notAfter}'
# View certificate events
kubectl describe certificate example-com
# Check all certificates nearing expiry
kubectl get certificates -A -o custom-columns=\
'NAMESPACE:.metadata.namespace,NAME:.metadata.name,READY:.status.conditions[0].status,EXPIRY:.status.notAfter,RENEWAL:.status.renewalTime'
Force Renewal
# Method 1: Use cmctl (recommended - sets the Issuing condition)
cmctl renew example-com
# Method 2: Delete and recreate the certificate request
kubectl delete certificaterequest -l cert-manager.io/certificate-name=example-com
# Method 3: Delete the secret (triggers re-issuance, but NOT recommended -
# does not rotate the private key cleanly)
kubectl delete secret example-com-tls
Renewal Events
# Certificate events during renewal
Events:
Type Reason Message
---- ------ -------
Normal Issuing Issuing certificate as Secret does not exist
Normal Generated Stored new private key in temporary Secret
Normal Requested Created new CertificateRequest resource
Normal Issuing The certificate has been successfully issued
Ingress Integration
Key Concepts
- Annotations: Trigger automatic certificate creation
- TLS Section: References the certificate secret
- Ingress Shim: Automatically creates Certificate resources
Automatic Certificate via Ingress Annotations
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example-ingress
annotations:
# Specify the issuer
cert-manager.io/cluster-issuer: "letsencrypt-prod"
# Or for namespace issuer
# cert-manager.io/issuer: "letsencrypt-staging"
# Optional: specify certificate duration
cert-manager.io/duration: "2160h"
cert-manager.io/renew-before: "360h"
# Optional: common name
cert-manager.io/common-name: "example.com"
spec:
ingressClassName: nginx
tls:
- hosts:
- example.com
- www.example.com
secretName: example-com-tls # Secret will be created
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: example-service
port:
number: 80
Manual Certificate with Ingress
# Create certificate first
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: example-com
namespace: default
spec:
secretName: example-com-tls
dnsNames:
- example.com
- www.example.com
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
---
# Reference in Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example-ingress
spec:
ingressClassName: nginx
tls:
- hosts:
- example.com
- www.example.com
secretName: example-com-tls # Pre-created by Certificate
rules:
- host: example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: example-service
port:
number: 80
Gateway API Integration
# Gateway with TLS
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: example-gateway
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
gatewayClassName: nginx
listeners:
- name: https
port: 443
protocol: HTTPS
hostname: example.com
tls:
mode: Terminate
certificateRefs:
- name: example-com-tls
kind: Secret
Quick Reference
Common Commands
| Command | Description |
|---|---|
kubectl get certificates -A |
List all certificates |
kubectl get clusterissuers |
List cluster-wide issuers |
kubectl get issuers -A |
List namespace issuers |
kubectl describe cert <name> |
Show certificate details and events |
kubectl get certificaterequests -A |
List certificate requests |
kubectl get orders -A |
List ACME orders |
kubectl get challenges -A |
List active challenges |
kubectl delete secret <tls-secret> |
Force certificate re-issuance |
cmctl status certificate <name> |
Check certificate status (cmctl) |
cmctl renew <name> |
Force certificate renewal (cmctl) |
cmctl check api |
Verify cert-manager API |
Certificate Status Values
| Status | Description |
|---|---|
True |
Certificate is ready and valid |
False |
Certificate issuance failed |
Unknown |
Status not yet determined |
Challenge Types Comparison
| Feature | HTTP01 | DNS01 |
|---|---|---|
| Port required | 80 | None |
| Wildcard support | No | Yes |
| Private networks | No | Yes |
| Setup complexity | Low | Medium |
| DNS provider needed | No | Yes |
Common Annotations
| Annotation | Description |
|---|---|
cert-manager.io/cluster-issuer |
ClusterIssuer to use |
cert-manager.io/issuer |
Namespace Issuer to use |
cert-manager.io/duration |
Certificate duration |
cert-manager.io/renew-before |
Renewal threshold |
cert-manager.io/common-name |
Certificate CN |
cert-manager.io/private-key-rotation-policy |
Key rotation policy |
Important Paths
| Path | Purpose |
|---|---|
/.well-known/acme-challenge/ |
HTTP01 challenge tokens |
_acme-challenge.<domain> |
DNS01 TXT record name |
Common Issues and Solutions
Issue: Certificate Stuck in "Issuing" State
Symptoms: Certificate shows Ready: False with reason Issuing.
Diagnosis:
# Check certificate status
kubectl describe certificate <name>
# Check certificate request
kubectl get certificaterequest -l cert-manager.io/certificate-name=<name>
kubectl describe certificaterequest <cr-name>
# For ACME, check orders
kubectl get orders -l cert-manager.io/certificate-name=<name>
kubectl describe order <order-name>
# Check challenges
kubectl get challenges -A
kubectl describe challenge <challenge-name>
Solutions:
# HTTP01: Verify ingress is accessible
curl -v http://example.com/.well-known/acme-challenge/test
# DNS01: Verify TXT record
dig TXT _acme-challenge.example.com
# Check cert-manager logs
kubectl logs -n cert-manager -l app=cert-manager -f
# Delete and recreate certificate request
kubectl delete certificaterequest -l cert-manager.io/certificate-name=<name>
Issue: HTTP01 Challenge Failing
Symptoms: Challenge stuck in pending or invalid state.
Solutions:
# Verify ingress controller is running
kubectl get pods -n ingress-nginx
# Check challenge pod
kubectl get pods -l acme.cert-manager.io/http01-solver=true
# Verify the challenge endpoint is accessible
kubectl describe challenge <name>
# Look for the token and test:
curl -v http://example.com/.well-known/acme-challenge/<token>
# Check firewall allows port 80 from internet
# Let's Encrypt must reach your server from external
# Verify ingress class matches
kubectl get ingress -o yaml | grep ingressClassName
Issue: DNS01 Challenge Failing
Symptoms: Challenge times out waiting for DNS propagation.
Solutions:
# Verify TXT record exists
dig TXT _acme-challenge.example.com @8.8.8.8
# Check DNS provider credentials
kubectl get secret <dns-credentials> -n cert-manager -o yaml
# Verify IAM permissions for cloud DNS
# Route53: route53:GetChange, route53:ChangeResourceRecordSets
# CloudFlare: Zone.Zone, Zone.DNS
# Cloud DNS: dns.changes.*, dns.managedZones.*, dns.resourceRecordSets.*
# Check cert-manager logs for DNS errors
kubectl logs -n cert-manager -l app=cert-manager | grep -i dns
# Increase propagation timeout if needed
# In solver config:
dns01:
cloudflare:
apiTokenSecretRef:
name: cloudflare-token
key: api-token
cnameStrategy: Follow
Issue: Certificate Not Renewing
Symptoms: Certificate expiry approaching but no renewal attempt.
Solutions:
# Check renewal time
kubectl get certificate <name> -o jsonpath='{.status.renewalTime}'
# Verify renewBefore is set correctly
kubectl get certificate <name> -o jsonpath='{.spec.renewBefore}'
# Check certificate is not in failed state
kubectl describe certificate <name>
# Force renewal
cmctl renew <name>
# Or
kubectl delete secret <tls-secret>
# Check cert-manager controller is running
kubectl get pods -n cert-manager
kubectl logs -n cert-manager -l app.kubernetes.io/component=controller
Issue: "Issuer Not Found"
Symptoms: Certificate shows IssuerNotFound error.
Solutions:
# Check issuer exists
kubectl get issuers -A
kubectl get clusterissuers
# Verify issuerRef in certificate
kubectl get certificate <name> -o yaml | grep -A5 issuerRef
# For Issuer (not ClusterIssuer), must be in same namespace
# Check namespaces match
kubectl get certificate <name> -o jsonpath='{.metadata.namespace}'
# Common mistake: using Issuer when ClusterIssuer exists
# Ensure kind is correct:
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer # Not Issuer
Issue: Secret Not Created
Symptoms: Certificate is ready but Secret doesn't exist.
Solutions:
# Verify secretName in certificate
kubectl get certificate <name> -o jsonpath='{.spec.secretName}'
# Check for RBAC issues
kubectl auth can-i create secrets --as=system:serviceaccount:cert-manager:cert-manager
# Check cert-manager logs
kubectl logs -n cert-manager -l app=cert-manager | grep -i secret
# Verify secret namespace
# Secret is created in same namespace as Certificate
Issue: Rate Limit Exceeded
Symptoms: Order fails with "too many certificates" error.
Solutions:
# Check Let's Encrypt rate limits:
# - 50 certs per registered domain per week
# - 5 duplicate certs per week
# - 300 new orders per 3 hours
# Use staging for testing
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
spec:
acme:
server: https://acme-staging-v02.api.letsencrypt.org/directory
# Reuse certificates across services instead of creating duplicates
# Use wildcard certificates where possible
# Wait for rate limit window to reset (usually 1 week)
Issue: Private Key Rotation Not Working
Symptoms: Same private key after renewal.
Solutions:
# Ensure rotationPolicy is set
apiVersion: cert-manager.io/v1
kind: Certificate
spec:
privateKey:
rotationPolicy: Always # Must be explicitly set
Issue: Webhook Connection Refused
Symptoms: Cannot create Certificate resources, webhook errors.
Solutions:
# Check webhook pod
kubectl get pods -n cert-manager -l app=webhook
# Check webhook service
kubectl get svc -n cert-manager cert-manager-webhook
# Verify webhook certificate
kubectl get secret -n cert-manager cert-manager-webhook-ca
# Restart cert-manager components
kubectl rollout restart deployment -n cert-manager cert-manager
kubectl rollout restart deployment -n cert-manager cert-manager-webhook
kubectl rollout restart deployment -n cert-manager cert-manager-cainjector
# Check network policies
kubectl get networkpolicies -n cert-manager
Debugging Commands
# Full diagnostic
cmctl check api
# View all cert-manager resources
kubectl get certificates,certificaterequests,orders,challenges -A
# Export certificate details
kubectl get certificate <name> -o yaml > cert-debug.yaml
# Check certificate chain
kubectl get secret <tls-secret> -o jsonpath='{.data.tls\.crt}' | \
base64 -d | openssl crl2pkcs7 -nocrl -certfile /dev/stdin | \
openssl pkcs7 -print_certs -noout
# Verify certificate matches key
kubectl get secret <tls-secret> -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -modulus | md5sum
kubectl get secret <tls-secret> -o jsonpath='{.data.tls\.key}' | base64 -d | openssl rsa -noout -modulus | md5sum
# MD5 sums should match
Related Topics
The following topics complement this Cert-Manager cheatsheet:
-
Kubernetes Ingress Controllers - Nginx, Traefik, and other ingress controllers that use TLS certificates from cert-manager
-
HashiCorp Vault - Enterprise PKI backend integration with cert-manager for internal certificate management
-
Istio/Service Mesh - mTLS and certificate management for service mesh communication
-
Kubernetes Security - RBAC, Pod Security Standards, and network policies for securing cert-manager deployments
-
Let's Encrypt/ACME - Deep dive into the ACME protocol, rate limits, and certificate transparency logs
-
DNS Management - Route53, CloudFlare, and other DNS providers used for DNS01 challenges