Kubernetes RBAC and Multi-Tenancy

Comprehensive guide to Role-Based Access Control (RBAC), service accounts, namespace isolation, and multi-tenancy patterns in Kubernetes.

Overview

Kubernetes RBAC provides fine-grained access control to cluster resources through roles, bindings, and service accounts. Multi-tenancy enables multiple teams or applications to share cluster resources securely through namespace isolation, resource quotas, and network policies. RBAC is enabled by default in Kubernetes 1.6+ and follows the principle of least privilege.

Key Components:

• Roles/ClusterRoles: Define permissions (what can be done)
• RoleBindings/ClusterRoleBindings: Grant permissions (who can do it)
• Service Accounts: Provide identity for pods
• Subjects: Users, groups, or service accounts receiving permissions

⤢ zoom

Roles and ClusterRoles

Roles define permissions within a namespace, whilst ClusterRoles define cluster-wide permissions or permissions for cluster-scoped resources.

Key Concepts

• Role: Namespace-scoped permissions
• ClusterRole: Cluster-wide permissions or namespace-aggregatable permissions
• Verbs: Actions (get, list, watch, create, update, patch, delete, deletecollection)
• API Groups: Resource grouping (core="", apps, batch, rbac.authorization.k8s.io)
• Resources: Kubernetes objects (pods, services, deployments, etc.)
• Resource Names: Restrict permissions to specific resource instances

⤢ zoom

Creating Roles

# role-developer.yaml - Basic developer permissions
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: development
  name: developer
rules:
# Pod access
- apiGroups: [""]
  resources: ["pods", "pods/log", "pods/status"]
  verbs: ["get", "list", "watch"]
# Deployment management
- apiGroups: ["apps"]
  resources: ["deployments", "replicasets", "statefulsets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
# Service access
- apiGroups: [""]
  resources: ["services"]
  verbs: ["get", "list", "watch", "create", "update", "patch"]
# ConfigMap and Secret read
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list"]  # Limited access to secrets

# role-pod-reader.yaml - Read-only pod access
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: production
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get", "list"]

# role-secret-admin.yaml - Specific resource names
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: production
  name: secret-admin
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "update", "patch"]
  resourceNames: ["app-credentials", "db-password"]  # Only these secrets

# Create role imperatively
kubectl create role developer \
  --verb=get,list,watch,create,update,delete \
  --resource=pods,deployments,services \
  -n development

# View roles
kubectl get roles -n development
kubectl describe role developer -n development

# View role in YAML
kubectl get role developer -n development -o yaml

Creating ClusterRoles

# clusterrole-node-reader.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: node-reader
rules:
- apiGroups: [""]
  resources: ["nodes"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["nodes/status"]
  verbs: ["get"]

# clusterrole-namespace-admin.yaml - For multi-namespace access
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: namespace-admin
rules:
# Full access to most namespace resources
- apiGroups: ["", "apps", "batch", "extensions"]
  resources: ["*"]
  verbs: ["*"]
# Deny access to secrets (use specific role for secrets)
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list"]  # Read-only for secrets

# clusterrole-aggregation.yaml - Aggregated ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: monitoring
  labels:
    rbac.example.com/aggregate-to-monitoring: "true"
aggregationRule:
  clusterRoleSelectors:
  - matchLabels:
      rbac.example.com/aggregate-to-monitoring: "true"
rules: []  # Automatically filled by aggregation
---
# Component roles that get aggregated
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: monitoring-pods
  labels:
    rbac.example.com/aggregate-to-monitoring: "true"
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: monitoring-metrics
  labels:
    rbac.example.com/aggregate-to-monitoring: "true"
rules:
- apiGroups: ["metrics.k8s.io"]
  resources: ["pods", "nodes"]
  verbs: ["get", "list"]

# Create ClusterRole imperatively
kubectl create clusterrole node-reader \
  --verb=get,list,watch \
  --resource=nodes

# View ClusterRoles
kubectl get clusterroles
kubectl describe clusterrole node-reader

# View built-in ClusterRoles
kubectl get clusterroles | grep -E "^(view|edit|admin|cluster-admin)"

Built-in ClusterRoles

# View default user-facing roles
kubectl describe clusterrole view      # Read-only access
kubectl describe clusterrole edit      # Read-write, no RBAC changes
kubectl describe clusterrole admin     # Full namespace control
kubectl describe clusterrole cluster-admin  # Cluster-wide superuser

RoleBindings and ClusterRoleBindings

Bindings grant the permissions defined in a Role or ClusterRole to subjects (users, groups, or service accounts).

Key Concepts

• RoleBinding: Grants Role or ClusterRole permissions within a namespace
• ClusterRoleBinding: Grants ClusterRole permissions cluster-wide
• Subjects: Users (User), groups (Group), or service accounts (ServiceAccount)
• Scope: RoleBinding is namespace-scoped, ClusterRoleBinding is cluster-wide

⤢ zoom

Creating RoleBindings

# rolebinding-developer.yaml - Bind user to role
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: developer-binding
  namespace: development
subjects:
# User
- kind: User
  name: jane@example.com
  apiGroup: rbac.authorization.k8s.io
# Group
- kind: Group
  name: developers
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: developer
  apiGroup: rbac.authorization.k8s.io

# rolebinding-sa.yaml - Bind service account to role
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: app-pod-reader
  namespace: production
subjects:
- kind: ServiceAccount
  name: app-sa
  namespace: production
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

# rolebinding-clusterrole.yaml - Bind ClusterRole in namespace scope
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: admin-access
  namespace: production
subjects:
- kind: User
  name: admin@example.com
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: admin  # Built-in ClusterRole
  apiGroup: rbac.authorization.k8s.io

# Create RoleBinding imperatively
kubectl create rolebinding developer-binding \
  --role=developer \
  --user=jane@example.com \
  -n development

# Bind service account to role
kubectl create rolebinding app-binding \
  --role=pod-reader \
  --serviceaccount=production:app-sa \
  -n production

# View RoleBindings
kubectl get rolebindings -n development
kubectl describe rolebinding developer-binding -n development

Creating ClusterRoleBindings

# clusterrolebinding-cluster-admin.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: cluster-admin-binding
subjects:
- kind: User
  name: admin@example.com
  apiGroup: rbac.authorization.k8s.io
- kind: Group
  name: cluster-admins
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

# clusterrolebinding-readonly.yaml - Read-only cluster access
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: readonly-binding
subjects:
- kind: Group
  name: viewers
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: view
  apiGroup: rbac.authorization.k8s.io

# Create ClusterRoleBinding imperatively
kubectl create clusterrolebinding cluster-admin-binding \
  --clusterrole=cluster-admin \
  --user=admin@example.com

# Bind group to ClusterRole
kubectl create clusterrolebinding readonly-binding \
  --clusterrole=view \
  --group=viewers

# View ClusterRoleBindings
kubectl get clusterrolebindings
kubectl describe clusterrolebinding cluster-admin-binding

Testing Permissions

# Check if user can perform action
kubectl auth can-i create deployments --namespace=development --as=jane@example.com

# Check all permissions for user
kubectl auth can-i --list --as=jane@example.com -n development

# Check service account permissions
kubectl auth can-i get pods --as=system:serviceaccount:production:app-sa -n production

# Check group permissions
kubectl auth can-i create pods --as=test-user --as-group=developers -n development

Service Accounts

Service accounts provide identity for pods and applications running in the cluster.

Key Concepts

• Service Account: Identity for pods (namespace-scoped)
• Token: JWT automatically mounted in pods at /var/run/secrets/kubernetes.io/serviceaccount/token
• Default SA: Every namespace has a default service account
• Token Lifespan: Tokens can be time-bound or long-lived (legacy)
• Pod Identity: Pods use service account tokens to authenticate to API server

⤢ zoom

Creating Service Accounts

# serviceaccount.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-sa
  namespace: production
automountServiceAccountToken: true  # Default: true
---
# With image pull secrets
apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-sa-with-secrets
  namespace: production
imagePullSecrets:
- name: registry-credentials
secrets:
- name: additional-token

# Create service account
kubectl create serviceaccount app-sa -n production

# View service accounts
kubectl get serviceaccounts -n production
kubectl describe serviceaccount app-sa -n production

# Get service account token (Kubernetes 1.24+)
kubectl create token app-sa -n production

# Create long-lived token (for external access)
kubectl create token app-sa -n production --duration=8760h  # 1 year

Using Service Accounts in Pods

# pod-with-sa.yaml
apiVersion: v1
kind: Pod
metadata:
  name: app-pod
  namespace: production
spec:
  serviceAccountName: app-sa  # Use specific service account
  containers:
  - name: app
    image: myapp:v1
    env:
    # Access token from mounted volume
    - name: TOKEN_PATH
      value: /var/run/secrets/kubernetes.io/serviceaccount/token

# deployment-with-sa.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-deployment
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      serviceAccountName: app-sa
      automountServiceAccountToken: true
      containers:
      - name: app
        image: myapp:v1

# pod-no-sa-token.yaml - Disable token mounting
apiVersion: v1
kind: Pod
metadata:
  name: no-token-pod
  namespace: production
spec:
  serviceAccountName: app-sa
  automountServiceAccountToken: false  # Don't mount token
  containers:
  - name: app
    image: myapp:v1

Service Account Tokens (Kubernetes 1.24+)

# Create bound token (recommended for 1.24+)
kubectl create token app-sa -n production

# Create token with audience
kubectl create token app-sa -n production --audience=https://vault.example.com

# Create token bound to pod
kubectl create token app-sa -n production \
  --bound-object-kind=Pod \
  --bound-object-name=app-pod

# Legacy: Create service account token secret (deprecated)
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: app-sa-token
  namespace: production
  annotations:
    kubernetes.io/service-account.name: app-sa
type: kubernetes.io/service-account-token
EOF

Using Service Account from Client

# Get token
TOKEN=$(kubectl create token app-sa -n production)

# Use token with kubectl
kubectl --token=$TOKEN get pods -n production

# Use token with curl
API_SERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
curl -k -H "Authorization: Bearer $TOKEN" $API_SERVER/api/v1/namespaces/production/pods

Impersonation

Impersonation allows testing permissions and acting on behalf of other users or service accounts.

Key Concepts

• User Impersonation: Act as another user
• Group Impersonation: Act as member of a group
• ServiceAccount Impersonation: Act as a service account
• Permission Required: impersonate verb on users, groups, or serviceaccounts

⤢ zoom

Granting Impersonation Permissions

# clusterrole-impersonator.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: impersonator
rules:
# Impersonate any user
- apiGroups: [""]
  resources: ["users"]
  verbs: ["impersonate"]
# Impersonate any group
- apiGroups: [""]
  resources: ["groups"]
  verbs: ["impersonate"]
# Impersonate any service account
- apiGroups: [""]
  resources: ["serviceaccounts"]
  verbs: ["impersonate"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: impersonator-binding
subjects:
- kind: User
  name: admin@example.com
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: impersonator
  apiGroup: rbac.authorization.k8s.io

# clusterrole-limited-impersonator.yaml - Impersonate specific users/groups
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: developer-impersonator
rules:
# Impersonate specific users
- apiGroups: [""]
  resources: ["users"]
  verbs: ["impersonate"]
  resourceNames: ["dev-user-1", "dev-user-2"]
# Impersonate specific groups
- apiGroups: [""]
  resources: ["groups"]
  verbs: ["impersonate"]
  resourceNames: ["developers", "qa-team"]
# Impersonate service accounts in specific namespace
- apiGroups: [""]
  resources: ["serviceaccounts"]
  verbs: ["impersonate"]
  resourceNames: ["app-sa", "test-sa"]

Using Impersonation

# Impersonate user
kubectl get pods --as=jane@example.com

# Impersonate user in group
kubectl get pods --as=jane@example.com --as-group=developers

# Impersonate service account
kubectl get pods --as=system:serviceaccount:production:app-sa -n production

# Impersonate with multiple groups
kubectl get pods --as=john@example.com --as-group=developers --as-group=admins

# Check what impersonated user can do
kubectl auth can-i create deployments --as=jane@example.com -n development

# Test specific action as service account
kubectl auth can-i get secrets \
  --as=system:serviceaccount:production:app-sa \
  -n production

Impersonation Use Cases

# Test RBAC policies before applying
kubectl auth can-i --list --as=new-user@example.com -n development

# Debug permission issues
kubectl get pods -n production --as=system:serviceaccount:production:app-sa

# Execute commands as different user for auditing
kubectl create deployment test --image=nginx \
  --as=deployer@example.com \
  -n staging

# Validate multi-tenancy isolation
kubectl get secrets --all-namespaces \
  --as=tenant-a-user@example.com

Namespace Isolation Strategies

Implement multi-tenancy through namespace isolation with RBAC, network policies, and resource quotas.

Key Concepts

• Namespace: Logical cluster partition for multi-tenancy
• Resource Quotas: Limit resource consumption per namespace
• Limit Ranges: Set default resource requests/limits
• Network Policies: Control traffic between namespaces
• Pod Security Standards: Enforce security policies per namespace

⤢ zoom

Namespace Creation with Labels

# namespace-team-a.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: team-a
  labels:
    team: team-a
    environment: production
    compliance: pci
  annotations:
    contact: "team-a@example.com"
    description: "Team A production workloads"
---
apiVersion: v1
kind: Namespace
metadata:
  name: team-b
  labels:
    team: team-b
    environment: development
    compliance: general

# Create namespace
kubectl create namespace team-a

# Label namespace
kubectl label namespace team-a environment=production team=team-a

# Annotate namespace
kubectl annotate namespace team-a contact="team-a@example.com"

Resource Quotas

# resourcequota-team-a.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-quota
  namespace: team-a
spec:
  hard:
    # Compute resources
    requests.cpu: "10"
    requests.memory: 20Gi
    limits.cpu: "20"
    limits.memory: 40Gi
    # Object counts
    pods: "50"
    services: "20"
    configmaps: "30"
    secrets: "30"
    persistentvolumeclaims: "10"
    # Storage
    requests.storage: 100Gi
---
# resourcequota-object-counts.yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  name: object-quota
  namespace: team-a
spec:
  hard:
    count/deployments.apps: "15"
    count/statefulsets.apps: "5"
    count/jobs.batch: "10"
    count/cronjobs.batch: "5"
    count/services.loadbalancers: "3"

# Create resource quota
kubectl create quota compute-quota \
  --hard=cpu=10,memory=20Gi,pods=50 \
  -n team-a

# View quota
kubectl get resourcequota -n team-a
kubectl describe resourcequota compute-quota -n team-a

# Check quota usage
kubectl get resourcequota -n team-a -o yaml

Limit Ranges

# limitrange-team-a.yaml
apiVersion: v1
kind: LimitRange
metadata:
  name: resource-limits
  namespace: team-a
spec:
  limits:
  # Container limits
  - type: Container
    max:
      cpu: "2"
      memory: 4Gi
    min:
      cpu: 100m
      memory: 128Mi
    default:
      cpu: 500m
      memory: 512Mi
    defaultRequest:
      cpu: 200m
      memory: 256Mi
  # Pod limits
  - type: Pod
    max:
      cpu: "4"
      memory: 8Gi
  # PVC limits
  - type: PersistentVolumeClaim
    max:
      storage: 50Gi
    min:
      storage: 1Gi

# Apply limit range
kubectl apply -f limitrange-team-a.yaml

# View limit ranges
kubectl get limitrange -n team-a
kubectl describe limitrange resource-limits -n team-a

Network Isolation

# networkpolicy-deny-all.yaml - Default deny
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: team-a
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress
---
# networkpolicy-allow-namespace.yaml - Allow within namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-same-namespace
  namespace: team-a
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector: {}  # All pods in same namespace
---
# networkpolicy-allow-ingress.yaml - Allow from ingress controller
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-ingress
  namespace: team-a
spec:
  podSelector:
    matchLabels:
      expose: "true"
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: ingress-nginx
    ports:
    - protocol: TCP
      port: 8080
---
# networkpolicy-egress.yaml - Allow DNS and external
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-egress
  namespace: team-a
spec:
  podSelector: {}
  policyTypes:
  - Egress
  egress:
  # Allow DNS
  - to:
    - namespaceSelector:
        matchLabels:
          name: kube-system
    ports:
    - protocol: UDP
      port: 53
  # Allow external HTTPS
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
        except:
        - 10.0.0.0/8
        - 172.16.0.0/12
        - 192.168.0.0/16
    ports:
    - protocol: TCP
      port: 443

Pod Security Standards

# namespace-with-pss.yaml - Enforce restricted security
apiVersion: v1
kind: Namespace
metadata:
  name: team-a
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: latest
    pod-security.kubernetes.io/audit: restricted
    pod-security.kubernetes.io/warn: restricted

Multi-Tenant RBAC Pattern

# Complete multi-tenant setup for Team A
---
# Namespace
apiVersion: v1
kind: Namespace
metadata:
  name: team-a
  labels:
    team: team-a
    pod-security.kubernetes.io/enforce: baseline
---
# Service Account
apiVersion: v1
kind: ServiceAccount
metadata:
  name: team-a-sa
  namespace: team-a
---
# Role - Full namespace access except RBAC
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: team-a-developer
  namespace: team-a
rules:
- apiGroups: ["", "apps", "batch", "extensions"]
  resources: ["*"]
  verbs: ["*"]
# Deny RBAC changes
- apiGroups: ["rbac.authorization.k8s.io"]
  resources: ["roles", "rolebindings"]
  verbs: ["get", "list"]
---
# RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: team-a-binding
  namespace: team-a
subjects:
- kind: Group
  name: team-a-developers
  apiGroup: rbac.authorization.k8s.io
- kind: ServiceAccount
  name: team-a-sa
  namespace: team-a
roleRef:
  kind: Role
  name: team-a-developer
  apiGroup: rbac.authorization.k8s.io
---
# Resource Quota
apiVersion: v1
kind: ResourceQuota
metadata:
  name: team-a-quota
  namespace: team-a
spec:
  hard:
    requests.cpu: "10"
    requests.memory: 20Gi
    limits.cpu: "20"
    limits.memory: 40Gi
    pods: "50"
---
# Limit Range
apiVersion: v1
kind: LimitRange
metadata:
  name: team-a-limits
  namespace: team-a
spec:
  limits:
  - type: Container
    default:
      cpu: 500m
      memory: 512Mi
    defaultRequest:
      cpu: 200m
      memory: 256Mi
---
# Network Policy - Deny cross-namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-cross-namespace
  namespace: team-a
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector: {}

Least-Privilege Policy Design

Design RBAC policies following the principle of least privilege for security and compliance.

Key Principles

• Minimal Permissions: Grant only what's needed
• Resource-Specific: Limit to specific resources when possible
• Verb Restrictions: Use precise verbs, avoid wildcards
• Namespace Isolation: Prefer Roles over ClusterRoles
• Regular Audits: Review and remove unused permissions

⤢ zoom

Read-Only Access Pattern

# role-readonly.yaml - Minimal read access
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: app-readonly
  namespace: production
rules:
# Read pods and logs only
- apiGroups: [""]
  resources: ["pods", "pods/log", "pods/status"]
  verbs: ["get", "list", "watch"]
# Read services
- apiGroups: [""]
  resources: ["services"]
  verbs: ["get", "list"]
# Read configmaps (no secrets!)
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "list"]

CI/CD Service Account Pattern

# role-cicd.yaml - CI/CD deployment permissions
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: cicd-deployer
  namespace: production
rules:
# Manage deployments
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "watch", "create", "update", "patch"]
# Read and create services
- apiGroups: [""]
  resources: ["services"]
  verbs: ["get", "list", "create", "update"]
# Manage configmaps
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "list", "create", "update", "patch", "delete"]
# Read secrets (for verification, not creation)
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list"]
# View rollout status
- apiGroups: [""]
  resources: ["pods", "replicasets"]
  verbs: ["get", "list", "watch"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: cicd-sa
  namespace: production
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: cicd-deployer-binding
  namespace: production
subjects:
- kind: ServiceAccount
  name: cicd-sa
  namespace: production
roleRef:
  kind: Role
  name: cicd-deployer
  apiGroup: rbac.authorization.k8s.io

Monitoring Service Account Pattern

# clusterrole-monitoring.yaml - Read-only monitoring access
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: monitoring-reader
rules:
# Read nodes and metrics
- apiGroups: [""]
  resources: ["nodes", "nodes/stats", "nodes/metrics"]
  verbs: ["get", "list"]
# Read pods across all namespaces
- apiGroups: [""]
  resources: ["pods", "pods/log"]
  verbs: ["get", "list", "watch"]
# Read services and endpoints
- apiGroups: [""]
  resources: ["services", "endpoints"]
  verbs: ["get", "list", "watch"]
# Read metrics
- apiGroups: ["metrics.k8s.io"]
  resources: ["pods", "nodes"]
  verbs: ["get", "list"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: prometheus
  namespace: monitoring
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: monitoring-reader-binding
subjects:
- kind: ServiceAccount
  name: prometheus
  namespace: monitoring
roleRef:
  kind: ClusterRole
  name: monitoring-reader
  apiGroup: rbac.authorization.k8s.io

Secrets Management Pattern

# role-secrets-readonly.yaml - Read-only secret access
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: secrets-reader
  namespace: production
rules:
# Read specific secrets only
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]
  resourceNames:
  - app-config
  - db-credentials
---
# role-secrets-admin.yaml - Full secret access (highly restricted)
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: secrets-admin
  namespace: production
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
# Should only be granted to vault/sealed-secrets operators

Operator Service Account Pattern

# clusterrole-operator.yaml - Custom operator permissions
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: database-operator
rules:
# Manage CRDs
- apiGroups: ["apiextensions.k8s.io"]
  resources: ["customresourcedefinitions"]
  verbs: ["get", "list", "watch"]
# Manage custom resources
- apiGroups: ["databases.example.com"]
  resources: ["databases", "databases/status"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
# Manage statefulsets for databases
- apiGroups: ["apps"]
  resources: ["statefulsets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
# Manage services and PVCs
- apiGroups: [""]
  resources: ["services", "persistentvolumeclaims"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
# Read secrets for database credentials
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list", "watch"]

Anti-Patterns to Avoid

# BAD: Wildcard everything (never do this except for cluster-admin)
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: bad-role
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]

# BAD: Excessive secret access
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: bad-secrets
  namespace: production
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["*"]  # Too permissive

# BAD: ClusterRole when Role would suffice
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: bad-clusterrole
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list"]
# Should be a Role in specific namespace instead

# GOOD: Specific permissions
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: good-role
  namespace: production
rules:
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "watch", "update", "patch"]
  resourceNames: ["web-app", "api-app"]  # Specific resources

Auditing and Access Reviews

Monitor and audit RBAC permissions, access patterns, and security events.

Key Concepts

• Audit Logs: Record all API requests and authorisation decisions
• Audit Policy: Define what events to log
• Access Reviews: Regular review of who has access to what
• RBAC Discovery: Tools to analyze and visualize permissions
• Compliance: Track and report on access controls

⤢ zoom

Enable Audit Logging

# audit-policy.yaml - Comprehensive audit policy
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
# Log authentication failures
- level: Metadata
  omitStages:
  - RequestReceived
  namespaces: ["*"]
  verbs: ["*"]
  resources:
  - group: ""
    resources: ["secrets", "configmaps"]

# Log all secret access
- level: RequestResponse
  omitStages:
  - RequestReceived
  resources:
  - group: ""
    resources: ["secrets"]

# Log RBAC changes
- level: RequestResponse
  omitStages:
  - RequestReceived
  resources:
  - group: "rbac.authorization.k8s.io"
    resources: ["roles", "rolebindings", "clusterroles", "clusterrolebindings"]

# Log pod exec/attach
- level: Request
  omitStages:
  - RequestReceived
  resources:
  - group: ""
    resources: ["pods/exec", "pods/attach", "pods/portforward"]

# Log namespace operations
- level: RequestResponse
  omitStages:
  - RequestReceived
  resources:
  - group: ""
    resources: ["namespaces"]

# Log metadata for everything else
- level: Metadata
  omitStages:
  - RequestReceived

# Configure API server with audit policy (add to kube-apiserver manifest)
# /etc/kubernetes/manifests/kube-apiserver.yaml
--audit-policy-file=/etc/kubernetes/audit-policy.yaml
--audit-log-path=/var/log/kubernetes/audit.log
--audit-log-maxage=30
--audit-log-maxbackup=10
--audit-log-maxsize=100

# View audit logs
sudo tail -f /var/log/kubernetes/audit.log

# Query audit logs with jq
sudo cat /var/log/kubernetes/audit.log | \
  jq 'select(.verb=="delete" and .objectRef.resource=="secrets")'

# Find failed authorization attempts
sudo cat /var/log/kubernetes/audit.log | \
  jq 'select(.responseStatus.code==403)'

Audit Log Analysis

# Find all requests by user
jq 'select(.user.username=="jane@example.com")' audit.log

# Find all secret access
jq 'select(.objectRef.resource=="secrets")' audit.log

# Find all failed authorisation
jq 'select(.responseStatus.code==403)' audit.log | jq -s 'group_by(.user.username) | map({user: .[0].user.username, count: length})'

# Find exec/attach commands
jq 'select(.objectRef.subresource=="exec" or .objectRef.subresource=="attach")' audit.log

# Find RBAC changes
jq 'select(.objectRef.apiGroup=="rbac.authorization.k8s.io")' audit.log

# Summarise API usage by user
jq -s 'group_by(.user.username) | map({user: .[0].user.username, requests: length}) | sort_by(.requests) | reverse' audit.log

RBAC Discovery and Analysis

# List all ClusterRoleBindings and subjects
kubectl get clusterrolebindings -o json | \
  jq -r '.items[] | {name: .metadata.name, subjects: .subjects, role: .roleRef.name}'

# Find all bindings for a user
kubectl get rolebindings,clusterrolebindings --all-namespaces -o json | \
  jq '.items[] | select(.subjects[]? | .name=="jane@example.com")'

# List all service accounts with cluster-admin
kubectl get clusterrolebindings -o json | \
  jq -r '.items[] | select(.roleRef.name=="cluster-admin") | .subjects[]? | select(.kind=="ServiceAccount") | .name'

# Find unused service accounts (no pods using them)
comm -23 \
  <(kubectl get sa --all-namespaces -o json | jq -r '.items[] | "\(.metadata.namespace)/\(.metadata.name)"' | sort) \
  <(kubectl get pods --all-namespaces -o json | jq -r '.items[] | "\(.metadata.namespace)/\(.spec.serviceAccountName)"' | sort | uniq)

# List all permissions for a service account
kubectl auth can-i --list \
  --as=system:serviceaccount:production:app-sa \
  -n production

RBAC Analysis Tools

# Install kubectl-who-can plugin
kubectl krew install who-can

# Find who can perform action
kubectl who-can create pods -n production
kubectl who-can delete secrets --all-namespaces
kubectl who-can '*' '*'  # Find cluster admins

# Install rbac-lookup
kubectl krew install rbac-lookup

# Look up user permissions
kubectl rbac-lookup jane@example.com
kubectl rbac-lookup --kind serviceaccount --namespace production

# Install rback (RBAC visualizer)
kubectl krew install rbac-view
kubectl rbac-view

# Use kubectl-access-matrix
kubectl krew install access-matrix
kubectl access-matrix --as=jane@example.com -n production

Regular Access Review Checklist

#!/bin/bash
# rbac-review.sh - Regular RBAC audit script

echo "=== RBAC Access Review ==="
echo

echo "1. ClusterAdmin bindings:"
kubectl get clusterrolebindings -o json | \
  jq -r '.items[] | select(.roleRef.name=="cluster-admin") | "  - \(.metadata.name): \(.subjects)"'
echo

echo "2. Service accounts with cluster roles:"
kubectl get clusterrolebindings -o json | \
  jq -r '.items[] | select(.subjects[]?.kind=="ServiceAccount") | "  - \(.metadata.name): \(.roleRef.name) -> \(.subjects)"'
echo

echo "3. Subjects with wildcard permissions:"
kubectl get roles,clusterroles --all-namespaces -o json | \
  jq -r '.items[] | select(.rules[]? | .verbs[]?=="*" or .resources[]?=="*") | "  - \(.metadata.namespace // "cluster")/\(.metadata.name)"'
echo

echo "4. Recent failed authorization (last hour):"
sudo journalctl -u kubelet --since "1 hour ago" | grep -i "forbidden" | tail -10
echo

echo "5. Namespaces without resource quotas:"
comm -23 \
  <(kubectl get namespaces -o json | jq -r '.items[].metadata.name' | sort) \
  <(kubectl get resourcequotas --all-namespaces -o json | jq -r '.items[].metadata.namespace' | sort | uniq)
echo

echo "=== Review Complete ==="

Compliance Reporting

# Open Policy Agent policy for compliance checks
# rbac-compliance.rego
package kubernetes.rbac.compliance

import future.keywords.contains
import future.keywords.if

# Deny cluster-admin to non-admin users
deny[msg] {
    binding := input.clusterrolebindings[_]
    binding.roleRef.name == "cluster-admin"
    subject := binding.subjects[_]
    not contains(subject.name, "admin")
    msg := sprintf("Non-admin user %s has cluster-admin", [subject.name])
}

# Warn about service accounts with cluster-wide permissions
warn[msg] {
    binding := input.clusterrolebindings[_]
    subject := binding.subjects[_]
    subject.kind == "ServiceAccount"
    msg := sprintf("ServiceAccount %s/%s has cluster-wide permissions via %s",
                   [subject.namespace, subject.name, binding.metadata.name])
}

# Require all namespaces to have resource quotas
deny[msg] {
    namespace := input.namespaces[_]
    not namespace_has_quota(namespace.metadata.name)
    namespace.metadata.name != "kube-system"
    namespace.metadata.name != "kube-public"
    msg := sprintf("Namespace %s missing ResourceQuota", [namespace.metadata.name])
}

namespace_has_quota(ns) if {
    quota := input.resourcequotas[_]
    quota.metadata.namespace == ns
}

Continuous Monitoring

# Watch for RBAC changes
kubectl get roles,rolebindings,clusterroles,clusterrolebindings \
  --all-namespaces \
  --watch

# Monitor authentication failures
kubectl logs -n kube-system -l component=kube-apiserver \
  --tail=100 -f | grep -i "forbidden"

# Track service account token usage
kubectl get events --all-namespaces \
  --field-selector reason=FailedMount,reason=FailedCreate \
  --watch

# Alert on cluster-admin changes (with Prometheus)
# Create ServiceMonitor for kube-apiserver audit events
# Alert when cluster-admin binding modified

Quick Reference

Common kubectl Commands

# Roles and ClusterRoles
kubectl get roles -n <namespace>
kubectl get clusterroles
kubectl describe role <role-name> -n <namespace>
kubectl create role <name> --verb=<verbs> --resource=<resources> -n <namespace>
kubectl create clusterrole <name> --verb=<verbs> --resource=<resources>

# RoleBindings and ClusterRoleBindings
kubectl get rolebindings -n <namespace>
kubectl get clusterrolebindings
kubectl create rolebinding <name> --role=<role> --user=<user> -n <namespace>
kubectl create clusterrolebinding <name> --clusterrole=<role> --user=<user>

# Service Accounts
kubectl get serviceaccounts -n <namespace>
kubectl create serviceaccount <name> -n <namespace>
kubectl create token <sa-name> -n <namespace>
kubectl describe serviceaccount <name> -n <namespace>

# Permission Testing
kubectl auth can-i <verb> <resource> -n <namespace>
kubectl auth can-i --list
kubectl auth can-i <verb> <resource> --as=<user> -n <namespace>
kubectl auth can-i create pods --as=system:serviceaccount:ns:sa -n ns

# Impersonation
kubectl get pods --as=<user>
kubectl get pods --as=<user> --as-group=<group>
kubectl get pods --as=system:serviceaccount:<namespace>:<sa-name>

# Resource Quotas and Limits
kubectl get resourcequota -n <namespace>
kubectl describe resourcequota <name> -n <namespace>
kubectl get limitrange -n <namespace>
kubectl create quota <name> --hard=cpu=10,memory=20Gi -n <namespace>

# Audit and Analysis
kubectl get clusterrolebindings -o wide
kubectl get rolebindings --all-namespaces
kubectl api-resources --verbs=list --namespaced -o name

RBAC Verbs

Verb	Description
get	Read a single resource
list	List all resources of a type
watch	Watch resources for changes
create	Create new resources
update	Update entire resource (PUT)
patch	Partially update resource (PATCH)
delete	Delete a resource
deletecollection	Delete multiple resources
*	All verbs (avoid except for admin)

Common Resource Types

Resource	API Group	Namespaced
pods	"" (core)	Yes
services	""	Yes
secrets	""	Yes
configmaps	""	Yes
deployments	apps	Yes
statefulsets	apps	Yes
jobs	batch	Yes
cronjobs	batch	Yes
roles	rbac.authorization.k8s.io	Yes
rolebindings	rbac.authorization.k8s.io	Yes
nodes	""	No
namespaces	""	No
clusterroles	rbac.authorization.k8s.io	No
clusterrolebindings	rbac.authorization.k8s.io	No
persistentvolumes	""	No

Built-in ClusterRoles

ClusterRole	Description
cluster-admin	Full cluster access (superuser)
admin	Full namespace access, can manage RBAC
edit	Read-write namespace access, cannot manage RBAC
view	Read-only namespace access
system:node	Kubelet permissions for nodes
system:discovery	Read-only discovery endpoints

Subject Types

# User
- kind: User
  name: jane@example.com
  apiGroup: rbac.authorization.k8s.io

# Group
- kind: Group
  name: developers
  apiGroup: rbac.authorization.k8s.io

# Service Account
- kind: ServiceAccount
  name: app-sa
  namespace: production

# All authenticated users
- kind: Group
  name: system:authenticated
  apiGroup: rbac.authorization.k8s.io

# All users (including unauthenticated)
- kind: Group
  name: system:unauthenticated
  apiGroup: rbac.authorization.k8s.io

Common Issues and Solutions

Permission Denied Errors

Issue	Solution
Error from server (Forbidden): pods is forbidden	User/SA lacks permissions - check RoleBindings with kubectl auth can-i --list --as=<user>
Service account can't access API	Ensure RoleBinding exists linking SA to Role, verify token is mounted in pod
Error: serviceaccounts "default" not found	Default SA should exist automatically - check namespace exists and is not corrupted
Cross-namespace access denied	Use ClusterRole with RoleBinding in each namespace, or ClusterRoleBinding for cluster-wide access

RBAC Debugging

Issue	Solution
Can't determine effective permissions	Use kubectl auth can-i --list --as=<user> -n <namespace> to see all permissions
Changes not taking effect	RBAC is eventually consistent - wait 30-60 seconds, or restart API server for immediate effect
Role exists but binding doesn't work	Verify roleRef matches Role/ClusterRole exactly (name, kind, apiGroup) - roleRef is immutable
Wildcards not working as expected	Wildcards (*) only match within same API group - use multiple rules for different groups

Service Account Issues

Issue	Solution
Token not mounted in pod	Check automountServiceAccountToken: true on SA and pod spec, verify volume mount exists
Token expired (1.24+)	Tokens are time-bound by default - regenerate with kubectl create token or use TokenRequest API
Can't find service account token secret	Kubernetes 1.24+ doesn't auto-create secrets - use kubectl create token instead
Pod can't authenticate to API	Ensure CA certificate is valid, check /var/run/secrets/kubernetes.io/serviceaccount/ contents

Multi-Tenancy Issues

Issue	Solution
Quota exceeded	Check kubectl describe resourcequota -n <namespace> and adjust quota or reduce usage
Network policy blocking traffic	Ensure DNS egress allowed, check namespace selectors, verify CNI supports NetworkPolicy
Cross-tenant access	Verify NetworkPolicy denies cross-namespace traffic, audit RoleBindings for namespace isolation
Resource limits causing CrashLoopBackOff	Review LimitRange settings, adjust defaults, or set explicit resource requests in pods

Audit and Compliance Issues

Issue	Solution
Audit logs not appearing	Verify audit policy file path in kube-apiserver, check API server logs for errors
Audit logs too large	Refine audit policy to reduce verbosity, implement log rotation, use remote audit backend
Can't track secret access	Set level: RequestResponse for secrets in audit policy (sensitive - encrypt audit logs)
Compliance violations not detected	Implement OPA/Gatekeeper policies, use admission webhooks for enforcement

Best Practices Violations

Issue	Solution
Too many cluster-admin bindings	Regular review and removal, use namespace-scoped admin role instead where possible
Service accounts with excessive permissions	Apply least privilege - grant only necessary verbs and resources, use resourceNames
No resource quotas on namespaces	Implement ResourceQuota on all tenant namespaces to prevent resource exhaustion
Secrets accessible by default SA	Never grant secret access to default SA - create dedicated service accounts
Wildcard permissions in production	Replace wildcards with explicit resource/verb lists, use aggregation for composition