FluxCD

GitOps toolkit for Kubernetes that enables continuous delivery through declarative configuration stored in Git repositories.

Overview

FluxCD is a set of continuous delivery tools that keeps Kubernetes clusters synchronised with configuration sources such as Git repositories and Helm repositories. It automates updates to configuration when there are new code deployments, follows the GitOps principles of declarative configuration and version control, and provides a modular architecture through its toolkit components (source-controller, kustomize-controller, helm-controller, notification-controller, image-automation-controller).

flowchart LR
    subgraph "GitOps Workflow"
        A[Developer] --> B[Git Repository]
        B --> C[Source Controller]
        C --> D[Kustomize/Helm Controller]
        D --> E{Reconcile}
        E -->|Apply| F[Kubernetes Cluster]
        F --> G[Notification Controller]
        G -->|Alert| H[Slack/Teams/PagerDuty]
    end

GitOps Principles

FluxCD implements core GitOps principles for declarative, versioned, and automated infrastructure management.

Key Concepts

Declarative Configuration: Desired system state is expressed declaratively in Git
Version Control: Git as the single source of truth for all configuration
Automated Synchronisation: Changes are automatically applied through reconciliation loops
Pull-based Deployment: Cluster pulls state from Git rather than CI pushing to cluster
Drift Detection: Continuous comparison between desired and actual state

flowchart TD
    subgraph "GitOps Principles"
        A[Declarative] --> B[Git as Source of Truth]
        B --> C[Versioned and Immutable]
        C --> D[Automatically Applied]
        D --> E[Continuously Reconciled]
        E --> A
    end

Benefits of GitOps

Benefit	Description
Audit Trail	Complete history of all changes in Git
Rollback	Revert to any previous state via Git
Security	No direct cluster access needed for deployments
Consistency	Same process for all environments
Self-healing	Automatic drift correction

FluxCD Architecture

graph TB
    subgraph "Flux Controllers"
        A[Source Controller] --> B[Kustomize Controller]
        A --> C[Helm Controller]
        A --> D[Image Automation Controller]
        B --> E[Kubernetes API]
        C --> E
        D --> A
        F[Notification Controller] --> G[External Systems]
        E --> F
    end

    subgraph "Sources"
        H[Git Repository] --> A
        I[Helm Repository] --> A
        J[OCI Repository] --> A
        K[Bucket S3/GCS] --> A
    end

Common Commands/Patterns

# Install Flux CLI
# Linux
curl -s https://fluxcd.io/install.sh | sudo bash

# macOS
brew install fluxcd/tap/flux

# Windows
choco install flux

# Verify installation
flux check --pre

# Bootstrap Flux on a cluster with GitHub
flux bootstrap github \
  --owner=my-github-username \
  --repository=fleet-infra \
  --branch=main \
  --path=clusters/my-cluster \
  --personal

# Bootstrap with GitLab
flux bootstrap gitlab \
  --owner=my-gitlab-group \
  --repository=fleet-infra \
  --branch=main \
  --path=clusters/my-cluster \
  --token-auth

# Check Flux status
flux check

# View all Flux resources
flux get all

# Export Flux resources
flux export source git --all > sources.yaml
flux export kustomization --all > kustomizations.yaml

Examples

Manual Flux installation:

# Install Flux components
flux install \
  --namespace=flux-system \
  --components-extra=image-reflector-controller,image-automation-controller

# Generate Flux manifests for GitOps
flux install \
  --export > flux-system.yaml

Repository Structure and Manifests

Organising Git repositories for effective FluxCD management.

Key Concepts

Monorepo: Single repository containing all cluster configurations
Multi-repo: Separate repositories for platform and application teams
GitRepository: Custom resource defining a Git source
Kustomization: Custom resource defining what to reconcile from a source

graph TD
    subgraph "Monorepo Structure"
        A[fleet-infra/] --> B[clusters/]
        A --> C[infrastructure/]
        A --> D[apps/]
        B --> E[production/]
        B --> F[staging/]
        C --> G[controllers/]
        C --> H[configs/]
        D --> I[base/]
        D --> J[overlays/]
    end

Repository Structure Patterns

Monorepo pattern:

fleet-infra/
├── clusters/
│   ├── production/
│   │   ├── flux-system/         # Flux components
│   │   ├── infrastructure.yaml  # Infrastructure Kustomization
│   │   └── apps.yaml           # Apps Kustomization
│   └── staging/
│       ├── flux-system/
│       ├── infrastructure.yaml
│       └── apps.yaml
├── infrastructure/
│   ├── controllers/
│   │   ├── cert-manager/
│   │   ├── ingress-nginx/
│   │   └── kustomization.yaml
│   └── configs/
│       ├── cluster-issuers/
│       └── kustomization.yaml
└── apps/
    ├── base/
    │   └── podinfo/
    │       ├── kustomization.yaml
    │       ├── deployment.yaml
    │       └── service.yaml
    └── overlays/
        ├── production/
        │   └── podinfo/
        └── staging/
            └── podinfo/

GitRepository Resource

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: flux-system
  namespace: flux-system
spec:
  # Sync interval
  interval: 1m0s

  # Repository URL
  url: ssh://git@github.com/myorg/fleet-infra

  # Branch/tag/commit to track
  ref:
    branch: main

  # Secret containing SSH key or token
  secretRef:
    name: flux-system

  # Ignore specific paths
  ignore: |
    # Exclude all
    /*
    # Include specific directories
    !/clusters/production/
---
# Git repository with HTTPS authentication
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: app-repo
  namespace: flux-system
spec:
  interval: 5m
  url: https://github.com/myorg/app-repo
  ref:
    branch: main
  secretRef:
    name: app-repo-auth
---
# Secret for HTTPS auth
apiVersion: v1
kind: Secret
metadata:
  name: app-repo-auth
  namespace: flux-system
type: Opaque
stringData:
  username: git
  password: ghp_xxxxxxxxxxxxxxxxxxxx

Kustomization Resource

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: infrastructure
  namespace: flux-system
spec:
  # Reconciliation interval
  interval: 10m0s

  # Retry on failure
  retryInterval: 1m0s

  # Timeout for apply operations
  timeout: 5m0s

  # Source reference
  sourceRef:
    kind: GitRepository
    name: flux-system

  # Path to kustomization directory
  path: ./infrastructure/controllers

  # Prune resources removed from source
  prune: true

  # Wait for resources to be ready
  wait: true

  # Dependencies
  dependsOn:
    - name: cert-manager

  # Health checks
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: nginx-ingress-controller
      namespace: ingress-nginx

  # Post-build variable substitution
  postBuild:
    substitute:
      cluster_name: production
      domain: example.com
    substituteFrom:
      - kind: ConfigMap
        name: cluster-vars
      - kind: Secret
        name: cluster-secrets

Common Commands/Patterns

# Create a GitRepository source
flux create source git flux-system \
  --url=ssh://git@github.com/myorg/fleet-infra \
  --branch=main \
  --ssh-key-algorithm=ecdsa \
  --ssh-ecdsa-curve=p521

# Create a Kustomization
flux create kustomization apps \
  --source=flux-system \
  --path="./apps/production" \
  --prune=true \
  --interval=10m

# List sources
flux get sources git
flux get sources helm
flux get sources oci

# List kustomizations
flux get kustomizations

# Reconcile a source immediately
flux reconcile source git flux-system

# Reconcile a kustomization
flux reconcile kustomization apps --with-source

Examples

Complete cluster bootstrap structure:

# clusters/production/flux-system/gotk-sync.yaml
---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: flux-system
  namespace: flux-system
spec:
  interval: 1m0s
  ref:
    branch: main
  secretRef:
    name: flux-system
  url: ssh://git@github.com/myorg/fleet-infra
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: flux-system
  namespace: flux-system
spec:
  interval: 10m0s
  path: ./clusters/production
  prune: true
  sourceRef:
    kind: GitRepository
    name: flux-system

# clusters/production/infrastructure.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: infrastructure
  namespace: flux-system
spec:
  interval: 1h
  retryInterval: 1m
  timeout: 5m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./infrastructure/controllers
  prune: true
  wait: true

# clusters/production/apps.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: apps
  namespace: flux-system
spec:
  dependsOn:
    - name: infrastructure
  interval: 1h
  retryInterval: 1m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./apps/production
  prune: true
  wait: true

Kustomize and Helm Integration

FluxCD natively supports both Kustomize overlays and Helm charts for application deployment.

Key Concepts

Kustomize Controller: Reconciles Kustomization resources and applies manifests
Helm Controller: Reconciles HelmRelease resources and manages Helm charts
HelmRepository: Source for Helm charts from traditional repositories
OCIRepository: Source for Helm charts stored in OCI registries

flowchart LR
    subgraph "Kustomize Flow"
        A[GitRepository] --> B[Kustomize Controller]
        B --> C[Apply Manifests]
    end

    subgraph "Helm Flow"
        D[HelmRepository/OCIRepository] --> E[Helm Controller]
        F[GitRepository - values] --> E
        E --> G[Install/Upgrade Release]
    end

Kustomize Integration

# Kustomization with patches and overlays
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: app-production
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./apps/production
  prune: true

  # Strategic merge patches
  patches:
    - patch: |
        apiVersion: apps/v1
        kind: Deployment
        metadata:
          name: app
        spec:
          replicas: 5
      target:
        kind: Deployment
        name: app

    # JSON patch
    - patch: |
        - op: replace
          path: /spec/replicas
          value: 3
      target:
        kind: Deployment
        labelSelector: "app=nginx"

  # Images override
  images:
    - name: app
      newName: myregistry/app
      newTag: v1.2.3

HelmRepository Resource

# Traditional Helm repository
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: bitnami
  namespace: flux-system
spec:
  interval: 1h
  url: https://charts.bitnami.com/bitnami
---
# OCI Helm repository
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: oci-charts
  namespace: flux-system
spec:
  interval: 1h
  type: oci
  url: oci://ghcr.io/myorg/charts
  secretRef:
    name: oci-creds
---
# Authenticated Helm repository
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: private-charts
  namespace: flux-system
spec:
  interval: 1h
  url: https://charts.example.com
  secretRef:
    name: helm-repo-auth
---
apiVersion: v1
kind: Secret
metadata:
  name: helm-repo-auth
  namespace: flux-system
type: Opaque
stringData:
  username: admin
  password: secretpassword

HelmRelease Resource

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: nginx
  namespace: default
spec:
  # Reconciliation interval
  interval: 30m

  # Chart reference
  chart:
    spec:
      chart: nginx
      version: ">=15.0.0 <16.0.0"
      sourceRef:
        kind: HelmRepository
        name: bitnami
        namespace: flux-system
      interval: 1h

  # Helm release name
  releaseName: nginx-production

  # Target namespace
  targetNamespace: web

  # Install configuration
  install:
    createNamespace: true
    remediation:
      retries: 3

  # Upgrade configuration
  upgrade:
    cleanupOnFail: true
    remediation:
      retries: 3
      remediateLastFailure: true

  # Rollback configuration
  rollback:
    timeout: 5m
    cleanupOnFail: true

  # Uninstall configuration
  uninstall:
    keepHistory: false

  # Helm values
  values:
    replicaCount: 3
    service:
      type: LoadBalancer
    resources:
      limits:
        cpu: 200m
        memory: 256Mi
      requests:
        cpu: 100m
        memory: 128Mi

Advanced HelmRelease Patterns

# HelmRelease with values from ConfigMap/Secret
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: app
  namespace: default
spec:
  interval: 30m
  chart:
    spec:
      chart: app
      sourceRef:
        kind: HelmRepository
        name: private-charts
        namespace: flux-system

  # Values from multiple sources
  valuesFrom:
    - kind: ConfigMap
      name: app-values
      valuesKey: values.yaml
    - kind: Secret
      name: app-secrets
      valuesKey: secrets.yaml
      optional: false

  # Inline values (lowest priority)
  values:
    replicaCount: 2
---
# HelmRelease from Git repository
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: app-from-git
  namespace: default
spec:
  interval: 30m
  chart:
    spec:
      chart: ./charts/app
      sourceRef:
        kind: GitRepository
        name: app-repo
        namespace: flux-system
      interval: 1h
  values:
    image:
      tag: latest
---
# HelmRelease from OCI registry
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: app-oci
  namespace: default
spec:
  interval: 30m
  chart:
    spec:
      chart: oci://ghcr.io/myorg/charts/app
      version: "1.2.3"
      sourceRef:
        kind: HelmRepository
        name: oci-charts
        namespace: flux-system

Common Commands/Patterns

# Create Helm repository source
flux create source helm bitnami \
  --url=https://charts.bitnami.com/bitnami \
  --interval=1h

# Create HelmRelease
flux create helmrelease nginx \
  --source=HelmRepository/bitnami \
  --chart=nginx \
  --target-namespace=web \
  --create-target-namespace \
  --values=./nginx-values.yaml

# List Helm releases
flux get helmreleases -A

# Reconcile a HelmRelease
flux reconcile helmrelease nginx --with-source

# Suspend HelmRelease (pause reconciliation)
flux suspend helmrelease nginx

# Resume HelmRelease
flux resume helmrelease nginx

# Export HelmRelease
flux export helmrelease nginx > nginx-release.yaml

Examples

Complete Helm-based deployment:

# infrastructure/controllers/cert-manager/helmrelease.yaml
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: cert-manager
  namespace: cert-manager
spec:
  interval: 1h
  chart:
    spec:
      chart: cert-manager
      version: "1.14.x"
      sourceRef:
        kind: HelmRepository
        name: jetstack
        namespace: flux-system
  install:
    createNamespace: true
    crds: CreateReplace
    remediation:
      retries: 3
  upgrade:
    crds: CreateReplace
    remediation:
      retries: 3
  values:
    installCRDs: false
    prometheus:
      enabled: true
      servicemonitor:
        enabled: true

Kustomize overlay with Helm:

# apps/production/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../base
  - helmrelease.yaml
patches:
  - patch: |
      - op: replace
        path: /spec/values/replicaCount
        value: 5
    target:
      kind: HelmRelease
      name: app

Sync Policies and Automation

Controlling how and when FluxCD synchronises resources with the cluster.

Key Concepts

Reconciliation Interval: How often Flux checks for changes
Prune: Remove resources deleted from source
Wait: Block until resources are ready
Dependencies: Order of resource deployment
Health Checks: Custom readiness conditions

stateDiagram-v2
    [*] --> Pending: Resource created
    Pending --> Reconciling: Interval elapsed
    Reconciling --> Ready: Apply successful
    Reconciling --> NotReady: Apply failed
    NotReady --> Reconciling: Retry interval
    Ready --> Reconciling: Source changed
    Ready --> Stale: Suspended
    Stale --> Reconciling: Resumed

Reconciliation Configuration

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: apps
  namespace: flux-system
spec:
  # How often to reconcile
  interval: 10m

  # How long to wait before retrying failed reconciliation
  retryInterval: 2m

  # Maximum time for reconciliation
  timeout: 5m

  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./apps/production

  # Prune resources removed from source
  prune: true

  # Wait for all resources to become ready
  wait: true

  # Force apply resources (replace instead of patch)
  force: false

Dependencies and Ordering

# Infrastructure must be ready before apps
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: apps
  namespace: flux-system
spec:
  dependsOn:
    - name: infrastructure-controllers
    - name: infrastructure-configs
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./apps/production
  prune: true
---
# Cert-manager before ingress
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: ingress-nginx
  namespace: flux-system
spec:
  dependsOn:
    - name: cert-manager
  interval: 1h
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./infrastructure/ingress-nginx
  prune: true
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: ingress-nginx-controller
      namespace: ingress-nginx

Health Checks and Readiness

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: apps
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./apps/production
  prune: true
  wait: true

  # Custom health checks
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: backend
      namespace: production
    - apiVersion: apps/v1
      kind: StatefulSet
      name: database
      namespace: production
    - apiVersion: networking.k8s.io/v1
      kind: Ingress
      name: frontend
      namespace: production

Post-Build Substitutions

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: apps
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./apps/production
  prune: true

  # Variable substitution
  postBuild:
    # Inline substitutions
    substitute:
      CLUSTER_NAME: production
      DOMAIN: prod.example.com
      ENVIRONMENT: production

    # Substitutions from ConfigMap/Secret
    substituteFrom:
      - kind: ConfigMap
        name: cluster-vars
      - kind: Secret
        name: cluster-secrets

Deployment using substitutions:

# apps/base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app
  namespace: ${NAMESPACE}
spec:
  replicas: ${REPLICAS:=2}
  template:
    spec:
      containers:
        - name: app
          image: ${IMAGE_REPO}/app:${IMAGE_TAG}
          env:
            - name: CLUSTER_NAME
              value: ${CLUSTER_NAME}
            - name: DATABASE_URL
              value: ${DATABASE_URL}

Image Automation

# Image repository scanning
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageRepository
metadata:
  name: app
  namespace: flux-system
spec:
  image: myregistry/app
  interval: 1m
  secretRef:
    name: registry-auth
---
# Image policy for selecting versions
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: app
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: app
  policy:
    semver:
      range: 1.x.x
---
# Alternative: Alphabetical policy
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: app-alpha
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: app
  policy:
    alphabetical:
      order: asc
---
# Alternative: Numerical policy
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImagePolicy
metadata:
  name: app-numerical
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: app
  filterTags:
    pattern: '^main-[a-f0-9]+-(?P<ts>[0-9]+)'
    extract: '$ts'
  policy:
    numerical:
      order: asc
---
# Image update automation
apiVersion: image.toolkit.fluxcd.io/v1
kind: ImageUpdateAutomation
metadata:
  name: app
  namespace: flux-system
spec:
  interval: 30m
  sourceRef:
    kind: GitRepository
    name: flux-system
  git:
    checkout:
      ref:
        branch: main
    commit:
      author:
        email: flux@example.com
        name: Flux
      messageTemplate: |
        Automated image update

        Automation: {{ .AutomationObject }}

        Files:
        {{ range $filename, $_ := .Changed.FileChanges -}}
        - {{ $filename }}
        {{ end -}}

        Objects:
        {{ range $resource, $changes := .Changed.Objects -}}
        - {{ $resource.Kind }}/{{ $resource.Name }}
        {{ end -}}
    push:
      branch: main
  update:
    path: ./apps
    strategy: Setters

Deployment with image markers:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app
spec:
  template:
    spec:
      containers:
        - name: app
          image: myregistry/app:1.0.0 # {"$imagepolicy": "flux-system:app"}

Common Commands/Patterns

# Suspend reconciliation
flux suspend kustomization apps
flux suspend helmrelease nginx

# Resume reconciliation
flux resume kustomization apps
flux resume helmrelease nginx

# Force reconciliation
flux reconcile kustomization apps --with-source

# Check reconciliation status
flux get kustomizations
flux get helmreleases

# View image policies
flux get image policy -A

# Trigger image scan
flux reconcile image repository app

Examples

Progressive delivery with dependencies:

# Deploy in order: infrastructure -> databases -> apps
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: infrastructure
  namespace: flux-system
spec:
  interval: 1h
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./infrastructure
  prune: true
  wait: true
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: databases
  namespace: flux-system
spec:
  dependsOn:
    - name: infrastructure
  interval: 1h
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./databases
  prune: true
  wait: true
  healthChecks:
    - apiVersion: apps/v1
      kind: StatefulSet
      name: postgresql
      namespace: databases
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: apps
  namespace: flux-system
spec:
  dependsOn:
    - name: databases
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./apps
  prune: true
  wait: true

Multi-Cluster Management

Managing multiple Kubernetes clusters with FluxCD.

Key Concepts

Fleet Management: Central repository managing multiple clusters
Cluster-specific Configuration: Overlays for each cluster
Remote Clusters: Connecting to clusters outside the management cluster
Workload Identity: Secure cross-cluster authentication

graph TB
    subgraph "Management Plane"
        A[Git Repository] --> B[fleet-infra]
    end

    subgraph "Clusters"
        C[Production Cluster]
        D[Staging Cluster]
        E[Development Cluster]
    end

    B --> C
    B --> D
    B --> E

    C --> F[Flux Controllers]
    D --> G[Flux Controllers]
    E --> H[Flux Controllers]

Multi-Cluster Repository Structure

fleet-infra/
├── clusters/
│   ├── production/
│   │   ├── flux-system/
│   │   │   ├── gotk-components.yaml
│   │   │   ├── gotk-sync.yaml
│   │   │   └── kustomization.yaml
│   │   ├── infrastructure.yaml
│   │   └── apps.yaml
│   ├── staging/
│   │   ├── flux-system/
│   │   ├── infrastructure.yaml
│   │   └── apps.yaml
│   └── development/
│       ├── flux-system/
│       ├── infrastructure.yaml
│       └── apps.yaml
├── infrastructure/
│   ├── controllers/
│   └── configs/
├── apps/
│   ├── base/
│   ├── production/
│   ├── staging/
│   └── development/
└── tenants/
    ├── team-a/
    └── team-b/

Cluster-Specific Configuration

# clusters/production/apps.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: apps
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./apps/production
  prune: true
  postBuild:
    substitute:
      CLUSTER_NAME: production
      CLUSTER_REGION: eu-west-1
      REPLICAS: "5"
    substituteFrom:
      - kind: ConfigMap
        name: cluster-config
---
# ConfigMap with cluster-specific values
apiVersion: v1
kind: ConfigMap
metadata:
  name: cluster-config
  namespace: flux-system
data:
  DOMAIN: prod.example.com
  DATABASE_HOST: prod-db.example.com

Remote Cluster Management

# Connect to remote cluster
apiVersion: v1
kind: Secret
metadata:
  name: remote-cluster
  namespace: flux-system
stringData:
  # Kubeconfig for remote cluster
  value: |
    apiVersion: v1
    kind: Config
    clusters:
      - name: remote
        cluster:
          server: https://remote-cluster.example.com
          certificate-authority-data: LS0tLS...
    contexts:
      - name: remote
        context:
          cluster: remote
          user: flux
    current-context: remote
    users:
      - name: flux
        user:
          token: eyJhbGciOi...
---
# Kustomization targeting remote cluster
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: remote-apps
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./apps/remote
  prune: true
  kubeConfig:
    secretRef:
      name: remote-cluster
      key: value

Multi-Tenancy

# Tenant namespace with RBAC
apiVersion: v1
kind: Namespace
metadata:
  name: team-a
  labels:
    toolkit.fluxcd.io/tenant: team-a
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: flux-team-a
  namespace: team-a
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: flux-team-a
  namespace: team-a
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - kind: ServiceAccount
    name: flux-team-a
    namespace: team-a
---
# Tenant GitRepository
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: team-a
  namespace: team-a
spec:
  interval: 1m
  url: https://github.com/myorg/team-a-apps
  ref:
    branch: main
  secretRef:
    name: team-a-git-auth
---
# Tenant Kustomization with service account
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: team-a-apps
  namespace: team-a
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: team-a
  path: ./apps
  prune: true
  serviceAccountName: flux-team-a
  targetNamespace: team-a

Common Commands/Patterns

# Bootstrap multiple clusters
flux bootstrap github \
  --owner=myorg \
  --repository=fleet-infra \
  --branch=main \
  --path=clusters/production

flux bootstrap github \
  --owner=myorg \
  --repository=fleet-infra \
  --branch=main \
  --path=clusters/staging

# Check cluster-specific resources
flux get all --context production-cluster
flux get all --context staging-cluster

# Compare configurations across clusters
diff <(flux get kustomizations --context production) \
     <(flux get kustomizations --context staging)

Examples

App-of-apps pattern for multi-cluster:

# clusters/production/tenants.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: tenants
  namespace: flux-system
spec:
  dependsOn:
    - name: infrastructure
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./tenants
  prune: true

# tenants/team-a/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - namespace.yaml
  - rbac.yaml
  - git-repo.yaml
  - apps-kustomization.yaml

Common Troubleshooting Scenarios

Diagnosing and resolving common FluxCD issues.

Key Concepts

Events: Kubernetes events from Flux controllers
Conditions: Status conditions on Flux resources
Logs: Controller logs for detailed error information
Reconciliation: Understanding the reconciliation process

flowchart TD
    A[Issue Detected] --> B{Check Status}
    B -->|Not Ready| C[Check Conditions]
    B -->|Stalled| D[Check Events]
    C --> E[Check Controller Logs]
    D --> E
    E --> F{Issue Type}
    F -->|Source| G[Git/Helm Auth]
    F -->|Apply| H[Manifest Errors]
    F -->|Health| I[Resource Issues]
    G --> J[Fix Credentials]
    H --> K[Fix Manifests]
    I --> L[Fix Resources]

Common Commands/Patterns

# Check overall Flux health
flux check

# Get all Flux resources with status
flux get all -A

# Describe specific resource
flux get kustomization apps -o yaml
flux get helmrelease nginx -o yaml

# View events for Flux resources
kubectl get events -n flux-system --sort-by='.lastTimestamp'

# View controller logs
kubectl logs -n flux-system deployment/source-controller -f
kubectl logs -n flux-system deployment/kustomize-controller -f
kubectl logs -n flux-system deployment/helm-controller -f

# Debug specific kustomization
flux logs --kind=Kustomization --name=apps

# Debug specific HelmRelease
flux logs --kind=HelmRelease --name=nginx

# Trace reconciliation
flux trace kustomization apps
flux trace helmrelease nginx

# Force reconciliation for debugging
flux reconcile source git flux-system
flux reconcile kustomization apps --with-source

# Check for failed resources
flux get all -A --status-selector ready=false

Common Issues and Solutions

Issue	Cause	Solution
`ssh: handshake failed`	SSH key not recognised	Regenerate SSH key and add to Git provider
`unable to clone`	Invalid credentials	Check secret contains correct token/key
`kustomize build failed`	Invalid kustomization.yaml	Run `kustomize build` locally to debug
`Helm install failed`	Chart version not found	Check HelmRepository is synced, verify version exists
`health check timeout`	Resource not becoming ready	Check pods/deployments in target namespace
`dependency not ready`	Dependent kustomization failed	Fix the dependency first
`drift detected`	Manual changes to cluster	Enable `prune: true` or fix source
`rate limit exceeded`	Too many Git/registry requests	Increase sync interval
`context deadline exceeded`	Timeout during apply	Increase `timeout` in spec
`namespace not found`	Target namespace doesn't exist	Add namespace to source or use `targetNamespace`

Debugging Source Issues

# Check GitRepository status
kubectl describe gitrepository flux-system -n flux-system

# Verify SSH connectivity
flux create secret git test-ssh \
  --url=ssh://git@github.com/myorg/repo \
  --ssh-key-algorithm=ecdsa \
  --ssh-ecdsa-curve=p521

# Check stored artifact
kubectl get gitrepository flux-system -n flux-system -o jsonpath='{.status.artifact}'

# View source-controller logs for auth errors
kubectl logs -n flux-system deployment/source-controller | grep -i error

# Test Helm repository access
helm repo add test https://charts.example.com
helm search repo test

Debugging Kustomization Issues

# Get detailed status
kubectl get kustomization apps -n flux-system -o yaml

# Check last applied revision
kubectl get kustomization apps -n flux-system \
  -o jsonpath='{.status.lastAppliedRevision}'

# View kustomize-controller errors
kubectl logs -n flux-system deployment/kustomize-controller | grep apps

# Test kustomize build locally
kustomize build ./apps/production

# Validate manifests
kubectl apply --dry-run=client -k ./apps/production

Debugging HelmRelease Issues

# Get detailed HelmRelease status
kubectl get helmrelease nginx -n default -o yaml

# Check Helm history
helm history nginx -n default

# View helm-controller errors
kubectl logs -n flux-system deployment/helm-controller | grep nginx

# Test Helm template locally
helm template nginx bitnami/nginx -f values.yaml

# Debug Helm install
helm install nginx bitnami/nginx --dry-run --debug -f values.yaml

# Check HelmChart artifact
kubectl get helmchart -n flux-system

Examples

Troubleshooting authentication failure:

# Check the error
flux get source git flux-system
# NAME         REVISION  SUSPENDED  READY  MESSAGE
# flux-system            False      False  ssh: handshake failed

# Regenerate the deploy key
flux create secret git flux-system \
  --url=ssh://git@github.com/myorg/fleet-infra \
  --ssh-key-algorithm=ecdsa \
  --ssh-ecdsa-curve=p521

# Get the public key
flux create secret git flux-system \
  --url=ssh://git@github.com/myorg/fleet-infra \
  --ssh-key-algorithm=ecdsa \
  --export | yq '.stringData["identity.pub"]'

# Add public key to GitHub as deploy key

# Force reconciliation
flux reconcile source git flux-system

Troubleshooting failed kustomization:

# Check the error
flux get kustomization apps
# NAME  REVISION  SUSPENDED  READY  MESSAGE
# apps            False      False  kustomize build failed

# Get detailed error
kubectl describe kustomization apps -n flux-system

# Check conditions
kubectl get kustomization apps -n flux-system -o jsonpath='{.status.conditions}'

# View controller logs
kubectl logs -n flux-system deployment/kustomize-controller --tail=100 | grep apps

# Test locally
cd fleet-infra
kustomize build ./apps/production

# Check for YAML syntax errors
yamllint ./apps/production/

Troubleshooting HelmRelease upgrade failure:

# Check status
flux get helmrelease nginx
# NAME   REVISION  SUSPENDED  READY  MESSAGE
# nginx  14.2.0    False      False  Helm upgrade failed

# Get failure reason
kubectl describe helmrelease nginx -n default

# Check Helm history
helm history nginx -n default
# REVISION  STATUS  DESCRIPTION
# 5         failed  Upgrade failed

# Rollback manually if needed
helm rollback nginx 4 -n default

# Fix the values and reapply
flux reconcile helmrelease nginx

Quick Reference

Command	Description
`flux check`	Verify Flux installation and prerequisites
`flux bootstrap`	Bootstrap Flux on a cluster
`flux install`	Install Flux components
`flux uninstall`	Uninstall Flux components
`flux get all`	List all Flux resources
`flux get sources git`	List GitRepository sources
`flux get sources helm`	List HelmRepository sources
`flux get kustomizations`	List Kustomizations
`flux get helmreleases`	List HelmReleases
`flux create source git`	Create a GitRepository source
`flux create source helm`	Create a HelmRepository source
`flux create kustomization`	Create a Kustomization
`flux create helmrelease`	Create a HelmRelease
`flux reconcile source git`	Trigger source reconciliation
`flux reconcile kustomization`	Trigger kustomization reconciliation
`flux reconcile helmrelease`	Trigger HelmRelease reconciliation
`flux suspend`	Suspend reconciliation
`flux resume`	Resume reconciliation
`flux export`	Export resources as YAML
`flux logs`	View Flux controller logs
`flux trace`	Trace resource reconciliation
`flux diff kustomization`	Show diff for kustomization

Useful Flags

Flag	Description
`--export`	Export resource as YAML
`-A, --all-namespaces`	List resources in all namespaces
`--with-source`	Reconcile source before resource
`--interval`	Set reconciliation interval
`--prune`	Enable resource pruning
`--wait`	Wait for resources to be ready
`--timeout`	Set operation timeout
`-o yaml/json`	Output format
`--context`	Kubernetes context to use

Resource Status Values

Status	Description
Ready=True	Resource is reconciled and healthy
Ready=False	Resource has reconciliation issues
Ready=Unknown	Status cannot be determined
Suspended=True	Reconciliation is paused

Flux Custom Resources

Resource	Controller	Purpose
GitRepository	source-controller	Git source
HelmRepository	source-controller	Helm chart source
OCIRepository	source-controller	OCI artifact source
Bucket	source-controller	S3/GCS bucket source
Kustomization	kustomize-controller	Kustomize deployment
HelmRelease	helm-controller	Helm chart deployment
ImageRepository	image-reflector-controller	Image scanning
ImagePolicy	image-reflector-controller	Image selection policy
ImageUpdateAutomation	image-automation-controller	Auto-update images
Alert	notification-controller	Event alerting
Provider	notification-controller	Alert destination
Receiver	notification-controller	Webhook receiver

Common Issues and Solutions

Issue	Cause	Solution
`failed to checkout`	Invalid Git reference	Verify branch/tag exists in repository
`authentication required`	Missing or invalid credentials	Update secret with correct credentials
`kustomize build failed`	Invalid Kustomize configuration	Test with `kustomize build` locally
`Helm install failed`	Chart or values error	Test with `helm template` locally
`context deadline exceeded`	Operation timeout	Increase `timeout` in resource spec
`dependency not ready`	Dependent resource failed	Fix the dependency first
`namespace not found`	Target namespace missing	Create namespace or add to source
`resource conflict`	Resource owned by another tool	Remove conflicting resource
`rate limit exceeded`	Too many API requests	Increase reconciliation interval
`certificate error`	TLS verification failed	Add CA certificate or skip verification

FluxCD

Overview

GitOps Principles

Key Concepts

Benefits of GitOps

FluxCD Architecture

Common Commands/Patterns

Examples

Repository Structure and Manifests

Key Concepts

Repository Structure Patterns

GitRepository Resource

Kustomization Resource

Common Commands/Patterns

Examples

Kustomize and Helm Integration

Key Concepts

Kustomize Integration

HelmRepository Resource

HelmRelease Resource

Advanced HelmRelease Patterns

Common Commands/Patterns

Examples

Sync Policies and Automation

Key Concepts

Reconciliation Configuration

Dependencies and Ordering

Health Checks and Readiness

Post-Build Substitutions

Image Automation

Common Commands/Patterns

Examples

Multi-Cluster Management

Key Concepts

Multi-Cluster Repository Structure

Cluster-Specific Configuration

Remote Cluster Management

Multi-Tenancy

Common Commands/Patterns

Examples

Common Troubleshooting Scenarios

Key Concepts

Common Commands/Patterns

Common Issues and Solutions

Debugging Source Issues

Debugging Kustomization Issues

Debugging HelmRelease Issues

Examples

Quick Reference

Useful Flags

Resource Status Values

Flux Custom Resources

Common Issues and Solutions

Related Topics