Argo Workflows

Kubernetes-native workflow engine for orchestrating parallel jobs, DAGs, and complex pipelines with container-based task execution.

Overview

Argo Workflows is an open-source container-native workflow engine for orchestrating parallel jobs on Kubernetes. It implements workflows as Kubernetes CRDs (Custom Resource Definitions), enabling users to define complex job orchestration with dependencies, parallelism, and artifact passing. Argo Workflows supports DAG-based and step-based workflows, making it suitable for CI/CD pipelines, data processing, machine learning workflows, and batch processing.

⤢ zoom

Installation and Setup

Argo Workflows can be installed in various modes depending on your requirements for namespace isolation and security.

Key Concepts

• Workflow Controller: Watches for Workflow resources and creates pods to execute tasks
• Argo Server: UI and API server for workflow management
• Executor: Runs inside workflow pods to manage containers and artifacts
• Artifact Repository: Storage for workflow artifacts (S3, GCS, MinIO, Artifactory)

⤢ zoom

Common Commands/Patterns

# Install Argo Workflows (cluster-wide)
kubectl create namespace argo
kubectl apply -n argo -f https://github.com/argoproj/argo-workflows/releases/latest/download/install.yaml

# Install Argo Workflows (namespace-scoped)
kubectl apply -n argo -f https://github.com/argoproj/argo-workflows/releases/latest/download/namespace-install.yaml

# Install using Helm
helm repo add argo https://argoproj.github.io/argo-helm
helm install argo-workflows argo/argo-workflows -n argo --create-namespace

# Install Argo CLI
# Linux
curl -sLO https://github.com/argoproj/argo-workflows/releases/latest/download/argo-linux-amd64.gz
gunzip argo-linux-amd64.gz && chmod +x argo-linux-amd64 && sudo mv argo-linux-amd64 /usr/local/bin/argo

# macOS
brew install argo

# Access the UI
kubectl -n argo port-forward deployment/argo-server 2746:2746

# Configure authentication (create token)
kubectl -n argo create sa argo-user
kubectl -n argo create rolebinding argo-user --clusterrole=argo-admin --serviceaccount=argo:argo-user
# Kubernetes 1.24+ no longer auto-creates a token Secret per ServiceAccount.
# Mint a short-lived token directly:
TOKEN=$(kubectl -n argo create token argo-user)
echo "Bearer $TOKEN"

# For a long-lived token, create a token Secret explicitly:
# kubectl -n argo apply -f - <<EOF
# apiVersion: v1
# kind: Secret
# metadata:
#   name: argo-user.service-account-token
#   annotations:
#     kubernetes.io/service-account.name: argo-user
# type: kubernetes.io/service-account-token
# EOF
# TOKEN=$(kubectl -n argo get secret argo-user.service-account-token -o jsonpath='{.data.token}' | base64 -d)

Examples

Configure artifact repository:

apiVersion: v1
kind: ConfigMap
metadata:
  name: artifact-repositories
  namespace: argo
data:
  default-v1: |
    archiveLogs: true
    s3:
      bucket: my-bucket
      endpoint: s3.amazonaws.com
      region: eu-west-1
      accessKeySecret:
        name: s3-creds
        key: accessKey
      secretKeySecret:
        name: s3-creds
        key: secretKey

Workflow Manifest Structure

The Workflow CRD defines the complete specification for workflow execution, including templates, inputs, outputs, and execution parameters.

Key Concepts

• Workflow: The top-level resource defining the entire workflow
• Spec: Contains templates, entrypoint, arguments, and workflow-level settings
• Entrypoint: The template to execute when the workflow starts
• Templates: Reusable units of work (containers, scripts, DAGs, steps)
• Arguments: Input parameters and artifacts passed to the workflow

⤢ zoom

Basic Workflow Structure

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  # Workflow name (must be unique, often use generateName)
  generateName: my-workflow-
  namespace: argo
  labels:
    workflows.argoproj.io/archive-strategy: "false"
spec:
  # Template to execute first
  entrypoint: main

  # Workflow-level arguments
  arguments:
    parameters:
      - name: message
        value: "Hello World"
      - name: environment
        value: "production"

  # Service account for pod execution
  serviceAccountName: argo-workflow

  # Pod-level security context
  securityContext:
    runAsNonRoot: true
    runAsUser: 1000

  # Workflow timeout
  activeDeadlineSeconds: 3600

  # TTL after completion
  ttlStrategy:
    secondsAfterCompletion: 3600
    secondsAfterSuccess: 1800
    secondsAfterFailure: 7200

  # Pod garbage collection
  podGC:
    strategy: OnPodCompletion

  # Retry strategy for all templates
  retryStrategy:
    limit: 3
    retryPolicy: "Always"
    backoff:
      duration: "5s"
      factor: 2
      maxDuration: "1m"

  # Volume definitions
  volumes:
    - name: shared-data
      emptyDir: {}
    - name: config
      configMap:
        name: workflow-config

  # Templates definition
  templates:
    - name: main
      container:
        image: alpine:latest
        command: [echo]
        args: ["{{workflow.parameters.message}}"]

Workflow-Level Settings

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: advanced-workflow-
spec:
  entrypoint: main

  # Node selector for all pods
  nodeSelector:
    kubernetes.io/os: linux

  # Tolerations
  tolerations:
    - key: "dedicated"
      operator: "Equal"
      value: "workflows"
      effect: "NoSchedule"

  # Affinity rules
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: topology.kubernetes.io/zone
                operator: In
                values:
                  - eu-west-1a
                  - eu-west-1b

  # Priority and preemption
  priority: 100
  priorityClassName: high-priority

  # Workflow-level parallelism
  parallelism: 10

  # Synchronisation (mutex/semaphore)
  synchronization:
    mutex:
      name: my-mutex

  # Workflow metadata for pods
  podMetadata:
    labels:
      app: argo-workflow
    annotations:
      sidecar.istio.io/inject: "false"

  # Image pull secrets
  imagePullSecrets:
    - name: registry-creds

  # Hooks for workflow lifecycle
  hooks:
    exit:
      template: cleanup
    running:
      template: notify-start

  templates:
    - name: main
      container:
        image: alpine:latest
        command: [echo, "Hello"]

    - name: cleanup
      container:
        image: alpine:latest
        command: [echo, "Cleaning up"]

    - name: notify-start
      container:
        image: alpine:latest
        command: [echo, "Workflow started"]

Common Commands/Patterns

# Submit a workflow
argo submit workflow.yaml -n argo

# Submit with parameter override
argo submit workflow.yaml -p message="Custom message" -n argo

# Submit from stdin
cat workflow.yaml | argo submit -n argo -

# List workflows
argo list -n argo

# Get workflow details
argo get my-workflow -n argo

# Watch workflow progress
argo watch my-workflow -n argo

# View workflow logs
argo logs my-workflow -n argo

# Delete workflow
argo delete my-workflow -n argo

# Retry failed workflow
argo retry my-workflow -n argo

# Resubmit workflow
argo resubmit my-workflow -n argo

# Terminate running workflow
argo terminate my-workflow -n argo

# Stop workflow (allow running pods to complete)
argo stop my-workflow -n argo

Examples

WorkflowTemplate for reusability:

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: build-and-test
  namespace: argo
spec:
  entrypoint: main
  arguments:
    parameters:
      - name: repo-url
      - name: branch
        value: main

  templates:
    - name: main
      steps:
        - - name: checkout
            template: git-clone
            arguments:
              parameters:
                - name: url
                  value: "{{workflow.parameters.repo-url}}"
                - name: branch
                  value: "{{workflow.parameters.branch}}"
        - - name: build
            template: build-app
        - - name: test
            template: run-tests

    - name: git-clone
      inputs:
        parameters:
          - name: url
          - name: branch
      container:
        image: alpine/git
        command: [git, clone, "--branch", "{{inputs.parameters.branch}}", "{{inputs.parameters.url}}"]

    - name: build-app
      container:
        image: golang:1.21
        command: [go, build, -o, app]
        workingDir: /workspace

    - name: run-tests
      container:
        image: golang:1.21
        command: [go, test, ./...]
        workingDir: /workspace
---
# Invoke the WorkflowTemplate
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: invoke-template-
spec:
  workflowTemplateRef:
    name: build-and-test
  arguments:
    parameters:
      - name: repo-url
        value: "https://github.com/myorg/myapp"
      - name: branch
        value: "develop"

Template Types

Argo Workflows supports multiple template types for different execution patterns and use cases.

Key Concepts

• Container: Runs a single container (most common)
• Script: Runs inline scripts with automatic result capture
• Resource: Creates/patches/deletes Kubernetes resources
• DAG: Defines tasks with dependencies
• Steps: Sequential or parallel task execution
• Suspend: Pauses workflow execution

⤢ zoom

Container Template

templates:
  - name: container-example
    # Input parameters
    inputs:
      parameters:
        - name: image-tag
          default: "latest"

    # Container specification
    container:
      image: "myregistry/myapp:{{inputs.parameters.image-tag}}"
      command: ["/bin/sh", "-c"]
      args:
        - |
          echo "Running task"
          ./process-data.sh
          echo "Task completed"

      # Environment variables
      env:
        - name: ENV_VAR
          value: "value"
        - name: SECRET_VAR
          valueFrom:
            secretKeyRef:
              name: my-secret
              key: password

      # Resource limits
      resources:
        requests:
          memory: "256Mi"
          cpu: "100m"
        limits:
          memory: "512Mi"
          cpu: "500m"

      # Volume mounts
      volumeMounts:
        - name: shared-data
          mountPath: /data

      # Working directory
      workingDir: /app

Script Template

templates:
  # Python script
  - name: python-script
    inputs:
      parameters:
        - name: data
    script:
      image: python:3.11-slim
      command: [python]
      source: |
        import json
        import sys

        data = '''{{inputs.parameters.data}}'''
        parsed = json.loads(data)

        result = {
            'processed': True,
            'count': len(parsed),
            'items': parsed
        }

        # Output to stdout (captured as result)
        print(json.dumps(result))

      resources:
        requests:
          memory: "128Mi"
          cpu: "50m"

  # Bash script
  - name: bash-script
    inputs:
      parameters:
        - name: filename
    script:
      image: alpine:latest
      command: [sh]
      source: |
        #!/bin/sh
        set -e

        filename="{{inputs.parameters.filename}}"

        # Process file
        wc -l "$filename"

        # Output result
        echo "Processed: $filename"

  # Node.js script
  - name: node-script
    script:
      image: node:18-alpine
      command: [node]
      source: |
        const https = require('https');

        const options = {
          hostname: 'api.example.com',
          path: '/data',
          method: 'GET'
        };

        const req = https.request(options, (res) => {
          let data = '';
          res.on('data', (chunk) => data += chunk);
          res.on('end', () => console.log(data));
        });

        req.end();

Resource Template

templates:
  # Create a Kubernetes resource
  - name: create-configmap
    inputs:
      parameters:
        - name: name
        - name: data
    resource:
      action: create
      manifest: |
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: "{{inputs.parameters.name}}"
          namespace: default
        data:
          config.yaml: |
            {{inputs.parameters.data}}

  # Patch a resource
  - name: scale-deployment
    inputs:
      parameters:
        - name: deployment-name
        - name: replicas
    resource:
      action: patch
      mergeStrategy: strategic
      manifest: |
        apiVersion: apps/v1
        kind: Deployment
        metadata:
          name: "{{inputs.parameters.deployment-name}}"
          namespace: default
        spec:
          replicas: {{inputs.parameters.replicas}}

  # Delete a resource
  - name: delete-job
    inputs:
      parameters:
        - name: job-name
    resource:
      action: delete
      manifest: |
        apiVersion: batch/v1
        kind: Job
        metadata:
          name: "{{inputs.parameters.job-name}}"
          namespace: default

  # Create and wait for completion
  - name: run-job
    resource:
      action: create
      successCondition: status.succeeded > 0
      failureCondition: status.failed > 3
      manifest: |
        apiVersion: batch/v1
        kind: Job
        metadata:
          generateName: data-job-
        spec:
          template:
            spec:
              containers:
                - name: job
                  image: alpine
                  command: ["/bin/sh", "-c", "echo Done"]
              restartPolicy: Never

Suspend Template

templates:
  # Manual approval
  - name: manual-approval
    suspend: {}

  # Timed suspension
  - name: wait-period
    suspend:
      duration: "1h"

  # Complete workflow with suspend
  - name: main
    steps:
      - - name: build
          template: build-app
      - - name: approval
          template: manual-approval
      - - name: deploy
          template: deploy-app

Examples

HTTP template (plugin):

templates:
  - name: http-request
    inputs:
      parameters:
        - name: url
        - name: method
          default: "GET"
    http:
      url: "{{inputs.parameters.url}}"
      method: "{{inputs.parameters.method}}"
      headers:
        - name: Content-Type
          value: application/json
      successCondition: response.statusCode == 200
      body: |
        {"key": "value"}

Container with lifecycle hooks:

templates:
  - name: with-hooks
    container:
      image: myapp:latest
      command: [./run.sh]
      lifecycle:
        postStart:
          exec:
            command: ["/bin/sh", "-c", "echo 'Started'"]
        preStop:
          exec:
            command: ["/bin/sh", "-c", "echo 'Stopping'"]

DAG and Step-Based Workflows

Argo Workflows supports two primary patterns for orchestrating tasks: DAG (Directed Acyclic Graph) and Steps.

Key Concepts

• DAG: Tasks defined with explicit dependencies, executed in parallel when possible
• Steps: Sequential phases, each containing parallel tasks
• Dependencies: Define execution order in DAGs
• When expressions: Conditional task execution

⤢ zoom

DAG Template

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dag-workflow-
spec:
  entrypoint: main
  templates:
    - name: main
      dag:
        # Fail fast on first failure
        failFast: true

        tasks:
          # Root tasks (no dependencies)
          - name: checkout
            template: git-clone

          # Parallel tasks depending on checkout
          - name: build-frontend
            template: build
            dependencies: [checkout]
            arguments:
              parameters:
                - name: component
                  value: frontend

          - name: build-backend
            template: build
            dependencies: [checkout]
            arguments:
              parameters:
                - name: component
                  value: backend

          # Task depending on multiple predecessors
          - name: integration-tests
            template: test
            dependencies: [build-frontend, build-backend]

          # Conditional task
          - name: deploy
            template: deploy-app
            dependencies: [integration-tests]
            when: "{{workflow.parameters.deploy}} == true"

          # Task with artifact dependency
          - name: analyse-results
            template: analyse
            dependencies: [integration-tests]
            arguments:
              artifacts:
                - name: test-results
                  from: "{{tasks.integration-tests.outputs.artifacts.results}}"

    - name: git-clone
      container:
        image: alpine/git
        command: [git, clone, "https://github.com/myorg/myapp", "/workspace"]
      outputs:
        artifacts:
          - name: source
            path: /workspace

    - name: build
      inputs:
        parameters:
          - name: component
      container:
        image: node:18
        command: [npm, run, build]
        workingDir: "/workspace/{{inputs.parameters.component}}"

    - name: test
      container:
        image: node:18
        command: [npm, test]
      outputs:
        artifacts:
          - name: results
            path: /workspace/test-results

    - name: deploy-app
      container:
        image: kubectl:latest
        command: [kubectl, apply, -f, deployment.yaml]

    - name: analyse
      inputs:
        artifacts:
          - name: test-results
            path: /results
      container:
        image: python:3.11
        command: [python, analyse.py]

Steps Template

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: steps-workflow-
spec:
  entrypoint: main
  templates:
    - name: main
      steps:
        # Step 1: Single task
        - - name: checkout
            template: git-clone

        # Step 2: Parallel tasks
        - - name: lint
            template: run-lint
          - name: type-check
            template: run-type-check
          - name: unit-tests
            template: run-unit-tests

        # Step 3: Conditional execution
        - - name: build
            template: build-app
            when: "{{steps.unit-tests.outputs.result}} == passed"

        # Step 4: Sequential with output passing
        - - name: publish
            template: publish-artifact
            arguments:
              parameters:
                - name: version
                  value: "{{steps.build.outputs.parameters.version}}"

        # Step 5: Multiple parallel deployments
        - - name: deploy-staging
            template: deploy
            arguments:
              parameters:
                - name: environment
                  value: staging
          - name: notify
            template: send-notification

    - name: git-clone
      container:
        image: alpine/git
        command: [git, clone, "https://github.com/myorg/myapp"]

    - name: run-lint
      container:
        image: node:18
        command: [npm, run, lint]

    - name: run-type-check
      container:
        image: node:18
        command: [npm, run, type-check]

    - name: run-unit-tests
      container:
        image: node:18
        command: [npm, test]
      outputs:
        parameters:
          - name: result
            valueFrom:
              path: /tmp/result

    - name: build-app
      container:
        image: node:18
        command: [npm, run, build]
      outputs:
        parameters:
          - name: version
            valueFrom:
              path: /tmp/version

    - name: publish-artifact
      inputs:
        parameters:
          - name: version
      container:
        image: docker:latest
        command: [docker, push, "myregistry/myapp:{{inputs.parameters.version}}"]

    - name: deploy
      inputs:
        parameters:
          - name: environment
      container:
        image: kubectl:latest
        command: [kubectl, apply, -f, "{{inputs.parameters.environment}}/"]

    - name: send-notification
      container:
        image: curlimages/curl
        command: [curl, -X, POST, "https://hooks.slack.com/..."]

Advanced DAG Patterns

templates:
  - name: complex-dag
    dag:
      tasks:
        # Task with retry
        - name: flaky-task
          template: might-fail
          continueOn:
            failed: true

        # Task with custom timeout
        - name: long-running
          template: process-data
          activeDeadlineSeconds: 3600

        # Task group with dependencies on any/all
        - name: aggregate
          template: combine-results
          dependencies: [task-a, task-b, task-c]
          depends: "task-a.Succeeded && (task-b.Succeeded || task-c.Succeeded)"

        # Conditional based on parameter
        - name: prod-only
          template: deploy-prod
          when: "{{workflow.parameters.env}} == production"

        # With hooks
        - name: with-hooks
          template: process
          hooks:
            exit:
              template: cleanup
            success:
              template: notify-success
            failure:
              template: notify-failure

When Expression Syntax

# Basic comparisons
when: "{{steps.test.outputs.result}} == passed"
when: "{{workflow.parameters.count}} > 10"
when: "{{tasks.check.outputs.result}} != error"

# Status checks
when: "{{steps.build.status}} == Succeeded"
when: "{{tasks.test.status}} == Failed"

# Boolean expressions
when: "{{workflow.parameters.deploy}} == true"

# String operations
when: "'{{workflow.parameters.env}}' =~ 'prod'"
when: "'{{steps.output.result}}' contains 'success'"

# Complex conditions (DAG only)
depends: "task-a.Succeeded && task-b.Succeeded"
depends: "task-a || task-b"  # Either succeeded
depends: "(task-a.Succeeded || task-a.Skipped) && task-b.Succeeded"

Examples

Diamond pattern DAG:

templates:
  - name: diamond
    dag:
      tasks:
        - name: A
          template: task
        - name: B
          template: task
          dependencies: [A]
        - name: C
          template: task
          dependencies: [A]
        - name: D
          template: task
          dependencies: [B, C]

Nested steps and DAGs:

templates:
  - name: main
    steps:
      - - name: setup
          template: setup-env
      - - name: parallel-dags
          template: parallel-processing
      - - name: cleanup
          template: cleanup

  - name: parallel-processing
    dag:
      tasks:
        - name: process-a
          template: process
        - name: process-b
          template: process
        - name: combine
          template: aggregate
          dependencies: [process-a, process-b]

Artifacts Management

Artifacts enable passing files and data between workflow steps and storing workflow outputs.

Key Concepts

• Input artifacts: Files downloaded before task execution
• Output artifacts: Files uploaded after task execution
• Artifact repository: Backend storage (S3, GCS, MinIO, Artifactory)
• Artifact passing: Reference outputs from previous tasks

⤢ zoom

Artifact Repository Configuration

# Controller ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
  name: artifact-repositories
  namespace: argo
data:
  default-v1: |
    archiveLogs: true
    s3:
      bucket: argo-artifacts
      endpoint: s3.amazonaws.com
      region: eu-west-1
      insecure: false
      accessKeySecret:
        name: argo-artifacts
        key: accessKey
      secretKeySecret:
        name: argo-artifacts
        key: secretKey
      keyFormat: "{{workflow.namespace}}/{{workflow.name}}/{{pod.name}}"
---
# MinIO configuration
apiVersion: v1
kind: ConfigMap
metadata:
  name: artifact-repositories
  namespace: argo
data:
  default-v1: |
    s3:
      bucket: argo-artifacts
      endpoint: minio.argo:9000
      insecure: true
      accessKeySecret:
        name: minio-creds
        key: accessKey
      secretKeySecret:
        name: minio-creds
        key: secretKey
---
# GCS configuration
apiVersion: v1
kind: ConfigMap
metadata:
  name: artifact-repositories
  namespace: argo
data:
  default-v1: |
    gcs:
      bucket: argo-artifacts
      serviceAccountKeySecret:
        name: gcs-creds
        key: serviceAccountKey

Input and Output Artifacts

templates:
  - name: generate-data
    container:
      image: python:3.11
      command: [python, -c]
      args:
        - |
          import json
          data = {'items': list(range(100))}
          with open('/tmp/data.json', 'w') as f:
              json.dump(data, f)
    outputs:
      artifacts:
        - name: data
          path: /tmp/data.json
          # Optional: archive settings
          archive:
            none: {}  # No compression
          # Optional: S3 settings override
          s3:
            key: "custom/path/data.json"

  - name: process-data
    inputs:
      artifacts:
        - name: input-data
          path: /data/input.json
          # Optional source
          optional: false
    container:
      image: python:3.11
      command: [python, process.py]
    outputs:
      artifacts:
        - name: processed
          path: /data/output.json
          # Archive with compression
          archive:
            tar:
              compressionLevel: 9

  - name: aggregate-results
    inputs:
      artifacts:
        # Multiple inputs
        - name: result-1
          path: /results/1.json
        - name: result-2
          path: /results/2.json
        - name: result-3
          path: /results/3.json
          optional: true  # May not exist
    container:
      image: python:3.11
      command: [python, aggregate.py]

Artifact Sources

templates:
  - name: from-various-sources
    inputs:
      artifacts:
        # From S3
        - name: from-s3
          path: /data/s3-file
          s3:
            endpoint: s3.amazonaws.com
            bucket: my-bucket
            key: path/to/file
            accessKeySecret:
              name: s3-creds
              key: accessKey
            secretKeySecret:
              name: s3-creds
              key: secretKey

        # From GCS
        - name: from-gcs
          path: /data/gcs-file
          gcs:
            bucket: my-bucket
            key: path/to/file
            serviceAccountKeySecret:
              name: gcs-creds
              key: serviceAccountKey

        # From Git
        - name: from-git
          path: /src
          git:
            repo: https://github.com/myorg/myrepo.git
            revision: main
            depth: 1
            usernameSecret:
              name: git-creds
              key: username
            passwordSecret:
              name: git-creds
              key: password

        # From HTTP
        - name: from-http
          path: /data/http-file
          http:
            url: https://example.com/data.json
            headers:
              - name: Authorization
                value: "Bearer token"

        # Raw data
        - name: config
          path: /config/settings.yaml
          raw:
            data: |
              setting1: value1
              setting2: value2
    container:
      image: alpine:latest
      command: [ls, -la, /data]

Passing Artifacts Between Tasks

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: artifact-passing-
spec:
  entrypoint: main
  templates:
    - name: main
      dag:
        tasks:
          - name: generate
            template: generate-data

          - name: process
            template: process-data
            dependencies: [generate]
            arguments:
              artifacts:
                - name: input-data
                  from: "{{tasks.generate.outputs.artifacts.data}}"

          - name: store
            template: store-results
            dependencies: [process]
            arguments:
              artifacts:
                - name: results
                  from: "{{tasks.process.outputs.artifacts.processed}}"

    - name: generate-data
      container:
        image: alpine:latest
        command: [sh, -c]
        args: ["echo 'test data' > /tmp/output.txt"]
      outputs:
        artifacts:
          - name: data
            path: /tmp/output.txt

    - name: process-data
      inputs:
        artifacts:
          - name: input-data
            path: /input/data.txt
      container:
        image: alpine:latest
        command: [sh, -c]
        args: ["cat /input/data.txt | tr 'a-z' 'A-Z' > /output/result.txt"]
      outputs:
        artifacts:
          - name: processed
            path: /output/result.txt

    - name: store-results
      inputs:
        artifacts:
          - name: results
            path: /final/results.txt
      container:
        image: alpine:latest
        command: [cat, /final/results.txt]

Common Commands/Patterns

# View artifact information
argo get my-workflow -n argo

# Download artifacts
argo logs my-workflow -n argo --artifact-name=output

# List workflow artifacts
kubectl get workflow my-workflow -o jsonpath='{.status.nodes[*].outputs.artifacts}'

Examples

Artifact garbage collection:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: artifact-gc-
spec:
  entrypoint: main
  artifactGC:
    strategy: OnWorkflowCompletion
    serviceAccountName: artifact-gc
    forceFinalizerRemoval: false

  templates:
    - name: main
      container:
        image: alpine:latest
        command: [echo, "Hello"]
      outputs:
        artifacts:
          - name: output
            path: /tmp/output
            artifactGC:
              strategy: OnWorkflowDeletion  # Override per artifact

Large file handling:

templates:
  - name: large-file-handler
    inputs:
      artifacts:
        - name: large-file
          path: /data/large.bin
          # Use streaming for large files
          archive:
            none: {}
    container:
      image: alpine:latest
      command: [sh, -c]
      args: ["split -b 100m /data/large.bin /output/part-"]
    outputs:
      artifacts:
        - name: parts
          path: /output/
          archive:
            tar: {}

Parameters and Outputs

Parameters and outputs enable data passing between templates and workflow customisation.

Key Concepts

• Workflow parameters: Top-level inputs to the workflow
• Template parameters: Inputs to individual templates
• Output parameters: Values exported from templates
• Result: Special output from script templates
• Global parameters: Workflow-wide accessible values

⤢ zoom

Workflow Parameters

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: parameterised-
spec:
  entrypoint: main

  # Workflow-level parameters
  arguments:
    parameters:
      - name: message
        value: "default message"
      - name: count
        value: "5"
      - name: environment
        enum:
          - development
          - staging
          - production
        value: development
      - name: config
        # Complex parameter (JSON)
        value: |
          {
            "setting1": "value1",
            "setting2": "value2"
          }

  templates:
    - name: main
      container:
        image: alpine:latest
        command: [sh, -c]
        args:
          - |
            echo "Message: {{workflow.parameters.message}}"
            echo "Count: {{workflow.parameters.count}}"
            echo "Environment: {{workflow.parameters.environment}}"
            echo "Config: {{workflow.parameters.config}}"

Template Parameters

templates:
  - name: parameterised-task
    inputs:
      parameters:
        - name: required-param
          # No default = required
        - name: optional-param
          default: "default value"
        - name: enum-param
          enum: [option1, option2, option3]
          default: option1

    container:
      image: alpine:latest
      command: [echo]
      args:
        - "{{inputs.parameters.required-param}}"
        - "{{inputs.parameters.optional-param}}"

  - name: call-parameterised
    steps:
      - - name: step1
          template: parameterised-task
          arguments:
            parameters:
              - name: required-param
                value: "passed value"
              - name: optional-param
                value: "custom value"

Output Parameters

templates:
  # Output from file
  - name: file-output
    container:
      image: alpine:latest
      command: [sh, -c]
      args:
        - |
          echo "v1.2.3" > /tmp/version
          echo '{"status": "success"}' > /tmp/result.json
    outputs:
      parameters:
        - name: version
          valueFrom:
            path: /tmp/version
        - name: result
          valueFrom:
            path: /tmp/result.json

  # Output from JSON path
  - name: json-output
    container:
      image: alpine:latest
      command: [sh, -c]
      args: ["echo '{\"name\": \"test\", \"value\": 42}' > /tmp/data.json"]
    outputs:
      parameters:
        - name: extracted-name
          valueFrom:
            path: /tmp/data.json
            jqFilter: '.name'
        - name: extracted-value
          valueFrom:
            path: /tmp/data.json
            jqFilter: '.value'

  # Script result output
  - name: script-output
    script:
      image: python:3.11
      command: [python]
      source: |
        import json
        result = {"computed": True, "value": 42}
        # stdout is captured as 'result'
        print(json.dumps(result))
    outputs:
      parameters:
        - name: result
          valueFrom:
            path: /tmp/result  # Script stdout

  # Expression-based output
  - name: expression-output
    container:
      image: alpine:latest
      command: [echo, "done"]
    outputs:
      parameters:
        - name: computed
          valueFrom:
            expression: "5 * 10"
        - name: status
          valueFrom:
            expression: "'completed'"

Passing Parameters Between Tasks

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: param-passing-
spec:
  entrypoint: main
  templates:
    - name: main
      dag:
        tasks:
          - name: generate
            template: generate-version

          - name: use-version
            template: deploy
            dependencies: [generate]
            arguments:
              parameters:
                - name: version
                  value: "{{tasks.generate.outputs.parameters.version}}"

          - name: conditional
            template: notify
            dependencies: [use-version]
            when: "{{tasks.use-version.outputs.parameters.status}} == success"
            arguments:
              parameters:
                - name: message
                  value: "Deployed {{tasks.generate.outputs.parameters.version}}"

    # Steps example
    - name: steps-example
      steps:
        - - name: step1
            template: generate-version
        - - name: step2
            template: deploy
            arguments:
              parameters:
                - name: version
                  value: "{{steps.step1.outputs.parameters.version}}"

    - name: generate-version
      container:
        image: alpine:latest
        command: [sh, -c, "echo 'v1.2.3' > /tmp/version"]
      outputs:
        parameters:
          - name: version
            valueFrom:
              path: /tmp/version

    - name: deploy
      inputs:
        parameters:
          - name: version
      container:
        image: alpine:latest
        command: [sh, -c]
        args: ["echo 'Deploying {{inputs.parameters.version}}' && echo 'success' > /tmp/status"]
      outputs:
        parameters:
          - name: status
            valueFrom:
              path: /tmp/status

    - name: notify
      inputs:
        parameters:
          - name: message
      container:
        image: curlimages/curl
        command: [echo, "{{inputs.parameters.message}}"]

Global Parameters

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: global-params-
spec:
  entrypoint: main
  templates:
    - name: main
      dag:
        tasks:
          - name: setup
            template: setup-globals

          - name: use-globals
            template: use-global-values
            dependencies: [setup]

    - name: setup-globals
      container:
        image: alpine:latest
        command: [sh, -c]
        args: ["echo 'global-value' > /tmp/global"]
      outputs:
        parameters:
          - name: global-param
            valueFrom:
              path: /tmp/global
            globalName: my-global  # Makes it globally accessible

    - name: use-global-values
      container:
        image: alpine:latest
        command: [echo]
        args:
          # Access global parameter
          - "{{workflow.outputs.parameters.my-global}}"

Built-in Parameters

templates:
  - name: builtin-params
    container:
      image: alpine:latest
      command: [sh, -c]
      args:
        - |
          echo "Workflow name: {{workflow.name}}"
          echo "Workflow namespace: {{workflow.namespace}}"
          echo "Workflow UID: {{workflow.uid}}"
          echo "Workflow creation timestamp: {{workflow.creationTimestamp}}"
          echo "Pod name: {{pod.name}}"
          echo "Node name: {{node.name}}"
          echo "Template name: {{template.name}}"
          echo "Retries: {{retries}}"

Examples

Parameter validation:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: validated-params-
spec:
  entrypoint: main
  arguments:
    parameters:
      - name: count
        value: "10"

  templates:
    - name: main
      steps:
        - - name: validate
            template: check-params
            arguments:
              parameters:
                - name: count
                  value: "{{workflow.parameters.count}}"
        - - name: process
            template: process-items
            when: "{{steps.validate.outputs.parameters.valid}} == true"

    - name: check-params
      inputs:
        parameters:
          - name: count
      script:
        image: python:3.11
        command: [python]
        source: |
          count = int("{{inputs.parameters.count}}")
          if count > 0 and count <= 100:
              print("true")
          else:
              print("false")
      outputs:
        parameters:
          - name: valid
            valueFrom:
              path: /tmp/result

    - name: process-items
      container:
        image: alpine:latest
        command: [echo, "Processing items"]

Common Patterns

Argo Workflows supports several common orchestration patterns for complex pipeline scenarios.

Key Concepts

• Fan-out: Single task spawning multiple parallel tasks
• Fan-in: Multiple tasks converging to a single task
• Loops: Iterating over lists or sequences
• Recursion: Self-referencing templates
• Retry: Automatic retry on failure

⤢ zoom

Fan-Out Pattern

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: fan-out-
spec:
  entrypoint: main
  arguments:
    parameters:
      - name: items
        value: '["item1", "item2", "item3", "item4", "item5"]'

  templates:
    - name: main
      steps:
        - - name: fan-out
            template: process-item
            arguments:
              parameters:
                - name: item
                  value: "{{item}}"
            withParam: "{{workflow.parameters.items}}"

    - name: process-item
      inputs:
        parameters:
          - name: item
      container:
        image: alpine:latest
        command: [echo, "Processing: {{inputs.parameters.item}}"]

Fan-In Pattern

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: fan-in-
spec:
  entrypoint: main

  templates:
    - name: main
      dag:
        tasks:
          # Fan-out
          - name: process-1
            template: process
            arguments:
              parameters:
                - name: id
                  value: "1"
          - name: process-2
            template: process
            arguments:
              parameters:
                - name: id
                  value: "2"
          - name: process-3
            template: process
            arguments:
              parameters:
                - name: id
                  value: "3"

          # Fan-in: aggregate all results
          - name: aggregate
            template: combine-results
            dependencies: [process-1, process-2, process-3]
            arguments:
              parameters:
                - name: results
                  value: |
                    [
                      "{{tasks.process-1.outputs.parameters.result}}",
                      "{{tasks.process-2.outputs.parameters.result}}",
                      "{{tasks.process-3.outputs.parameters.result}}"
                    ]

    - name: process
      inputs:
        parameters:
          - name: id
      container:
        image: alpine:latest
        command: [sh, -c, "echo 'result-{{inputs.parameters.id}}' > /tmp/result"]
      outputs:
        parameters:
          - name: result
            valueFrom:
              path: /tmp/result

    - name: combine-results
      inputs:
        parameters:
          - name: results
      script:
        image: python:3.11
        command: [python]
        source: |
          import json
          results = json.loads('''{{inputs.parameters.results}}''')
          combined = {'total': len(results), 'items': results}
          print(json.dumps(combined))

Loop Patterns

templates:
  # withItems: iterate over list
  - name: with-items-example
    steps:
      - - name: process
          template: task
          arguments:
            parameters:
              - name: value
                value: "{{item}}"
          withItems:
            - item1
            - item2
            - item3

  # withParam: iterate over JSON array
  - name: with-param-example
    inputs:
      parameters:
        - name: items
    steps:
      - - name: process
          template: task
          arguments:
            parameters:
              - name: value
                value: "{{item}}"
          withParam: "{{inputs.parameters.items}}"

  # withSequence: numeric iteration
  - name: with-sequence-example
    steps:
      - - name: process
          template: task
          arguments:
            parameters:
              - name: index
                value: "{{item}}"
          withSequence:
            count: "10"  # 0-9
            start: "1"   # 1-10
            end: "5"     # 1-5
            format: "%03d"  # 001, 002, etc.

  # Complex item iteration
  - name: complex-items
    steps:
      - - name: deploy
          template: deploy-service
          arguments:
            parameters:
              - name: name
                value: "{{item.name}}"
              - name: version
                value: "{{item.version}}"
              - name: replicas
                value: "{{item.replicas}}"
          withItems:
            - { name: "frontend", version: "1.0", replicas: "3" }
            - { name: "backend", version: "2.1", replicas: "5" }
            - { name: "worker", version: "1.5", replicas: "2" }

Dynamic Fan-Out

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: dynamic-fanout-
spec:
  entrypoint: main

  templates:
    - name: main
      steps:
        # First: generate the list of items
        - - name: generate-items
            template: generate-list

        # Then: fan-out based on dynamic output
        - - name: process
            template: process-item
            arguments:
              parameters:
                - name: item
                  value: "{{item}}"
            withParam: "{{steps.generate-items.outputs.parameters.items}}"

        # Finally: aggregate
        - - name: aggregate
            template: aggregate-results

    - name: generate-list
      script:
        image: python:3.11
        command: [python]
        source: |
          import json
          # Generate dynamic list
          items = [f"item-{i}" for i in range(5)]
          print(json.dumps(items))
      outputs:
        parameters:
          - name: items
            valueFrom:
              path: /tmp/result

    - name: process-item
      inputs:
        parameters:
          - name: item
      container:
        image: alpine:latest
        command: [echo, "{{inputs.parameters.item}}"]

    - name: aggregate-results
      container:
        image: alpine:latest
        command: [echo, "All items processed"]

Retry and Error Handling

templates:
  # Template-level retry
  - name: with-retry
    retryStrategy:
      limit: 3
      retryPolicy: "Always"
      backoff:
        duration: "10s"
        factor: 2
        maxDuration: "5m"
      affinity:
        nodeAntiAffinity: {}  # Retry on different nodes
    container:
      image: alpine:latest
      command: [sh, -c, "exit $((RANDOM % 2))"]  # Random success/failure

  # Conditional retry
  - name: conditional-retry
    retryStrategy:
      limit: 5
      retryPolicy: "OnError"  # Only on error, not failure
      expression: "asInt(lastRetry.exitCode) == 1"  # Retry only on exit code 1
    container:
      image: alpine:latest
      command: [./might-fail.sh]

  # Continue on failure
  - name: allow-failure
    dag:
      tasks:
        - name: might-fail
          template: risky-task
          continueOn:
            failed: true
            error: true

        - name: always-run
          template: cleanup
          dependencies: [might-fail]

Recursion Pattern

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: recursive-
spec:
  entrypoint: main
  arguments:
    parameters:
      - name: depth
        value: "5"

  templates:
    - name: main
      inputs:
        parameters:
          - name: depth
      steps:
        - - name: process
            template: process-level
            arguments:
              parameters:
                - name: level
                  value: "{{inputs.parameters.depth}}"

        - - name: recurse
            template: main
            arguments:
              parameters:
                - name: depth
                  value: "{{=asInt(inputs.parameters.depth) - 1}}"
            when: "{{inputs.parameters.depth}} > 0"

    - name: process-level
      inputs:
        parameters:
          - name: level
      container:
        image: alpine:latest
        command: [echo, "Processing level {{inputs.parameters.level}}"]

Conditional Branching

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: branching-
spec:
  entrypoint: main
  arguments:
    parameters:
      - name: environment
        value: "staging"

  templates:
    - name: main
      steps:
        - - name: determine-path
            template: check-environment

        - - name: staging-deploy
            template: deploy-staging
            when: "{{workflow.parameters.environment}} == staging"

          - name: production-deploy
            template: deploy-production
            when: "{{workflow.parameters.environment}} == production"

          - name: development-deploy
            template: deploy-development
            when: "{{workflow.parameters.environment}} == development"

    - name: check-environment
      container:
        image: alpine:latest
        command: [echo, "Environment: {{workflow.parameters.environment}}"]

    - name: deploy-staging
      container:
        image: kubectl:latest
        command: [echo, "Deploying to staging"]

    - name: deploy-production
      container:
        image: kubectl:latest
        command: [echo, "Deploying to production"]

    - name: deploy-development
      container:
        image: kubectl:latest
        command: [echo, "Deploying to development"]

Examples

Matrix builds (multiple dimensions):

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: matrix-build-
spec:
  entrypoint: main

  templates:
    - name: main
      steps:
        - - name: build
            template: build-version
            arguments:
              parameters:
                - name: os
                  value: "{{item.os}}"
                - name: arch
                  value: "{{item.arch}}"
            withItems:
              - { os: "linux", arch: "amd64" }
              - { os: "linux", arch: "arm64" }
              - { os: "darwin", arch: "amd64" }
              - { os: "darwin", arch: "arm64" }
              - { os: "windows", arch: "amd64" }

    - name: build-version
      inputs:
        parameters:
          - name: os
          - name: arch
      container:
        image: golang:1.21
        command: [sh, -c]
        args:
          - |
            export GOOS={{inputs.parameters.os}}
            export GOARCH={{inputs.parameters.arch}}
            go build -o /output/app-${GOOS}-${GOARCH} .
        env:
          - name: CGO_ENABLED
            value: "0"

Pipeline with approval gate:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: approval-pipeline-
spec:
  entrypoint: main

  templates:
    - name: main
      steps:
        - - name: build
            template: build-app
        - - name: test
            template: run-tests
        - - name: deploy-staging
            template: deploy
            arguments:
              parameters:
                - name: environment
                  value: staging
        - - name: approval
            template: wait-for-approval
        - - name: deploy-production
            template: deploy
            arguments:
              parameters:
                - name: environment
                  value: production

    - name: build-app
      container:
        image: node:18
        command: [npm, run, build]

    - name: run-tests
      container:
        image: node:18
        command: [npm, test]

    - name: deploy
      inputs:
        parameters:
          - name: environment
      container:
        image: kubectl:latest
        command: [kubectl, apply, -f, "{{inputs.parameters.environment}}/"]

    - name: wait-for-approval
      suspend: {}

Quick Reference

Command	Description
argo submit workflow.yaml	Submit a workflow
argo submit -p key=value	Submit with parameter
argo list	List workflows
argo get WORKFLOW	Get workflow details
argo logs WORKFLOW	View workflow logs
argo watch WORKFLOW	Watch workflow progress
argo delete WORKFLOW	Delete workflow
argo retry WORKFLOW	Retry failed workflow
argo resubmit WORKFLOW	Resubmit workflow
argo stop WORKFLOW	Stop workflow gracefully
argo terminate WORKFLOW	Terminate immediately
argo suspend WORKFLOW	Suspend workflow
argo resume WORKFLOW	Resume suspended workflow
argo cron list	List cron workflows
argo template list	List workflow templates
argo cluster-template list	List cluster templates

Useful Flags

Flag	Description
-n, --namespace	Specify namespace
-w, --watch	Watch workflow
--wait	Wait for completion
-o json/yaml/wide	Output format
-l, --selector	Label selector
--node-field-selector	Node field selector
-p, --parameter	Override parameter
-f, --from	Submit from file
--generate-name	Override generateName
--dry-run	Print workflow without submitting

Template Reference Variables

Variable	Description
{{workflow.name}}	Workflow name
{{workflow.namespace}}	Workflow namespace
{{workflow.uid}}	Workflow UID
{{workflow.parameters.NAME}}	Workflow parameter
{{inputs.parameters.NAME}}	Template input parameter
{{inputs.artifacts.NAME}}	Template input artifact
{{outputs.parameters.NAME}}	Template output parameter
{{outputs.artifacts.NAME}}	Template output artifact
{{steps.NAME.outputs.parameters.PARAM}}	Step output parameter
{{tasks.NAME.outputs.parameters.PARAM}}	DAG task output parameter
{{pod.name}}	Pod name
{{retries}}	Current retry count
{{item}}	Current loop item

Workflow Status Values

Status	Description
Pending	Workflow has been submitted
Running	Workflow is executing
Succeeded	Workflow completed successfully
Failed	Workflow failed
Error	Workflow encountered an error
Skipped	Workflow was skipped

Common Issues and Solutions

Issue	Cause	Solution
Pod has unbound immediate PersistentVolumeClaims	PVC not available	Check PVC exists and is bound
ImagePullBackOff	Cannot pull container image	Verify image name and pull secrets
OOMKilled	Container exceeded memory limit	Increase memory limits in template
Deadline exceeded	Workflow timeout	Increase activeDeadlineSeconds
Forbidden: pod would exceed resource quota	Quota exceeded	Reduce parallelism or increase quota
Invalid spec: templates.X not found	Template reference error	Check template name spelling
failed to save outputs: artifact not found	Output path doesn't exist	Verify container creates output files
Workflow in error state	Controller error	Check controller logs
Unable to connect to S3	Artifact repository issue	Verify S3 credentials and endpoint
context deadline exceeded	Network timeout	Check network policies and DNS

Debugging Commands

# View controller logs
kubectl logs -n argo deployment/workflow-controller -f

# View specific pod logs
argo logs WORKFLOW --pod POD_NAME

# Describe workflow
kubectl describe workflow WORKFLOW -n argo

# Get workflow events
kubectl get events --field-selector involvedObject.name=WORKFLOW -n argo

# Check pod status
kubectl get pods -l workflows.argoproj.io/workflow=WORKFLOW -n argo

# Debug with shell
argo exec WORKFLOW --container main -- /bin/sh

Log Analysis

# Get logs for all pods in workflow
argo logs WORKFLOW --all

# Get logs for specific step
argo logs WORKFLOW --step STEP_NAME

# Follow logs
argo logs WORKFLOW -f

# Get logs since time
argo logs WORKFLOW --since 1h

# Filter logs by container
argo logs WORKFLOW --container main

---

Related Topics

The following topics complement Argo Workflows and are commonly used together:

1. Kubernetes - Understanding Kubernetes pods, volumes, and resource management is essential for Argo Workflows
2. ArgoCD - GitOps continuous delivery that can work alongside Argo Workflows for deployment automation
3. Helm - Package manager for Kubernetes, useful for deploying Argo Workflows and dependencies
4. Docker - Container runtime used by workflow tasks
5. CI/CD Patterns - Understanding pipeline patterns helps design effective workflows
6. Prometheus/Grafana - Monitoring workflow execution and performance metrics