Jenkins

Automation server for building, testing, and deploying software through continuous integration and continuous delivery pipelines.

Overview

Jenkins is an open-source automation server that enables developers to reliably build, test, and deploy their software. It provides hundreds of plugins to support building, deploying, and automating any project, with Pipeline as Code being the modern approach to defining CI/CD workflows.

⤢ zoom

Pipeline Syntax

Jenkins supports two pipeline syntaxes: Declarative and Scripted. Declarative is recommended for most use cases due to its simpler structure and built-in validation.

Key Concepts

Aspect	Declarative	Scripted
Structure	Predefined structure with pipeline block	Flexible Groovy-based scripts
Learning curve	Easier to learn	Requires Groovy knowledge
Validation	Built-in syntax validation	Runtime validation only
Flexibility	Limited but sufficient	Full Groovy power
Best for	Most CI/CD workflows	Complex custom logic

Declarative Pipeline

pipeline {
    agent any

    options {
        timeout(time: 30, unit: 'MINUTES')
        buildDiscarder(logRotator(numToKeepStr: '10'))
        disableConcurrentBuilds()
    }

    environment {
        APP_NAME = 'my-application'
        VERSION = '1.0.0'
    }

    stages {
        stage('Build') {
            steps {
                sh 'make build'
            }
        }

        stage('Test') {
            steps {
                sh 'make test'
            }
        }

        stage('Deploy') {
            when {
                branch 'main'
            }
            steps {
                sh 'make deploy'
            }
        }
    }

    post {
        always {
            cleanWs()
        }
        success {
            echo 'Pipeline succeeded!'
        }
        failure {
            echo 'Pipeline failed!'
        }
    }
}

Scripted Pipeline

node {
    try {
        stage('Checkout') {
            checkout scm
        }

        stage('Build') {
            sh 'make build'
        }

        stage('Test') {
            sh 'make test'
        }

        if (env.BRANCH_NAME == 'main') {
            stage('Deploy') {
                sh 'make deploy'
            }
        }

    } catch (Exception e) {
        currentBuild.result = 'FAILURE'
        throw e
    } finally {
        cleanWs()
    }
}

Agent Configuration

// Run on any available agent
agent any

// Run on agent with specific label
agent {
    label 'linux && docker'
}

// Run in Docker container
agent {
    docker {
        image 'node:18-alpine'
        args '-v /tmp:/tmp'
    }
}

// Run in Kubernetes pod
agent {
    kubernetes {
        yaml '''
        apiVersion: v1
        kind: Pod
        spec:
          containers:
          - name: maven
            image: maven:3.8-openjdk-17
            command:
            - cat
            tty: true
        '''
    }
}

// No agent for pipeline, define per stage
agent none

Jenkinsfile Structure and Best Practices

Key Concepts

The Jenkinsfile is a text file that contains the definition of a Jenkins Pipeline and is checked into source control. This enables Pipeline as Code, providing versioning, review, and audit trail for your CI/CD configuration.

⤢ zoom

Recommended Structure

#!/usr/bin/env groovy

// Define shared library (if using)
@Library('my-shared-library') _

pipeline {
    agent any

    // Pipeline-level options
    options {
        timestamps()
        ansiColor('xterm')
        timeout(time: 1, unit: 'HOURS')
        buildDiscarder(logRotator(
            numToKeepStr: '10',
            artifactNumToKeepStr: '5'
        ))
    }

    // Build parameters
    parameters {
        string(name: 'DEPLOY_ENV', defaultValue: 'staging', description: 'Target environment')
        booleanParam(name: 'RUN_TESTS', defaultValue: true, description: 'Run test suite')
        choice(name: 'LOG_LEVEL', choices: ['INFO', 'DEBUG', 'WARN'], description: 'Logging level')
    }

    // Environment variables
    environment {
        APP_NAME = 'my-app'
        REGISTRY = 'docker.io/myorg'
    }

    // Triggers
    triggers {
        pollSCM('H/5 * * * *')
        cron('H 2 * * 1-5')
    }

    stages {
        // Stage definitions
    }

    post {
        // Post actions
    }
}

Best Practices

// 1. Use meaningful stage names
stage('Build Application') {
    steps {
        sh './gradlew build'
    }
}

// 2. Keep stages focused and atomic
stage('Unit Tests') {
    steps {
        sh './gradlew test'
    }
    post {
        always {
            junit 'build/test-results/**/*.xml'
        }
    }
}

// 3. Use input for manual approval gates
stage('Deploy to Production') {
    steps {
        input message: 'Deploy to production?', ok: 'Deploy'
        sh './deploy.sh production'
    }
}

// 4. Fail fast with proper error handling
stage('Quality Gate') {
    steps {
        script {
            def qg = waitForQualityGate()
            if (qg.status != 'OK') {
                error "Pipeline aborted due to quality gate failure: ${qg.status}"
            }
        }
    }
}

// 5. Use stash/unstash for passing files between agents
stage('Build') {
    agent { label 'builder' }
    steps {
        sh 'make build'
        stash includes: 'dist/**', name: 'build-artefacts'
    }
}

stage('Deploy') {
    agent { label 'deployer' }
    steps {
        unstash 'build-artefacts'
        sh './deploy.sh'
    }
}

Common Pipeline Stages

Key Concepts

A typical CI/CD pipeline consists of sequential stages that validate, build, test, and deploy code. Each stage should have a single responsibility and clear success/failure criteria.

⤢ zoom

Complete Pipeline Example

pipeline {
    agent any

    environment {
        DOCKER_REGISTRY = 'docker.io/myorg'
        IMAGE_NAME = 'my-application'
        IMAGE_TAG = "${env.BUILD_NUMBER}"
    }

    stages {
        stage('Checkout') {
            steps {
                checkout scm

                script {
                    env.GIT_COMMIT_SHORT = sh(
                        script: 'git rev-parse --short HEAD',
                        returnStdout: true
                    ).trim()
                }
            }
        }

        stage('Build') {
            steps {
                sh '''
                    echo "Building application..."
                    ./gradlew clean build -x test
                '''
            }
        }

        stage('Unit Tests') {
            steps {
                sh './gradlew test'
            }
            post {
                always {
                    junit 'build/test-results/**/*.xml'
                    publishHTML([
                        allowMissing: false,
                        alwaysLinkToLastBuild: true,
                        keepAll: true,
                        reportDir: 'build/reports/tests/test',
                        reportFiles: 'index.html',
                        reportName: 'Unit Test Report'
                    ])
                }
            }
        }

        stage('Integration Tests') {
            steps {
                sh './gradlew integrationTest'
            }
        }

        stage('Code Quality') {
            parallel {
                stage('SonarQube Analysis') {
                    steps {
                        withSonarQubeEnv('SonarQube') {
                            sh './gradlew sonarqube'
                        }
                    }
                }

                stage('Security Scan') {
                    steps {
                        sh 'trivy fs --severity HIGH,CRITICAL .'
                    }
                }
            }
        }

        stage('Build Docker Image') {
            steps {
                sh """
                    docker build -t ${DOCKER_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG} .
                    docker tag ${DOCKER_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG} \
                        ${DOCKER_REGISTRY}/${IMAGE_NAME}:latest
                """
            }
        }

        stage('Push to Registry') {
            steps {
                withCredentials([usernamePassword(
                    credentialsId: 'docker-registry-creds',
                    usernameVariable: 'DOCKER_USER',
                    passwordVariable: 'DOCKER_PASS'
                )]) {
                    sh '''
                        echo $DOCKER_PASS | docker login -u $DOCKER_USER --password-stdin
                        docker push ${DOCKER_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}
                        docker push ${DOCKER_REGISTRY}/${IMAGE_NAME}:latest
                    '''
                }
            }
        }

        stage('Deploy to Staging') {
            steps {
                sh './scripts/deploy.sh staging ${IMAGE_TAG}'
            }
        }

        stage('Smoke Tests') {
            steps {
                sh './scripts/smoke-test.sh staging'
            }
        }

        stage('Deploy to Production') {
            when {
                branch 'main'
            }
            steps {
                input message: 'Deploy to production?', ok: 'Deploy'
                sh './scripts/deploy.sh production ${IMAGE_TAG}'
            }
        }
    }
}

Environment Variables and Credentials Management

Key Concepts

Jenkins provides several ways to manage environment variables and sensitive credentials. Always use the credentials store for secrets rather than hardcoding them in pipelines.

Built-in Environment Variables

pipeline {
    agent any

    stages {
        stage('Show Environment') {
            steps {
                echo "Build Number: ${env.BUILD_NUMBER}"
                echo "Build ID: ${env.BUILD_ID}"
                echo "Build URL: ${env.BUILD_URL}"
                echo "Job Name: ${env.JOB_NAME}"
                echo "Branch: ${env.BRANCH_NAME}"
                echo "Workspace: ${env.WORKSPACE}"
                echo "Jenkins URL: ${env.JENKINS_URL}"
                echo "Node Name: ${env.NODE_NAME}"
                echo "Executor Number: ${env.EXECUTOR_NUMBER}"

                // Git-specific (with Git plugin)
                echo "Git Commit: ${env.GIT_COMMIT}"
                echo "Git Branch: ${env.GIT_BRANCH}"
                echo "Git URL: ${env.GIT_URL}"
            }
        }
    }
}

Custom Environment Variables

pipeline {
    agent any

    // Pipeline-level variables
    environment {
        APP_NAME = 'my-application'
        VERSION = '1.0.0'

        // Credentials reference
        AWS_ACCESS_KEY_ID = credentials('aws-access-key')

        // Dynamic values
        BUILD_TIMESTAMP = sh(script: 'date +%Y%m%d%H%M%S', returnStdout: true).trim()
    }

    stages {
        stage('Stage Variables') {
            // Stage-level variables
            environment {
                STAGE_VAR = 'only-available-in-this-stage'
                DYNAMIC_VAR = "${env.APP_NAME}-${env.VERSION}"
            }
            steps {
                echo "Stage var: ${env.STAGE_VAR}"
            }
        }

        stage('Script Block Variables') {
            steps {
                script {
                    // Set variables dynamically
                    env.CUSTOM_VAR = 'custom-value'

                    // Conditional variable
                    if (env.BRANCH_NAME == 'main') {
                        env.DEPLOY_ENV = 'production'
                    } else {
                        env.DEPLOY_ENV = 'staging'
                    }
                }
                echo "Deploy env: ${env.DEPLOY_ENV}"
            }
        }
    }
}

Credentials Management

pipeline {
    agent any

    environment {
        // Simple secret text
        API_KEY = credentials('api-key-credential-id')

        // Username/password - creates VAR, VAR_USR, VAR_PSW
        DOCKER_CREDS = credentials('docker-credentials')
    }

    stages {
        stage('Using Credentials') {
            steps {
                // Username/password in environment
                sh '''
                    echo "User: $DOCKER_CREDS_USR"
                    docker login -u $DOCKER_CREDS_USR -p $DOCKER_CREDS_PSW
                '''

                // WithCredentials block for scoped access
                withCredentials([
                    string(credentialsId: 'github-token', variable: 'GITHUB_TOKEN'),
                    usernamePassword(
                        credentialsId: 'nexus-creds',
                        usernameVariable: 'NEXUS_USER',
                        passwordVariable: 'NEXUS_PASS'
                    ),
                    file(credentialsId: 'kubeconfig', variable: 'KUBECONFIG'),
                    sshUserPrivateKey(
                        credentialsId: 'ssh-key',
                        keyFileVariable: 'SSH_KEY',
                        usernameVariable: 'SSH_USER'
                    ),
                    certificate(
                        credentialsId: 'client-cert',
                        keystoreVariable: 'KEYSTORE',
                        passwordVariable: 'KEYSTORE_PASS'
                    )
                ]) {
                    sh '''
                        curl -H "Authorization: token $GITHUB_TOKEN" https://api.github.com/user
                        kubectl --kubeconfig=$KUBECONFIG get pods
                    '''
                }
            }
        }

        stage('AWS Credentials') {
            steps {
                withAWS(credentials: 'aws-credentials', region: 'eu-west-1') {
                    sh 'aws s3 ls'
                }
            }
        }
    }
}

Credentials Types

Type	Use Case	Access Pattern
Secret text	API keys, tokens	${VAR}
Username with password	Registry, database	${VAR_USR}, ${VAR_PSW}
Secret file	Kubeconfig, certificates	File path in ${VAR}
SSH Username with private key	Git, server access	Key file path
Certificate	Client certificates	Keystore file and password

Parallel Execution and Matrix Builds

Key Concepts

Parallel execution runs multiple stages simultaneously, reducing pipeline duration. Matrix builds test multiple combinations of parameters automatically.

⤢ zoom

Parallel Stages

pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                sh 'make build'
            }
        }

        stage('Parallel Tests') {
            parallel {
                stage('Unit Tests') {
                    steps {
                        sh 'make unit-test'
                    }
                }

                stage('Integration Tests') {
                    steps {
                        sh 'make integration-test'
                    }
                }

                stage('Lint') {
                    steps {
                        sh 'make lint'
                    }
                }

                stage('Security Scan') {
                    steps {
                        sh 'make security-scan'
                    }
                }
            }
        }

        stage('Deploy') {
            steps {
                sh 'make deploy'
            }
        }
    }
}

Parallel with Different Agents

pipeline {
    agent none

    stages {
        stage('Multi-Platform Build') {
            parallel {
                stage('Linux Build') {
                    agent { label 'linux' }
                    steps {
                        sh 'make build-linux'
                        stash includes: 'dist/linux/**', name: 'linux-build'
                    }
                }

                stage('Windows Build') {
                    agent { label 'windows' }
                    steps {
                        bat 'make build-windows'
                        stash includes: 'dist/windows/**', name: 'windows-build'
                    }
                }

                stage('macOS Build') {
                    agent { label 'macos' }
                    steps {
                        sh 'make build-macos'
                        stash includes: 'dist/macos/**', name: 'macos-build'
                    }
                }
            }
        }

        stage('Package') {
            agent any
            steps {
                unstash 'linux-build'
                unstash 'windows-build'
                unstash 'macos-build'
                sh './package-all.sh'
            }
        }
    }
}

Matrix Builds

pipeline {
    agent any

    stages {
        stage('Matrix Build') {
            matrix {
                axes {
                    axis {
                        name 'PLATFORM'
                        values 'linux', 'windows', 'macos'
                    }
                    axis {
                        name 'NODE_VERSION'
                        values '16', '18', '20'
                    }
                }

                excludes {
                    // Exclude specific combinations
                    exclude {
                        axis {
                            name 'PLATFORM'
                            values 'windows'
                        }
                        axis {
                            name 'NODE_VERSION'
                            values '16'
                        }
                    }
                }

                stages {
                    stage('Build') {
                        steps {
                            echo "Building on ${PLATFORM} with Node ${NODE_VERSION}"
                            sh "nvm use ${NODE_VERSION} && npm ci && npm run build"
                        }
                    }

                    stage('Test') {
                        steps {
                            sh "npm test"
                        }
                    }
                }
            }
        }
    }
}

Matrix with Agent per Cell

pipeline {
    agent none

    stages {
        stage('Browser Testing') {
            matrix {
                agent {
                    label "${BROWSER}-agent"
                }

                axes {
                    axis {
                        name 'BROWSER'
                        values 'chrome', 'firefox', 'safari'
                    }
                    axis {
                        name 'OS'
                        values 'linux', 'macos'
                    }
                }

                stages {
                    stage('Test') {
                        steps {
                            sh "npm run e2e -- --browser=${BROWSER}"
                        }
                    }
                }
            }
        }
    }
}

Controlling Parallelism

pipeline {
    agent any

    options {
        // Limit concurrent matrix cells
        parallelsAlwaysFailFast()
    }

    stages {
        stage('Parallel with Limits') {
            parallel {
                stage('Task 1') {
                    options {
                        timeout(time: 10, unit: 'MINUTES')
                    }
                    steps {
                        sh './task1.sh'
                    }
                }

                stage('Task 2') {
                    steps {
                        sh './task2.sh'
                    }
                }
            }

            // Fail entire parallel block if any stage fails
            failFast true
        }
    }
}

Post Actions

Key Concepts

Post actions execute after stages or the entire pipeline completes. They handle cleanup, notifications, and artefact archiving based on build outcome.

⤢ zoom

Post Action Conditions

Condition	When It Runs
always	Always, regardless of outcome
success	Only if build succeeded
failure	Only if build failed
unstable	Only if build is unstable (e.g., test failures)
changed	Only if build status changed from previous
fixed	Only if build was previously failing and now succeeds
regression	Only if build was previously successful and now fails
aborted	Only if build was manually aborted
cleanup	Always runs last, after all other post conditions

Comprehensive Post Actions

pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                sh 'make build'
            }
            post {
                // Stage-specific post actions
                success {
                    stash includes: 'dist/**', name: 'build-output'
                }
            }
        }

        stage('Test') {
            steps {
                sh 'make test'
            }
            post {
                always {
                    junit 'test-results/**/*.xml'
                }
            }
        }
    }

    post {
        always {
            // Archive artefacts
            archiveArtifacts artifacts: 'dist/**', fingerprint: true

            // Publish test results
            junit allowEmptyResults: true, testResults: '**/test-results/*.xml'

            // Publish HTML reports
            publishHTML([
                allowMissing: false,
                alwaysLinkToLastBuild: true,
                keepAll: true,
                reportDir: 'coverage',
                reportFiles: 'index.html',
                reportName: 'Coverage Report'
            ])
        }

        success {
            echo 'Build succeeded!'

            // Send success notification
            slackSend(
                channel: '#builds',
                color: 'good',
                message: "Build Succeeded: ${env.JOB_NAME} #${env.BUILD_NUMBER}"
            )

            // Tag successful builds
            script {
                if (env.BRANCH_NAME == 'main') {
                    sh "git tag -a v${env.BUILD_NUMBER} -m 'Release ${env.BUILD_NUMBER}'"
                    sh "git push origin v${env.BUILD_NUMBER}"
                }
            }
        }

        failure {
            echo 'Build failed!'

            // Send failure notification
            slackSend(
                channel: '#builds',
                color: 'danger',
                message: "Build Failed: ${env.JOB_NAME} #${env.BUILD_NUMBER}\n${env.BUILD_URL}"
            )

            // Email notification
            emailext(
                subject: "FAILED: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'",
                body: """<p>FAILED: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]':</p>
                         <p>Check console output at <a href='${env.BUILD_URL}'>${env.BUILD_URL}</a></p>""",
                recipientProviders: [
                    [$class: 'DevelopersRecipientProvider'],
                    [$class: 'CulpritsRecipientProvider']
                ]
            )
        }

        unstable {
            echo 'Build is unstable!'

            slackSend(
                channel: '#builds',
                color: 'warning',
                message: "Build Unstable: ${env.JOB_NAME} #${env.BUILD_NUMBER}"
            )
        }

        changed {
            echo 'Build status changed from previous run'
        }

        fixed {
            echo 'Build is back to normal!'

            slackSend(
                channel: '#builds',
                color: 'good',
                message: "Build Fixed: ${env.JOB_NAME} #${env.BUILD_NUMBER}"
            )
        }

        regression {
            echo 'Build status regressed!'
        }

        aborted {
            echo 'Build was aborted'
        }

        cleanup {
            // Always runs last
            cleanWs()

            // Remove Docker images
            sh 'docker system prune -f || true'
        }
    }
}

Shared Libraries

Key Concepts

Shared libraries allow you to define reusable pipeline code that can be shared across multiple projects. They promote DRY principles and standardise CI/CD practices across an organisation.

⤢ zoom

Library Structure

jenkins-shared-library/
├── vars/
│   ├── buildApp.groovy        # Global variables/functions
│   ├── deployApp.groovy
│   └── sendNotification.groovy
├── src/
│   └── org/
│       └── company/
│           ├── Utils.groovy    # Helper classes
│           └── Docker.groovy
├── resources/
│   └── templates/
│       └── deployment.yaml
└── README.md

Creating Library Functions (vars/)

// vars/buildApp.groovy
def call(Map config = [:]) {
    def appName = config.appName ?: 'default-app'
    def buildTool = config.buildTool ?: 'maven'

    pipeline {
        agent any

        stages {
            stage('Build') {
                steps {
                    script {
                        switch(buildTool) {
                            case 'maven':
                                sh 'mvn clean package'
                                break
                            case 'gradle':
                                sh './gradlew build'
                                break
                            case 'npm':
                                sh 'npm ci && npm run build'
                                break
                        }
                    }
                }
            }
        }
    }
}

// vars/deployApp.groovy
def call(String environment, String version) {
    echo "Deploying version ${version} to ${environment}"

    withCredentials([file(credentialsId: 'kubeconfig', variable: 'KUBECONFIG')]) {
        sh """
            kubectl --kubeconfig=\$KUBECONFIG set image deployment/app \
                app=myregistry/app:${version} \
                -n ${environment}
        """
    }
}

// vars/sendNotification.groovy
def call(String status, Map config = [:]) {
    def channel = config.channel ?: '#builds'
    def colour = status == 'success' ? 'good' : 'danger'

    slackSend(
        channel: channel,
        color: colour,
        message: "${status.toUpperCase()}: ${env.JOB_NAME} #${env.BUILD_NUMBER}"
    )
}

Creating Classes (src/)

// src/org/company/Docker.groovy
package org.company

class Docker implements Serializable {
    def script

    Docker(script) {
        this.script = script
    }

    def build(String imageName, String tag = 'latest') {
        script.sh "docker build -t ${imageName}:${tag} ."
    }

    def push(String imageName, String tag = 'latest', String registry) {
        script.withCredentials([script.usernamePassword(
            credentialsId: 'docker-creds',
            usernameVariable: 'USER',
            passwordVariable: 'PASS'
        )]) {
            script.sh """
                echo \$PASS | docker login -u \$USER --password-stdin ${registry}
                docker push ${registry}/${imageName}:${tag}
            """
        }
    }
}

Using Shared Libraries

// Method 1: Global library (configured in Jenkins)
@Library('my-shared-library') _

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                buildApp(appName: 'my-service', buildTool: 'gradle')
            }
        }
        stage('Deploy') {
            steps {
                deployApp('staging', env.BUILD_NUMBER)
            }
        }
    }
    post {
        success {
            sendNotification('success')
        }
        failure {
            sendNotification('failure')
        }
    }
}

// Method 2: Specific version
@Library('my-shared-library@v1.0.0') _

// Method 3: Dynamic library loading
library 'my-shared-library@main'

// Method 4: Using classes
@Library('my-shared-library') _
import org.company.Docker

pipeline {
    agent any
    stages {
        stage('Build & Push') {
            steps {
                script {
                    def docker = new Docker(this)
                    docker.build('my-app', env.BUILD_NUMBER)
                    docker.push('my-app', env.BUILD_NUMBER, 'docker.io/myorg')
                }
            }
        }
    }
}

Loading Resources

// vars/deployWithTemplate.groovy
def call(String appName, String namespace) {
    // Load template from resources
    def template = libraryResource 'templates/deployment.yaml'

    // Replace placeholders
    def manifest = template
        .replace('{{APP_NAME}}', appName)
        .replace('{{NAMESPACE}}', namespace)

    // Write and apply
    writeFile file: 'deployment.yaml', text: manifest
    sh "kubectl apply -f deployment.yaml"
}

Plugin Management

Key Concepts

Jenkins plugins extend functionality for source control, build tools, notifications, and integrations. Managing plugins properly ensures stability and security.

Essential Plugins

Category	Plugin	Purpose
Pipeline	Pipeline	Core pipeline functionality
Pipeline	Pipeline: Stage View	Visualise pipeline stages
Pipeline	Blue Ocean	Modern pipeline UI
SCM	Git	Git integration
SCM	GitHub	GitHub integration
Build	Docker Pipeline	Docker support in pipelines
Build	Kubernetes	K8s agent provisioning
Credentials	Credentials Binding	Inject credentials
Credentials	HashiCorp Vault	Vault integration
Notifications	Slack Notification	Slack messages
Notifications	Email Extension	Advanced email
Quality	JUnit	Test results
Quality	Code Coverage API	Coverage reports
Quality	SonarQube Scanner	Code analysis
Utils	Timestamper	Add timestamps to logs
Utils	AnsiColor	Coloured console output
Utils	Workspace Cleanup	Clean workspace

Managing Plugins via CLI

# List installed plugins
java -jar jenkins-cli.jar -s http://localhost:8080 list-plugins

# Install plugins
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin \
    git workflow-aggregator docker-workflow kubernetes

# Install plugin with dependencies
java -jar jenkins-cli.jar -s http://localhost:8080 install-plugin \
    blueocean -deploy

# Restart Jenkins safely
java -jar jenkins-cli.jar -s http://localhost:8080 safe-restart

Plugin Configuration as Code

# jenkins.yaml (Configuration as Code)
jenkins:
  systemMessage: "Jenkins configured as code"

unclassified:
  slackNotifier:
    teamDomain: "myteam"
    tokenCredentialId: "slack-token"
    room: "#jenkins"

  gitHubPluginConfig:
    configs:
      - name: "GitHub"
        apiUrl: "https://api.github.com"
        credentialsId: "github-token"
        manageHooks: true

tool:
  git:
    installations:
      - name: "Default"
        home: "git"

  maven:
    installations:
      - name: "Maven 3"
        properties:
          - installSource:
              installers:
                - maven:
                    id: "3.8.6"

plugins.txt for Docker

# plugins.txt - Install during Docker build
git:latest
workflow-aggregator:latest
docker-workflow:latest
kubernetes:latest
credentials-binding:latest
pipeline-stage-view:latest
blueocean:latest
slack:latest
junit:latest
htmlpublisher:latest
timestamper:latest
ansicolor:latest
ws-cleanup:latest

# Dockerfile
FROM jenkins/jenkins:lts
COPY plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN jenkins-plugin-cli --plugin-file /usr/share/jenkins/ref/plugins.txt

Troubleshooting Build Failures

Key Concepts

Effective troubleshooting requires systematic analysis of logs, environment state, and pipeline configuration. Jenkins provides multiple tools for diagnosing issues.

Common Issues and Solutions

Issue	Symptoms	Solution
Agent not available	Build queued indefinitely	Check agent status, labels, and connectivity
Workspace permission	Permission denied errors	Verify agent user permissions
Credential access	Credential not found	Check credential scope and ID
Timeout	Build aborted after timeout	Increase timeout or optimise steps
Memory issues	OutOfMemoryError	Increase heap size, check for leaks
Plugin conflicts	Random failures after update	Check plugin compatibility
SCM checkout fails	Could not clone	Verify credentials and network access
Docker socket	Cannot connect to Docker	Mount Docker socket, check permissions

Debug Techniques

pipeline {
    agent any

    options {
        // Add timestamps to console output
        timestamps()

        // Coloured output
        ansiColor('xterm')
    }

    stages {
        stage('Debug Environment') {
            steps {
                // Print all environment variables
                sh 'printenv | sort'

                // Show current directory and contents
                sh 'pwd && ls -la'

                // Check available tools
                sh '''
                    echo "=== Tool Versions ==="
                    java -version || true
                    mvn -version || true
                    node --version || true
                    docker --version || true
                    kubectl version --client || true
                '''

                // Show agent information
                echo "Node: ${env.NODE_NAME}"
                echo "Workspace: ${env.WORKSPACE}"
                echo "Build URL: ${env.BUILD_URL}"
            }
        }

        stage('Debug Credentials') {
            steps {
                // Test credential access (don't print values!)
                withCredentials([string(credentialsId: 'my-secret', variable: 'SECRET')]) {
                    sh 'if [ -n "$SECRET" ]; then echo "Credential loaded"; else echo "Credential empty"; fi'
                }
            }
        }

        stage('Debug Network') {
            steps {
                // Test connectivity
                sh '''
                    echo "=== Network Debug ==="
                    curl -sI https://github.com | head -5 || echo "Cannot reach GitHub"
                    curl -sI https://registry.hub.docker.com | head -5 || echo "Cannot reach Docker Hub"
                '''
            }
        }
    }
}

Accessing Logs

// Pipeline Replay - Modify and re-run failed pipeline
// Access via: Build > Replay

// Console Output - Full build log
// Access via: Build > Console Output

// Pipeline Steps - Detailed step breakdown
// Access via: Build > Pipeline Steps

// Blue Ocean - Visual pipeline view
// Access via: Open Blue Ocean

// Workspaces - Inspect build workspace
// Access via: Build > Workspaces

Scripted Debugging

pipeline {
    agent any

    stages {
        stage('Retry on Failure') {
            steps {
                retry(3) {
                    sh './flaky-script.sh'
                }
            }
        }

        stage('Wait for Condition') {
            steps {
                waitUntil {
                    script {
                        def response = sh(
                            script: 'curl -s -o /dev/null -w "%{http_code}" http://app:8080/health',
                            returnStdout: true
                        ).trim()
                        return response == '200'
                    }
                }
            }
        }

        stage('Catch and Handle') {
            steps {
                script {
                    try {
                        sh './might-fail.sh'
                    } catch (Exception e) {
                        echo "Step failed: ${e.message}"
                        // Collect debug info
                        sh 'dmesg | tail -50 || true'
                        sh 'docker logs app || true'
                        throw e
                    }
                }
            }
        }

        stage('Interactive Debug') {
            steps {
                // Pause for manual investigation
                input message: 'Paused for debugging. Continue?', ok: 'Continue'
            }
        }
    }
}

Log Parser Configuration

pipeline {
    agent any

    stages {
        stage('Build') {
            steps {
                sh 'make build 2>&1 | tee build.log'
            }
            post {
                always {
                    // Parse logs for errors
                    script {
                        def log = readFile('build.log')
                        if (log.contains('error:') || log.contains('FAILED')) {
                            unstable('Build has warnings or errors')
                        }
                    }

                    // Archive logs for analysis
                    archiveArtifacts artifacts: '*.log', allowEmptyArchive: true
                }
            }
        }
    }
}

Jenkins System Logs

# View Jenkins logs
tail -f /var/log/jenkins/jenkins.log

# Docker-based Jenkins
docker logs -f jenkins

# Kubernetes-based Jenkins
kubectl logs -f deployment/jenkins -n jenkins

# Check system resources
free -h
df -h
top -b -n 1 | head -20

Quick Reference

Pipeline Syntax

// Declarative structure
pipeline {
    agent any
    options { ... }
    environment { ... }
    parameters { ... }
    triggers { ... }
    stages { ... }
    post { ... }
}

// Common steps
sh 'command'                    // Shell command
bat 'command'                   // Windows batch
echo 'message'                  // Print message
dir('path') { ... }             // Change directory
deleteDir()                     // Delete workspace
writeFile file: 'f', text: 't'  // Write file
readFile 'file'                 // Read file
stash/unstash                   // Pass files between agents
archiveArtifacts                // Save artefacts

Conditionals

// When conditions
when {
    branch 'main'
    branch pattern: 'release/*'
    environment name: 'ENV', value: 'prod'
    expression { return params.RUN_TESTS }
    allOf { branch 'main'; environment name: 'X', value: 'Y' }
    anyOf { branch 'main'; branch 'develop' }
    not { branch 'main' }
    triggeredBy 'TimerTrigger'
    changeset '**/*.js'
}

Useful Groovy Snippets

// Get build user
wrap([$class: 'BuildUser']) {
    echo "Build triggered by: ${env.BUILD_USER}"
}

// Read JSON
def data = readJSON file: 'data.json'
def data = readJSON text: '{"key": "value"}'

// Write JSON
writeJSON file: 'out.json', json: [key: 'value']

// HTTP request
def response = httpRequest 'https://api.example.com/data'
echo "Status: ${response.status}"

// Parse shell output
def version = sh(script: 'cat VERSION', returnStdout: true).trim()

// Return status code
def status = sh(script: './check.sh', returnStatus: true)
if (status != 0) { error 'Check failed' }

Common Issues and Solutions

Problem	Cause	Solution
Scripts not permitted to use method	Script security sandbox	Approve in Manage Jenkins > In-process Script Approval
No such DSL method	Missing plugin	Install required plugin
Cannot run program: Permission denied	File not executable	Add chmod +x script.sh before execution
Couldn't find any revision to build	Branch not found	Check SCM configuration and branch name
ERROR: script returned exit code 1	Command failed	Check command output, use set -e for debugging
java.io.NotSerializableException	Non-serializable in CPS	Use @NonCPS annotation or serializable types
hudson.AbortException: No space left	Disk full	Clean workspace, prune Docker images
JENKINS_NODE_COOKIE error	Process killed on build end	Set JENKINS_NODE_COOKIE=dontKillMe
Waiting for next available executor	No agents available	Check agent status and resource availability
SSL peer shut down incorrectly	SSL/TLS issues	Update certificates, check network

Security Hardening

// Avoid shell injection
def userInput = params.USER_INPUT
// Bad: sh "echo ${userInput}"
// Good:
sh """
    echo '${userInput.replace("'", "'\\''")}'
"""

// Or use environment variables
withEnv(["SAFE_INPUT=${userInput}"]) {
    sh 'echo "$SAFE_INPUT"'
}

// Avoid credential exposure
// Bad: echo "Token: ${env.API_TOKEN}"
// Good: check without printing
sh '''
    if [ -z "$API_TOKEN" ]; then
        echo "Token not set"
        exit 1
    fi
'''

---

Related Topics

The following topics complement Jenkins and would enhance your CI/CD knowledge:

1. ArgoCD - GitOps continuous delivery for Kubernetes, pairs well with Jenkins for building and ArgoCD for deploying
2. Docker - Container building and management, essential for modern Jenkins pipelines
3. Kubernetes - Container orchestration where Jenkins can run as agents and deploy applications
4. GitHub Actions - Alternative CI/CD platform, useful to understand for comparison and migration
5. Helm - Kubernetes package manager, commonly used in Jenkins deployment stages
6. CI/CD Patterns - Best practices for pipeline design, branching strategies, and deployment patterns