Unit 6: CI/CD with Jenkins

1. Jenkins Foundations

Jenkins Architecture (Master/Agent Model)

Jenkins operates on a distributed architecture consisting of a Master (Controller) and multiple Agents (Nodes).

• Jenkins Master (Controller): The central control unit. It handles HTTP requests, serves the UI, manages plugins, schedules builds, and distributes workloads to agents. The Master should ideally only coordinate tasks, not run heavy builds, to avoid performance bottlenecks.
• Jenkins Agent (Node): A machine (physical, virtual, or container) that executes the build tasks dispatched by the Master. Agents can run on various operating systems (Linux, Windows, macOS) allowing Jenkins to build and test code across different environments.
• Communication: Master and Agents communicate via TCP/IP. Agents can be connected via SSH, Java Web Start (JNLP), or dynamically provisioned as Docker/Kubernetes containers.

Installation & UI Overview

• Installation: Jenkins requires a Java Runtime Environment (JRE/JDK). It can be installed natively (via apt or yum), run as a standalone .war file (java -jar jenkins.war), or deployed as a Docker container (docker run -p 8080:8080 jenkins/jenkins).
• UI Overview: The default dashboard provides a summary of all jobs (pipelines/projects). Key navigation elements include:
  • New Item: Create new jobs (Freestyle, Pipeline, Multibranch, etc.).
  • Manage Jenkins: The central configuration hub (plugins, system settings, global tools, nodes).
  • Build History: A timeline of past and currently running builds.
  • Views: Custom dashboards to organize jobs logically (e.g., by project or team).

Plugins Management

Jenkins’ core functionality is minimalistic; its power lies in its plugin ecosystem (1,800+ plugins).

• Accessed via Manage Jenkins -> Manage Plugins.
• Categories: Available, Installed, Advanced, Updates.
• Essential Plugins: Git, Pipeline, Docker, Credentials Binding, Maven Integration, Role-based Authorization Strategy.
• Note: Keep plugins updated to patch security vulnerabilities and ensure compatibility.

Security, Users, and Roles

• Authentication: Jenkins can authenticate users via its internal database, LDAP, Active Directory, or OAuth (e.g., GitHub authentication).
• Authorization: Managed via the Matrix-based security or the Role-Based Access Control (RBAC) plugin.
  • Global Roles: Administrator, Read-only.
  • Project Roles: Grants access to specific jobs using regex matching (e.g., dev-.*).
• Credentials: Stored securely in Jenkins to interact with external systems (Git, Docker Hub, AWS). Types include Username with password, SSH Username with private key, Secret text, and Secret file.

2. Jenkins Pipelines

Freestyle vs. Pipeline Jobs

• Freestyle Jobs: Configured entirely through the UI. Easy to set up for simple tasks but hard to version control, scale, or manage complex workflows.
• Pipeline Jobs: Defined using code (Infrastructure as Code paradigm). Supports complex deployment workflows, loops, conditional logic, and survives Jenkins restarts.

Declarative vs. Scripted Pipeline Syntax

• Declarative Pipeline: A newer, structured, and more restrictive syntax. It uses predefined blocks (pipeline, agent, stages, steps). It is easier to read and recommended for most users.
• Scripted Pipeline: Older syntax based heavily on Groovy. It executes sequentially from top to bottom and offers maximum flexibility for complex programmatic logic. Uses node blocks instead of pipeline.

Jenkinsfile Structure

A Jenkinsfile is a text file that contains the definition of a Jenkins Pipeline and is checked into source control.

Declarative Example:

pipeline {
    agent any
    parameters {
        string(name: 'ENV', defaultValue: 'dev', description: 'Deployment Environment')
    }
    environment {
        APP_VERSION = "1.0.${BUILD_NUMBER}"
    }
    stages {
        stage('Build') {
            steps {
                echo "Building version ${APP_VERSION} for ${params.ENV}..."
            }
        }
    }
}

Parameters & Environment Variables

• Parameters: Allow builds to accept user input (e.g., String, Boolean, Choice, Password). Accessed via params.PARAM_NAME.
• Environment Variables: Accessible via env.VAR_NAME. Jenkins provides default variables (e.g., BUILD_NUMBER, WORKSPACE, BRANCH_NAME). Custom variables are defined in the environment block.

Jenkins Multi-branch Pipelines

• Automatically creates a Jenkins Pipeline for each branch in a source control repository that contains a Jenkinsfile.
• Ideal for feature-branch development. If a developer pushes a new branch feature-X with a Jenkinsfile, Jenkins automatically detects it and triggers a build.

3. Pipeline Stages: Standard CI/CD Flow

A standard pipeline consists of logical stages representing the software delivery lifecycle.

1. Checkout Code from Git

Retrieves the source code from the repository. In Declarative pipelines, this is often implicitly handled, but can be explicitly defined:

stage('Checkout') {
    steps {
        git branch: 'main', credentialsId: 'github-creds', url: 'https://github.com/user/repo.git'
    }
}

2. Build

Compiles source code and resolves dependencies. (e.g., mvn clean compile, npm run build).

3. Test

Executes unit and integration tests. Failure in this stage typically halts the pipeline to prevent broken code from proceeding.

4. Package

Bundles the compiled code into a deployable artifact (e.g., .jar, .war, Docker Image, .zip).

5. Post Actions & Managing Artifacts

• Post Actions: Executed at the end of the pipeline or a stage, depending on the outcome (success, failure, always). Used for notifications (Slack, Email) or cleanup.
• Artifacts: Files produced by the build (e.g., binaries, test reports) that need to be saved.
  
  GROOVY

  post {
      always {
          archiveArtifacts artifacts: 'target/*.jar', fingerprint: true
          junit 'target/surefire-reports/*.xml'
      }
      failure {
          mail to: 'team@example.com', subject: 'Build Failed', body: 'Check Jenkins.'
      }
  }
  

4. Jenkins and Maven

Maven Installation & Global Tool Configuration

• Navigate to Manage Jenkins -> Global Tool Configuration.
• Under the Maven section, add a Maven installation. You can point to a local installation directory or instruct Jenkins to install it automatically from Apache.
• Name the tool (e.g., M3).

Running Maven Builds in Pipelines

You can reference the globally configured Maven tool in your Jenkinsfile.

pipeline {
    agent any
    tools {
        maven 'M3' // Matches the name in Global Tool Configuration
    }
    stages {
        stage('Maven Build') {
            steps {
                sh 'mvn clean package -DskipTests'
            }
        }
    }
}

Code Coverage & Test Reports

• Test Reports: Maven generates JUnit XML reports in target/surefire-reports. Jenkins reads these using the junit step.
• Code Coverage: Tools like JaCoCo (Java Code Coverage) generate reports. By installing the JaCoCo Jenkins plugin, you can publish coverage trends on the job dashboard.
  
  GROOVY

  stage('Test') {
      steps {
          sh 'mvn test jacoco:report'
      }
      post {
          always {
              junit 'target/surefire-reports/*.xml'
              jacoco execPattern: 'target/jacoco.exec'
          }
      }
  }
  

5. Docker and Jenkins Integration

Docker Inside Jenkins Agents

• Docker-in-Docker (DinD): Running a Docker daemon inside a Jenkins Docker container. Complex and requires privileged mode.
• Docker-out-of-Docker (DooD): Mounting the host's Docker socket (/var/run/docker.sock) into the Jenkins container. Jenkins uses the host's Docker engine. Recommended for simplicity and security.

Building Docker Images using Jenkins

Using the Docker Pipeline plugin, Jenkins can build images natively in the pipeline.

stage('Build Docker Image') {
    steps {
        script {
            dockerImage = docker.build("myrepo/myapp:${env.BUILD_ID}")
        }
    }
}

Publishing Images to Docker Hub/GHCR

Use withRegistry to authenticate and push to container registries like Docker Hub or GitHub Container Registry (GHCR).

stage('Push Image') {
    steps {
        script {
            docker.withRegistry('https://registry.hub.docker.com', 'dockerhub-credentials-id') {
                dockerImage.push('latest')
                dockerImage.push("${env.BUILD_ID}")
            }
        }
    }
}

6. Jenkins and GitHub Integration

• Connecting Repositories: Use Personal Access Tokens (PAT) or SSH keys as credentials to clone private repositories.
• Webhooks (Push-based): The most efficient way to trigger builds. GitHub sends an HTTP POST payload to Jenkins (http://<jenkins-url>/github-webhook/) whenever a code push or Pull Request event occurs.
• Polling (Pull-based): Jenkins periodically checks GitHub for changes (e.g., H/5 * * * * checks every 5 minutes). Inefficient and puts unnecessary load on both Jenkins and GitHub; Webhooks are preferred.
• GitHub Checks: Jenkins can send build statuses back to GitHub to block Pull Request merges if the Jenkins build fails.

7. Jenkins CI/CD Deployment Flows

Triggering Builds

• pollSCM: Cron-based polling of the source code repository.
• Webhook: Event-driven triggers from the SCM tool.
• Upstream/Downstream: A pipeline triggered by the completion of another pipeline.
• Scheduled: Time-based execution (e.g., nightly builds).

Pipeline Libraries (Shared Libraries)

As Jenkins usage grows, teams duplicate pipeline code. Shared Libraries allow you to abstract repetitive Groovy code into a centralized Git repository.

• Configured in Manage Jenkins -> System -> Global Pipeline Libraries.
• Imported at the top of a Jenkinsfile: @Library('my-shared-library') _
• Used to standardize enterprise build processes and security checks.

Jenkins Agents (SSH / SFTP / Container-based)

• SSH Agents: Jenkins Master connects to a remote Linux/Unix machine via SSH, installs an agent JAR, and executes commands.
• Container-based Agents: Using the Docker or Kubernetes plugins, Jenkins dynamically provisions an agent container, runs the build inside it, and terminates the container when finished. This ensures a clean, reproducible build environment and optimizes resource usage.

Deployments to Servers/Clouds

Deployments in Jenkins are usually shell scripts, API calls, or configuration management tool invocations.

• SSH/SCP: Transferring artifacts to a target server and restarting services via SSH.
• Cloud Integrations: Using plugins (like AWS Steps or Azure CLI) or container orchestration tools (like kubectl or helm) to deploy applications to cloud providers (AWS, Azure, GCP, Kubernetes).

8. Backup, Restore, and Pipeline Best Practices

Backup & Restore

• JENKINS_HOME: Almost all Jenkins configurations, jobs, and histories are stored in the JENKINS_HOME directory (often /var/lib/jenkins). Backing up this directory is the most reliable way to secure Jenkins.
• ThinBackup Plugin: A popular plugin that schedules regular backups of configuration files (ignoring heavy artifacts or workspaces to keep backup sizes small).
• Configuration as Code (JCasC): Modern approach where Jenkins infrastructure is defined in YAML files, drastically reducing the need to back up manual UI configurations.

Pipeline Best Practices

1. Always use Pipeline as Code (Jenkinsfile): Store the Jenkinsfile in version control alongside the application code.
2. Use Declarative Syntax: It is more structured, easier to maintain, and validates syntax before running.
3. Do not run builds on the Master: Keep the Master node strictly for orchestration to ensure UI responsiveness.
4. Clean up Workspaces: Use cleanWs() at the beginning or end of pipelines to prevent disk space exhaustion.
5. Use Shared Libraries: DRY (Don't Repeat Yourself). Abstract common pipeline logic.
6. Manage Credentials Securely: Never hardcode passwords or API keys in the Jenkinsfile; always use Jenkins Credentials Binding.
7. Isolate Build Environments: Use Docker as agents (agent { docker { image 'maven:3.8' } }) to ensure build dependencies are isolated and do not pollute the host machine.