Unit 3: Continuous Integration (CI) & Build Automation

I. CI Fundamentals and Benefits

Continuous Integration (originating with Grady Booch, 1991; popularised by Kent Beck in Extreme Programming, late 1990s) is the practice of merging every developer's work into a shared mainline many times a day, with each merge verified by an automated build and test run. The whole unit assumes this feedback loop.

• Core definition: developers push small commits to a shared repository; an automated server builds and tests each commit, rejecting anything that breaks the build.
• Trunk-based working: short-lived branches merged into main frequently — ideally at least once per day per developer — to minimise divergence.
• Fail fast: a broken build stops the line; fixing it is the team's top priority before new work proceeds.
• Automation over manual gates: compilation, testing and packaging run without human steps, triggered by a version-control event.
• Benefits:
  • Early defect detection: integration bugs surface within minutes, not at a release-day "big-bang" merge.
  • Reduced merge conflict cost: small, frequent merges shrink the diff each developer must reconcile.
  • Always-releasable mainline: a green build means the codebase is a candidate for deployment at any time.
  • Faster feedback: developers learn of failures while the change is still fresh in memory.

II. CI Pipeline Architecture

A CI pipeline is the ordered set of automated stages a commit passes through from source change to verified artifact.

A. Trigger and Source Stage

The pipeline begins when version control signals a change.

• Webhook trigger: the SCM (e.g. GitHub) sends an HTTP POST to the CI server on push or pull_request events.
• Checkout: the runner clones the exact commit SHA so the build is reproducible.
• Polling (fallback): the server queries the repo on a schedule (H/5 * * * *) when webhooks are unavailable.

B. Build and Test Stages

The commit is compiled, packaged and validated.

• Build stage: compile sources, resolve dependencies, produce a binary or bundle.
• Test stage: run unit tests, then integration tests, gating progression on pass/fail.
• Artifact stage: store the packaged output (JAR, Docker image, tarball) in an artifact registry keyed by build number.

C. Runners, Agents and Feedback

Execution is distributed and results reported back.

• Agents/runners: worker nodes (persistent or ephemeral containers) that execute stage steps in isolation.
• Fan-out parallelism: independent jobs (e.g. lint, unit test, build) run concurrently to shorten wall-clock time.
• Status reporting: the pipeline posts a commit status (green/red) back to the SCM and notifies via chat or email.

III. Popular CI Tools

Modern CI platforms differ chiefly in hosting model, configuration syntax and ecosystem integration.

A. Jenkins

An open-source, self-hosted automation server with a controller/agent topology.

• Extensibility: 1,800+ plugins cover SCM, build tools, cloud and notifications.
• Configuration: pipelines defined in a Jenkinsfile (Groovy) stored with the code.
• Trade-off: maximum flexibility and control, but the team maintains the infrastructure and plugin compatibility.

B. GitHub Actions

CI/CD built into GitHub, configured by YAML in .github/workflows/.

• Event model: workflows triggered by repository events (on: push, on: pull_request).
• Reusable units: community "actions" from the Marketplace (e.g. actions/checkout) compose steps.
• Hosted runners: GitHub-provided Linux/Windows/macOS VMs, billed by the minute; self-hosted runners also supported.

C. GitLab CI

CI/CD native to GitLab, configured by a .gitlab-ci.yml at the repo root.

• Stages and jobs: stages: defines order; jobs assigned to a stage run in parallel within it.
• GitLab Runner: an agent executing jobs via Docker, shell or Kubernetes executors.
• Integration: merge-request pipelines and built-in container registry tie CI tightly to the SCM.

D. CircleCI

A cloud-first CI platform configured by .circleci/config.yml.

• Orbs: shareable, versioned config packages that bundle jobs and commands.
• Executors: choice of docker, machine or macos environments per job.
• Caching and workspaces: explicit save_cache/restore_cache speed dependency-heavy builds.

IV. Build Tools Overview

Build tools automate dependency resolution, compilation and packaging so the CI server runs a single reproducible command.

A. Maven

A declarative Java build tool driven by a pom.xml.

• Convention over configuration: a fixed lifecycle — validate → compile → test → package → verify → install → deploy.
• Dependency management: coordinates (groupId:artifactId:version) resolved from Maven Central.
• Command: mvn clean package compiles, tests and produces a JAR/WAR in target/.

B. Gradle

A JVM build tool using a Groovy/Kotlin DSL (build.gradle).

• Task graph: a directed acyclic graph of tasks executed in dependency order.
• Incremental builds: up-to-date checks and a build cache skip unchanged work, faster than Maven on large projects.
• Command: gradle build runs compileJava, test and assemble.

C. npm

The Node.js package manager and script runner (package.json).

• Dependency files: dependencies and devDependencies; package-lock.json pins exact versions.
• Scripts: "scripts": { "build": "...", "test": "..." } invoked with npm run <name>.
• Command: npm ci installs from the lockfile for reproducible CI installs.

V. Writing Jenkins Declarative Pipelines

The declarative pipeline is a structured, opinionated syntax for defining a Jenkinsfile around a mandatory pipeline block.

A. Structure and Directives

Every declarative pipeline follows a fixed skeleton.

• agent: where the pipeline runs (any, none, docker, label).
• stages / stage: the ordered logical phases shown on the Jenkins UI.
• steps: the concrete commands inside a stage (sh, bat, echo).
• environment / post: variables shared across stages; post runs cleanup or notification on always, success or failure.

B. Worked Example

pipeline {
    agent any
    stages {
        stage('Build') {
            steps { sh 'mvn clean package' }
        }
        stage('Test') {
            steps { sh 'mvn test' }
        }
    }
    post {
        failure { echo 'Build failed — notifying team' }
    }
}

• agent any: run on any available agent.
• stage('Build'): compiles and packages via Maven; a non-zero exit fails the stage.
• post { failure }: executes only when a prior stage errored.

VI. Automated Builds

An automated build converts source into a deployable artifact with no manual intervention, triggered by each integration.

• Deterministic output: identical inputs (commit + pinned dependencies) yield identical artifacts; lockfiles enforce this.
• Clean environment: builds run in fresh containers to avoid "works on my machine" state leakage.
• Single command: the pipeline invokes one entry point (mvn package, gradle build, npm run build) so behaviour matches local runs.
• Artifact versioning: outputs tagged with the build number or commit SHA for traceability and rollback.
• Fast build principle: keep the commit build under ~10 minutes; offload slow steps to later pipeline stages.

VII. Automated Testing

Automated testing runs the suite on every build so functional regressions block the pipeline rather than reaching production.

A. The Test Pyramid

Tests are layered by scope, speed and quantity.

• Unit tests (base): many, fast, isolated — the bulk of the suite.
• Integration tests (middle): fewer, verify component interaction (DB, API).
• End-to-end tests (top): fewest, slowest, exercise the whole system through the UI.

B. Placement in the Pipeline

Tests gate progression by cost.

• Fast tests first: unit tests run in the commit stage for sub-minute feedback.
• Slow tests later: integration and E2E tests run in downstream stages, often in parallel.
• Fail-fast gating: any red test halts the pipeline and marks the commit as broken.

VIII. Test Automation Basics

Test automation basics establish how correctness is measured and how thoroughly the tests exercise the code.

A. Unit Testing

A unit test verifies the smallest testable piece of code in isolation.

• Arrange–Act–Assert: set up inputs, invoke the unit, assert the outcome.
• Isolation: dependencies replaced with mocks or stubs so failures localise to one unit.
• Frameworks: JUnit (Java), pytest (Python), Jest (JavaScript).
• Example:
  
  JAVA

    @Test
    void addsTwoNumbers() {
        assertEquals(5, Calculator.add(2, 3));
    }

  • assertEquals(5, ...): the test fails if add returns anything but 5.

B. Coverage

Coverage measures the proportion of code executed by the test suite.

• Line coverage: percentage of source lines run at least once.
• Branch coverage: percentage of decision outcomes (both if and else) exercised — stricter than line coverage.
• Tools: JaCoCo (Java), Istanbul/nyc (JavaScript), coverage.py (Python).
• Interpretation: high coverage shows code is exercised, not that it is correct; a threshold gate (e.g. fail below 80%) prevents regression in test discipline without treating 100% as the goal.