Unit 5: Overview of Container Orchestration and Microservices

I. Orientation: Containers and the Orchestration Problem

Containers package an application with its dependencies into a single portable, immutable image; orchestration is the automated management of many such containers across a fleet of machines. A single Docker host can run a handful of containers, but production systems run hundreds across many nodes, and doing that by hand does not scale — hence orchestration.

• Container: an isolated process using the host kernel via namespaces and cgroups; starts in milliseconds, unlike a VM which boots a full OS.
• Image: a read-only, layered, versioned artifact (e.g. nginx:1.25); the unit of deployment.
• Declarative model: you describe the desired state (e.g. "5 replicas"), and the system continuously reconciles actual state toward it.
• Immutability: running containers are not patched in place; a new image is rolled out and old ones destroyed.
• Ephemerality: containers are disposable — any container may die and be replaced, so state must live outside them.

A. Need for Orchestration

An orchestrator exists to run and maintain containerised workloads at scale without manual intervention.

• Scheduling: decides which node runs which container based on CPU/memory requests and node capacity.
• Self-healing: restarts crashed containers, replaces failed ones, and reschedules workloads off dead nodes automatically.
• Scaling: adds or removes container replicas in response to load or an explicit command, up and down.
• Service discovery and load balancing: gives containers stable names and distributes traffic across replicas as they come and go.
• Rolling updates and rollback: replaces old versions gradually with zero downtime and reverts on failure.
• Configuration and secret management: injects environment-specific config without rebuilding images.
• Manual limitation contrasted: with plain docker run, a node crash means lost containers, no automatic replacement, no built-in load balancing — the orchestrator supplies exactly these.

II. Kubernetes Architecture Overview

The control plane / worker node model

Kubernetes (open-sourced by Google, 2014; abbreviated K8s) is the dominant orchestrator, built around a control plane that makes decisions and worker nodes that run workloads.

A. Control Plane Components

The control plane holds cluster state and drives reconciliation toward the desired state.

• kube-apiserver: the front end and only component clients talk to; exposes the REST API and validates every request. All other components communicate through it.
• etcd: a distributed, consistent key-value store holding the entire cluster state; the single source of truth. Losing etcd means losing the cluster's memory.
• kube-scheduler: watches for unscheduled Pods and binds each to a suitable node using filtering (does it fit?) and scoring (which node is best?).
• kube-controller-manager: runs control loops (e.g. the node controller, replication controller) that each watch state and act to correct drift.
• cloud-controller-manager: integrates with cloud provider APIs for load balancers, storage volumes and node lifecycle.

B. Worker Node Components

Worker nodes execute the containers the control plane schedules onto them.

• kubelet: the node agent; receives Pod specs from the API server and ensures the described containers are running and healthy.
• Container runtime: the software that actually runs containers (e.g. containerd, CRI-O) via the Container Runtime Interface.
• kube-proxy: maintains network rules (iptables/IPVS) on each node so Service traffic reaches the right Pods.
• Reconciliation loop illustrated: you POST a Deployment → API server writes to etcd → controller creates Pod objects → scheduler assigns nodes → each kubelet starts the containers. No step is manual.

III. Basic Kubernetes Concepts

Pods, Deployments and Services — the everyday objects

Kubernetes objects are declared in YAML manifests and submitted to the API server; the three below are what developers touch most.

A. Pods

A Pod is the smallest deployable unit in Kubernetes — one or more containers that share a network namespace and storage.

• Shared context: containers in a Pod share one IP address and can reach each other on localhost; they are always co-scheduled on the same node.
• Usual shape: one main container plus optional sidecars (e.g. a logging agent).
• Ephemeral identity: a Pod gets a new IP when recreated; you never rely on a Pod's IP directly.
• Not created directly: in practice a controller (a Deployment) creates Pods so they can be replaced and scaled.

apiVersion: v1
kind: Pod
metadata:
  name: web
spec:
  containers:
  - name: nginx
    image: nginx:1.25
    ports:
    - containerPort: 80

B. Deployments

A Deployment declares a desired number of identical Pod replicas and manages their lifecycle over time.

• ReplicaSet underneath: the Deployment creates a ReplicaSet, which keeps exactly replicas Pods alive; if one dies, a new one is spawned.
• Rolling update: changing the image spins up new Pods and terminates old ones incrementally, honouring maxSurge and maxUnavailable.
• Rollback: kubectl rollout undo reverts to a previous ReplicaSet revision.
• Scaling: kubectl scale deployment web --replicas=5 changes desired count; the controller reconciles.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3
  selector:
    matchLabels: { app: web }
  template:
    metadata:
      labels: { app: web }
    spec:
      containers:
      - name: nginx
        image: nginx:1.25

C. Services

A Service provides a stable network endpoint and load balancing for a changing set of Pods.

• Why needed: Pod IPs are ephemeral; a Service gives a fixed virtual IP (ClusterIP) and DNS name that survive Pod churn.
• Label selector: the Service routes to Pods whose labels match its selector (e.g. app: web), automatically tracking replicas as they scale.
• Types contrasted:
  1. ClusterIP: internal-only virtual IP; default, for pod-to-pod traffic.
  2. NodePort / LoadBalancer: expose the Service outside the cluster — NodePort opens a port on every node; LoadBalancer provisions a cloud load balancer.

apiVersion: v1
kind: Service
metadata:
  name: web-svc
spec:
  selector: { app: web }
  ports:
  - port: 80
    targetPort: 80

IV. Microservices Architecture

Decomposing the application

A microservice architecture structures an application as a collection of small, independently deployable services, each owning one business capability. Containers and Kubernetes are the natural substrate for running them.

A. Why Microservices?

Microservices exist to let large systems evolve quickly by splitting them into independently owned pieces.

• Independent teams: each service is owned end-to-end by a small team ("two-pizza team") that ships on its own schedule.
• Technology freedom: one service can use Java, another Go — each picks the best tool for its job.
• Fault isolation: a failure in one service (e.g. recommendations) need not crash the whole system.
• Targeted scaling: scale only the busy service rather than the entire application.

B. Monolithic and Microservice Architecture

The two styles differ in how the application is packaged, deployed and scaled.

1. Monolithic architecture: all functionality compiled and deployed as one unit.
   • Single codebase and process: UI, business logic and data access ship together as one artifact (e.g. one .war file).
   • Strengths: simple to develop, test and deploy initially; no network hops between modules.
   • Weaknesses: a small change forces redeploying everything; tight coupling slows large teams; the whole app must scale together.
2. Microservice architecture: functionality split across many services.
   • Independent processes: each service runs and deploys separately, communicating over the network (REST/gRPC/messaging).
   • Decentralised data: each service owns its own database, avoiding shared-schema coupling.
   • Strengths: independent deployment and scaling, fault isolation, team autonomy.
   • Weaknesses: operational complexity — distributed transactions, network latency, monitoring and service discovery all become harder.

C. Benefits (scalability, independent deployment)

The two headline advantages of microservices are fine-grained scalability and decoupled release cycles.

• Scalability: scale each service independently to its own load — run 20 replicas of the checkout service and 2 of the profile service instead of scaling one large monolith uniformly, saving resources.
• Independent deployment: deploy a change to one service without rebuilding or redeploying others, enabling frequent, low-risk releases and faster time to market.
• Resilience: combined with orchestration self-healing, a crashed service instance is replaced automatically while the rest keep serving.
• Cost efficiency: granular scaling means capacity is allocated where demand actually is.

V. Intro to DevOps on AWS/GCP/Azure

Managed platforms for the same primitives (conceptual)

The major clouds offer managed services that implement the container, orchestration and CI/CD building blocks so teams need not run the infrastructure themselves.

A. Common DevOps Building Blocks Across Clouds

Each provider supplies the same conceptual categories under different names.

• Managed Kubernetes: runs the control plane for you — AWS EKS, Google GKE, Azure AKS. GKE reflects Kubernetes' Google origin.
• Container registries: store images — AWS ECR, Google Artifact Registry, Azure ACR.
• CI/CD pipelines: automate build-test-deploy — AWS CodePipeline/CodeBuild, Google Cloud Build, Azure DevOps Pipelines.
• Infrastructure as Code: declare infrastructure in files — AWS CloudFormation, Google/Azure via Terraform or native templates; provisioning becomes repeatable and version-controlled.
• Monitoring and logging: observe running systems — AWS CloudWatch, Google Cloud Operations, Azure Monitor.
• Serverless containers: run containers without managing nodes — AWS Fargate, Google Cloud Run, Azure Container Instances.

B. Why Teams Adopt Cloud DevOps

Managed services shift undifferentiated operational work onto the provider.

• Reduced operational burden: the provider patches, scales and secures the control plane and CI infrastructure.
• Elastic capacity: nodes and pipeline runners scale on demand and are billed for actual use.
• Integrated toolchain: registry, pipeline, cluster and monitoring interoperate natively, shortening the path from commit to production.