Unit 5: Overview of Container Orchestration and Microservices - Subjective Questions

Container orchestration automates the deployment, scaling, networking, and management of containerized applications.

Need for Orchestration:

• Scale management: Running hundreds or thousands of containers manually is error-prone and impractical.
• High availability: Orchestrators automatically restart failed containers and reschedule them on healthy nodes.
• Load balancing: Traffic is distributed across container instances automatically.
• Service discovery: Containers can locate and communicate with each other dynamically.
• Rolling updates & rollbacks: Enables zero-downtime deployments and safe reversions.
• Resource optimization: Efficiently packs containers onto nodes based on CPU/memory requirements.
• Self-healing: Detects and replaces unhealthy containers without manual intervention.

Why manual management fails at scale:

• No automated recovery from failures.
• Difficult to maintain consistent configuration across nodes.
• Scaling up/down requires manual effort and monitoring.
• Networking and secret management become unmanageable.

Tools like Kubernetes, Docker Swarm, and Nomad solve these problems by providing a declarative, automated control plane.

Kubernetes follows a master-worker (control plane - node) architecture.

Control Plane Components:

• kube-apiserver: The front-end of the control plane; exposes the Kubernetes API and handles all requests.
• etcd: A consistent, distributed key-value store holding all cluster state and configuration data.
• kube-scheduler: Assigns newly created Pods to suitable nodes based on resource requirements and constraints.
• kube-controller-manager: Runs controller processes (node controller, replication controller, endpoints controller) that regulate cluster state.
• cloud-controller-manager: Integrates with underlying cloud provider APIs.

Worker Node Components:

• kubelet: An agent on each node that ensures containers described in PodSpecs are running and healthy.
• kube-proxy: Maintains network rules and handles Pod networking/load balancing.
• Container Runtime: Software that runs containers (e.g., containerd, CRI-O).

Workflow:

1. User submits a desired state via kubectl to the API server.
2. State is stored in etcd.
3. Scheduler assigns Pods to nodes.
4. kubelet on the node instructs the container runtime to launch containers.
5. Controllers continuously reconcile actual state with desired state.

This declarative, self-healing model is the core strength of Kubernetes.

A Pod is the smallest and simplest deployable unit in Kubernetes. It represents a single instance of a running process and can contain one or more tightly coupled containers.

Key characteristics:

• Containers in a Pod share the same network namespace (same IP address and port space).
• They share storage volumes.
• They are always scheduled together on the same node.
• They can communicate via localhost.

Why Pods instead of containers:

• Co-location: Helper containers (e.g., logging, sidecars) often need to run alongside the main application container.
• Shared resources: Grouping enables shared networking and storage.
• Atomic scheduling: Kubernetes schedules and scales at the Pod level, simplifying management.
• Abstraction: Provides a higher-level abstraction over container runtimes.

Note: Pods are ephemeral — when a Pod dies, it is not resurrected but replaced by a new Pod (often via a Deployment or ReplicaSet).

A Deployment is a Kubernetes object that provides declarative updates for Pods and ReplicaSets.

Functions of a Deployment:

• Declarative management: You describe the desired state (e.g., number of replicas, image version), and the Deployment controller works to achieve it.
• Rolling updates: Gradually replaces old Pods with new ones without downtime.
• Rollbacks: Reverts to a previous stable version if an update fails.
• Scaling: Easily scale the number of replicas up or down.
• Self-healing: Maintains the desired number of Pod replicas via the underlying ReplicaSet.

Rolling Update Process:

1. A new version of the application image is specified.
2. Kubernetes creates new Pods incrementally.
3. Old Pods are terminated gradually as new ones become ready.
4. Parameters like maxSurge and maxUnavailable control the pace.

Example YAML snippet:
yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:

• name: nginx
  image: nginx:1.21

Deployments ensure reliability, availability, and controlled updates for stateless applications.

A Service is an abstraction that defines a logical set of Pods and a policy to access them. Since Pods are ephemeral (their IPs change), Services provide a stable network endpoint.

Why Services are needed:

• Pods are created and destroyed dynamically.
• Provides stable IP/DNS name and load balancing across Pods.
• Enables service discovery.

Types of Services:

• ClusterIP (default):

  • Exposes the Service on an internal cluster IP.
  • Accessible only within the cluster.
  • Use case: internal communication between microservices.

• NodePort:

  • Exposes the Service on each node's IP at a static port.
  • Accessible externally via <NodeIP>:<NodePort>.
  • Use case: development/testing external access.

• LoadBalancer:

  • Provisions an external load balancer (via cloud provider).
  • Use case: production external access on cloud platforms.

• ExternalName:

  • Maps a Service to an external DNS name.
  • Use case: accessing external services via internal DNS.

Services use labels and selectors to dynamically match target Pods.

Monolithic vs Microservice Architecture:

Summary:

• Monolithic architecture is simpler to develop and deploy initially but becomes hard to scale and maintain as it grows.
• Microservices offer flexibility, independent scaling, and resilience but introduce operational complexity in networking, monitoring, and distributed data management.

Microservices decompose an application into small, independent, loosely-coupled services, each responsible for a specific business capability.

Key Motivations:

• Overcoming the limitations of monolithic systems (hard to scale, deploy, and maintain).
• Enabling agility and faster time-to-market.

Benefits:

• Independent Deployment: Each service can be deployed without affecting others, enabling continuous delivery.
• Scalability: Individual services can be scaled based on demand, optimizing resource usage.
• Technology Diversity: Teams can choose the best tech stack per service (polyglot).
• Fault Isolation: A failure in one service does not bring down the entire system.
• Team Autonomy: Small teams own specific services end-to-end, improving productivity.
• Easier Maintenance: Smaller codebases are easier to understand, test, and update.
• Reusability: Services can be reused across different applications.

Trade-offs to consider:

• Increased operational complexity.
• Network latency and distributed system challenges.
• Need for robust monitoring and DevOps practices.

1. Scalability:
Microservices allow granular, independent scaling of individual services.

• In a monolith, if only the payment module is under heavy load, you must scale the entire application, wasting resources.
• In microservices, you can scale only the payment service (e.g., from 2 to 10 instances) while leaving others unchanged.

Example: During a sale, an e-commerce platform can scale the checkout and inventory services independently while the user-profile service remains at baseline.

Types of Scaling:

• Horizontal scaling: Adding more service instances.
• Vertical scaling: Adding more resources to existing instances.

2. Independent Deployment:
Each microservice can be built, tested, and deployed separately.

• Enables Continuous Deployment (CD).
• Reduces deployment risk — a change to one service doesn't require redeploying the whole system.
• Faster release cycles and rollbacks.

Example: The recommendation service can be updated and deployed multiple times a day without touching the authentication service.

Combined Impact: These benefits enable agility, resilience, and efficient resource utilization in large-scale systems.

Kubernetes uses a layered abstraction to manage application instances.

Hierarchy:

Deployment
└── manages → ReplicaSet
└── manages → Pods
└── contains → Containers

1. Pod:

• Smallest deployable unit; wraps one or more containers.
• Ephemeral and can be replaced.

2. ReplicaSet:

• Ensures a specified number of identical Pod replicas are running at all times.
• If a Pod dies, the ReplicaSet creates a new one.
• Uses label selectors to identify managed Pods.

3. Deployment:

• A higher-level controller that manages ReplicaSets.
• Provides rolling updates, rollbacks, and versioning.
• When you update a Deployment, it creates a new ReplicaSet and gradually shifts Pods.

Workflow example:

1. You define a Deployment with replicas: 3.
2. Deployment creates a ReplicaSet.
3. ReplicaSet creates 3 Pods.
4. On update, Deployment creates a new ReplicaSet and performs a rolling update.

Summary: Deployments manage ReplicaSets, which manage Pods, which run containers — providing automation, resilience, and controlled updates.

1. kube-apiserver:

• The central management entity and front-end of the Kubernetes control plane.
• Exposes the Kubernetes REST API.
• All communication (from kubectl, controllers, kubelets) goes through the API server.
• Responsibilities:
  • Validates and processes API requests.
  • Serves as the gateway to the cluster.
  • Handles authentication, authorization, and admission control.

2. etcd:

• A distributed, consistent key-value store.
• Stores the entire cluster state and configuration (desired state, Pod specs, secrets, config maps).
• Based on the Raft consensus algorithm for consistency and fault tolerance.

Why they are critical:

• etcd is the single source of truth for the cluster. Losing it means losing all cluster state.
• The API server is the only component that directly reads/writes to etcd, ensuring controlled access.
• Together, they enable the declarative, self-healing model: controllers continuously compare actual state (via API server) with desired state (stored in etcd).

Best practices: etcd should be backed up regularly and run in a highly-available (odd-numbered) cluster.

Evolution of Deployment:

1. Traditional Deployment:

• Applications run directly on physical servers.
• Problems: No resource boundaries, poor utilization, apps interfere with each other, hard to scale.

2. Virtualized Deployment:

• Multiple Virtual Machines (VMs) run on a single physical server via a hypervisor.
• Advantages: Isolation, better resource utilization, scalability.
• Drawbacks: Each VM has a full OS — heavy, slow to boot, high overhead.

3. Container Deployment (Kubernetes era):

• Containers share the host OS kernel but are isolated.
• Advantages:
  • Lightweight: No full OS per container — fast startup.
  • Portable: Consistent across environments (dev, test, prod).
  • Efficient: Higher density and better resource utilization than VMs.
  • Agile: Faster deployment and rollback.

What Kubernetes adds:

• Orchestration of containers at scale.
• Self-healing, auto-scaling, service discovery, load balancing.
• Declarative configuration and automation.

Summary: Kubernetes builds on containers to provide production-grade, automated, and resilient application management — a major leap over traditional and VM-based approaches.

Service Discovery and Load Balancing are two core functions of Kubernetes Services.

The Problem:

• Pods are ephemeral and have dynamic IP addresses.
• A client cannot reliably connect to a Pod by its IP.

1. Service Discovery:

• Each Service gets a stable virtual IP (ClusterIP) and a DNS name.
• Kubernetes DNS (CoreDNS) automatically creates DNS records like my-service.my-namespace.svc.cluster.local.
• Applications reference the Service by name rather than Pod IP.

Mechanisms:

• Environment variables: Injected into Pods at creation.
• DNS-based discovery: Preferred and dynamic.

2. Load Balancing:

• A Service uses label selectors to identify backend Pods (the endpoints).
• Incoming traffic is distributed across all matching healthy Pods.
• kube-proxy maintains the network rules (via iptables or IPVS) to route and balance traffic.

Example flow:

1. Client requests http://payment-service.
2. DNS resolves to the Service's ClusterIP.
3. kube-proxy load-balances the request to one of the healthy payment Pods.

Result: Applications communicate reliably despite Pod churn, and traffic is evenly distributed for high availability and scalability.

While microservices offer many benefits, they introduce significant challenges.

Challenges of Microservices:

• Operational Complexity: Managing many services requires sophisticated orchestration, monitoring, and logging.
• Distributed System Challenges: Network latency, message serialization, and partial failures.
• Data Management: Maintaining data consistency across services (eventual consistency, distributed transactions) is hard.
• Testing Complexity: Integration and end-to-end testing across services is difficult.
• Debugging & Tracing: Requires distributed tracing tools (e.g., Jaeger, Zipkin).
• Deployment Overhead: Requires CI/CD pipelines, containerization, and orchestration (Kubernetes).
• Network Security: More inter-service communication increases the attack surface.

When a Monolith is Better:

• Small applications or startups with simple requirements.
• Small teams that cannot manage distributed system complexity.
• Early-stage products where requirements are still evolving.
• When low latency and simple deployment are priorities.
• Limited DevOps maturity/infrastructure.

Best Practice: Many organizations start with a well-structured monolith and migrate to microservices as the application and team grow (the Monolith First approach).

AWS (Amazon Web Services) provides a comprehensive suite of managed services to implement DevOps practices such as CI/CD, IaC, monitoring, and container orchestration.

Key AWS DevOps Services:

• AWS CodeCommit: A managed Git-based source control repository.
• AWS CodeBuild: Compiles source code, runs tests, and produces build artifacts (CI).
• AWS CodeDeploy: Automates application deployments to EC2, Lambda, or on-premises servers.
• AWS CodePipeline: Orchestrates the entire CI/CD workflow (build → test → deploy).
• AWS CloudFormation: Infrastructure as Code (IaC) — provisions resources declaratively via templates.
• Amazon ECS / EKS: Container orchestration (ECS = AWS-native, EKS = managed Kubernetes).
• Amazon CloudWatch: Monitoring, logging, and alerting.
• AWS CloudTrail: Auditing and API activity tracking.

DevOps Workflow on AWS:

1. Code stored in CodeCommit.
2. CodeBuild builds and tests.
3. CodeDeploy deploys the application.
4. CodePipeline automates the whole flow.
5. CloudWatch monitors the running application.

Benefits: Fully managed, scalable, integrated, and pay-as-you-go — reducing operational overhead.

Microsoft Azure offers integrated tools and services to support the complete DevOps lifecycle.

Azure DevOps (the platform):
A suite of services covering the entire application lifecycle:

• Azure Repos: Git repositories for source control.
• Azure Pipelines: CI/CD pipelines supporting multiple languages and platforms (including deployment to any cloud).
• Azure Boards: Agile planning, work item tracking, Kanban/Scrum boards.
• Azure Test Plans: Manual and exploratory testing tools.
• Azure Artifacts: Package management (npm, NuGet, Maven feeds).

Other Azure DevOps-supporting services:

• Azure Kubernetes Service (AKS): Managed Kubernetes for container orchestration.
• Azure Resource Manager (ARM) / Bicep: Infrastructure as Code.
• Azure Monitor & Application Insights: Monitoring, logging, and performance tracking.
• Azure Container Registry (ACR): Stores container images.

DevOps Workflow on Azure:

1. Plan work in Azure Boards.
2. Store code in Azure Repos.
3. Build & deploy via Azure Pipelines.
4. Deploy containers to AKS.
5. Monitor with Azure Monitor.

Benefits: Tight integration with Microsoft ecosystem, cross-platform support, and end-to-end lifecycle coverage.

Google Cloud Platform (GCP) provides managed services to enable DevOps practices, with strong emphasis on containers and Kubernetes (which Google originally created).

Key GCP DevOps Services:

• Cloud Source Repositories: Fully managed private Git repositories.
• Cloud Build: Serverless CI/CD platform that builds, tests, and deploys.
• Artifact Registry / Container Registry: Stores and manages container images and packages.
• Google Kubernetes Engine (GKE): Industry-leading managed Kubernetes service.
• Cloud Deploy: Managed continuous delivery service for GKE.
• Cloud Deployment Manager / Terraform: Infrastructure as Code.
• Cloud Operations (formerly Stackdriver): Monitoring, logging, and tracing.
• Cloud Run: Serverless container execution platform.

DevOps Workflow on GCP:

1. Code stored in Cloud Source Repositories.
2. Cloud Build triggers build & tests on commit.
3. Images pushed to Artifact Registry.
4. Deployed to GKE or Cloud Run.
5. Monitored via Cloud Operations Suite.

Key Strength: GCP excels in container orchestration since Kubernetes originated at Google, making GKE a mature and powerful option.

Conceptual Comparison of Major Cloud DevOps Services:

Observations:

• AWS: Broadest service portfolio and market leader; modular DevOps tools.
• Azure: Strongest integrated end-to-end DevOps platform (Azure DevOps); best for Microsoft ecosystem.
• GCP: Best-in-class Kubernetes (GKE) and container tooling; developer-friendly.

Common Ground: All three provide managed, scalable, pay-as-you-go services supporting the full DevOps lifecycle. The choice often depends on existing ecosystem, expertise, and specific requirements.

Declarative Model:
In Kubernetes, you declare the desired state of your application (e.g., "I want 3 replicas of nginx running") rather than issuing imperative commands. Kubernetes continuously works to make the actual state match the desired state.

• Desired state is defined in YAML/JSON manifests.
• Stored in etcd via the API server.

The Reconciliation Loop (Control Loop):
Controllers run continuous loops that:

1. Observe the current actual state of the cluster.
2. Compare it with the desired state.
3. Act to reconcile any differences.

Self-Healing Mechanism:

• Pod failure: If a Pod crashes, the ReplicaSet controller detects the discrepancy and creates a new Pod.
• Node failure: If a node goes down, Pods are rescheduled onto healthy nodes.
• Health checks: Liveness and readiness probes detect unhealthy containers and restart or remove them from service.

Example:

• Desired: 3 replicas.
• One Pod dies → actual = 2.
• Controller notices mismatch → creates 1 new Pod → actual = 3 again.

Benefit: This automated, continuous reconciliation makes Kubernetes resilient, reliable, and low-maintenance, requiring minimal manual intervention.

The Synergy: Microservices + Containers + Kubernetes

Microservices, containers, and orchestration form a natural, complementary technology stack.

1. Microservices → Containers:

• Each microservice is packaged into its own container, bundling code and dependencies.
• Containers provide isolation, portability, and consistency across environments.
• Each service can use its own tech stack without conflict.

2. Containers → Kubernetes (Orchestration):
As the number of microservice containers grows, manual management becomes impossible. Kubernetes provides:

• Deployment automation: Each microservice as a Deployment.
• Service discovery: Services connect microservices via stable DNS names.
• Independent scaling: Scale each microservice's Pods independently.
• Load balancing: Distribute traffic among Pod replicas.
• Self-healing: Restart failed microservice Pods automatically.
• Rolling updates: Update individual microservices with zero downtime.

Typical Deployment Flow:

1. Develop microservice → containerize (Docker).
2. Push image to a registry.
3. Define a Kubernetes Deployment and Service for each microservice.
4. Kubernetes schedules Pods across the cluster.
5. Services enable inter-microservice communication.
6. Monitor, scale, and update independently.

Why the Synergy Works:

• Microservices provide the architectural style.
• Containers provide packaging and isolation.
• Kubernetes provides the automated operational platform.

Together, they enable scalable, resilient, and agile cloud-native applications.

A Kubernetes worker node runs the actual application workloads (Pods). It contains three key components:

1. kubelet:

• The primary node agent that runs on every worker node.
• Communicates with the API server.
• Responsibilities:
  • Ensures containers described in PodSpecs are running and healthy.
  • Reports node and Pod status back to the control plane.
  • Executes liveness/readiness probes.
  • Manages the container lifecycle via the container runtime.
• Note: kubelet only manages containers created by Kubernetes.

2. kube-proxy:

• A network proxy running on each node.
• Responsibilities:
  • Maintains network rules that allow communication to Pods.
  • Implements the Service abstraction (load balancing traffic to Pods).
  • Uses iptables or IPVS to route packets.
• Enables both intra-cluster and external-to-Pod communication.

3. Container Runtime:

• The software responsible for actually running containers.
• Examples: containerd, CRI-O (Docker was deprecated as a runtime).
• Communicates with kubelet via the Container Runtime Interface (CRI).
• Responsibilities: Pulling images, starting/stopping containers, managing container storage and networking at the OS level.

Summary: kubelet manages Pod lifecycle, kube-proxy handles networking/load balancing, and the container runtime executes the containers — together making the node a functional part of the cluster.