Unit 5: Cloud-Native Development

I. Orientation — Principles of Cloud-Native Systems

Cloud-native development is an approach to designing, building, deploying, and operating applications that exploit cloud capabilities such as on-demand infrastructure, elastic scaling, automation, and managed services. Popularized through public-cloud platforms and organizations such as the Cloud Native Computing Foundation (founded 2015), it emphasizes distributed services, containers, declarative configuration, and resilient automation.

• Defining properties:
  • Service orientation: Applications are decomposed into independently deployable services with explicit APIs.
  • Elasticity: Computing capacity expands or contracts according to demand.
  • Automation: Build, test, deployment, recovery, and infrastructure provisioning use automated pipelines.
  • Resilience: Systems anticipate partial failure and apply retries, timeouts, redundancy, and graceful degradation.
  • Observability: Metrics, logs, and distributed traces reveal system health and request flow.
• Operational assumptions:
  • Failure is normal: Instances, networks, and dependencies may fail independently.
  • Instances are replaceable: Containers or virtual machines should be recreated rather than manually repaired.
  • State requires explicit treatment: Durable data must reside in databases, object stores, queues, or persistent volumes.
• Common platform components:
  • Containers: Docker-compatible images package code and dependencies.
  • Orchestration: Kubernetes schedules containers and maintains declared state.
  • Delivery: Continuous integration and continuous delivery—CI/CD—support frequent, repeatable releases.

II. Cloud-Native Architecture — Structure for Elastic and Resilient Applications

A. Cloud-native architecture

Cloud-native architecture organizes applications around independently operated components and automated cloud infrastructure.

• Architectural style: A user request commonly passes through an API gateway to services running in containers, while managed databases, caches, and message brokers provide supporting capabilities.
• API gateway: A component such as Amazon API Gateway or Kong provides a single entry point for routing, authentication, rate limiting, and protocol translation.
• Containers and orchestration: Kubernetes represents desired deployment state in declarative manifests; a Deployment with replicas: 3 instructs the controller to maintain three pod replicas.
• Resilience patterns:
  • Timeout: Stops a request from waiting indefinitely.
  • Retry with backoff: Repeats transiently failed operations after increasing delays.
  • Circuit breaker: Temporarily blocks calls to an unhealthy dependency.
  • Bulkhead: Isolates resource pools so one failure does not exhaust the entire system.
• Observability: A trace identifier propagated across HTTP headers connects spans from gateway, service, database, and queue operations.
• Trade-off: Distribution improves independent scaling and deployment but introduces network latency, partial failures, version compatibility, and operational complexity.

III. Service Decomposition — Independence Through Clear Boundaries

A. Loosely coupled services

Loosely coupled services minimize knowledge of one another’s internal implementation and communicate through stable contracts.

• High cohesion: Each service owns a focused business capability, such as catalog management, payment authorization, or shipment tracking.
• Contract-based interaction: An HTTP endpoint such as GET /orders/{id} exposes an interface without exposing the service’s classes or database tables.
• Independent deployment: Updating a recommendation algorithm should not require redeploying the customer-account service.
• Communication modes:
  1. Synchronous: REST or gRPC returns an immediate response but creates runtime dependency on the receiver.
  2. Asynchronous: Kafka or RabbitMQ delivers events such as OrderPlaced, reducing temporal coupling but requiring eventual-consistency handling.
• Database ownership: A service generally owns its data schema; direct cross-service table access would couple releases and bypass the API.
• Limitations: Excessively small services create “distributed monoliths,” where many coordinated calls and releases remain necessary despite physical separation.

IV. Dynamic Location — Finding Changing Service Instances

A. Service Discovery

Service discovery maps a logical service name to currently available network endpoints in an environment where instances frequently appear and disappear.

• Need: An autoscaled inventory service may move among hosts and receive changing IP addresses, making hard-coded addresses unsuitable.
• Discovery models:
  1. Client-side discovery: The client queries a registry and selects an instance; Netflix Eureka historically supported this pattern.
  2. Server-side discovery: A proxy or load balancer queries the registry and forwards the request; Kubernetes Services provide stable virtual endpoints for pods.
• Registration: Instances register themselves, or the platform detects them through deployment metadata and health probes.
• Health checking: Readiness checks exclude instances unable to serve traffic; Kubernetes can evaluate an endpoint such as /ready.
• DNS discovery: A name such as payments.default.svc.cluster.local resolves a Kubernetes Service without exposing individual pod addresses.
• Failure concern: Stale registry entries can route calls to dead instances, so leases, heartbeats, and rapid deregistration are required.

V. Traffic Distribution — Sharing Requests Across Instances

A. Load Balancing

Load balancing distributes traffic among healthy instances to improve availability, utilization, and response time.

• Placement layers:
  • Layer 4: Balances TCP or UDP connections using addresses and ports.
  • Layer 7: Examines HTTP information such as hostnames, paths, cookies, or headers.
• Algorithms:
  • Round robin: Sends successive requests to successive servers.
  • Least connections: Selects the server with the fewest active connections.
  • Weighted routing: Assigns more traffic to higher-capacity instances.
  • Consistent hashing: Maps a key to a server while minimizing remapping when membership changes.
• Health awareness: A load balancer removes an instance after failed health checks and restores it after recovery.
• Session handling: Stateless services allow any replica to process a request; sticky sessions reduce flexibility and can create uneven traffic.
• Deployment use: Canary routing might send 5% of traffic to version v2 and 95% to v1, limiting release risk.
• Limitation: Load balancing redistributes existing capacity; it does not create capacity, so it must work with autoscaling.

VI. Elastic Capacity — Adapting Resources to Demand

A. Autoscaling

Autoscaling automatically adjusts resources according to measured demand, schedules, or predicted workload.

• Horizontal scaling: Adds or removes instances; a Kubernetes Horizontal Pod Autoscaler can change a deployment from 3 to 10 pods.
• Vertical scaling: Changes CPU or memory allocated to an instance, although resizing may require restart or replacement.
• Scaling signal: Common metrics include CPU utilization, memory, request rate, queue depth, and latency percentiles.
• Replica estimate:

desiredReplicas = ceil(currentReplicas × currentMetric / targetMetric)

• Symbol definitions: currentReplicas is the number presently running; currentMetric is the observed average; targetMetric is the desired average; ceil rounds upward.
• Worked example: With 4 replicas at 75% average CPU and a 50% target, ceil(4 × 75/50) = 6 replicas.
• Stability controls: Cooldown periods, minimum and maximum replicas, and stabilization windows reduce rapid scale-out/scale-in oscillation.
• Constraint: Scaling is not instantaneous; startup time and database limits can remain bottlenecks.

VII. Persistent State — Data Across Distributed Services

A. Data Management

Data management in cloud-native systems balances service autonomy, durability, consistency, scalability, and regulatory requirements.

• Polyglot persistence: A service selects storage appropriate to its workload—PostgreSQL for transactions, Redis for caching, and an object store such as Amazon S3 for files.
• Consistency challenge: A business transaction spanning order, payment, and inventory services cannot safely depend on one shared local database transaction.
• Saga pattern: A sequence of local transactions coordinates a distributed process; if payment fails after inventory reservation, a compensating action releases the reservation.
• Eventual consistency: Replicas or services may temporarily disagree but converge after events propagate.
• Event sourcing: State is reconstructed from events such as AccountCredited rather than storing only the latest value; this provides an audit trail but complicates schema evolution.
• CQRS: Command Query Responsibility Segregation separates write models from optimized read models.
• Reliability controls: Backups, encryption, access policies, retention periods, and tested restoration procedures protect durable state.
• Caching risk: Redis can reduce database latency, but invalidation must prevent obsolete prices or permissions from being served.

VIII. Application Design Discipline — Portable Operational Practices

A. The twelve-factor app methodology

The twelve-factor app methodology, published by Heroku engineers in 2011, defines practices for portable software-as-a-service applications.

• Codebase: One version-controlled codebase supports many deployments, such as staging and production.
• Dependencies: Dependencies are explicitly declared and isolated through files such as package.json or requirements.txt.
• Config: Environment-specific configuration belongs in environment variables rather than committed source code.
• Backing services: Databases, queues, and caches are treated as replaceable attached resources identified by URLs.
• Build, release, run: Compilation creates a build; configuration combines with it to form a release; the release executes as processes.
• Processes: Application processes are stateless and share nothing; persistent sessions belong in an external store.
• Port binding: The application exports a service through a port, for example an HTTP server listening on $PORT.
• Concurrency: Capacity grows by running more process instances.
• Disposability: Fast startup and graceful shutdown support replacement and scaling.
• Development/production parity: Differences among environments are minimized.
• Logs: Applications emit event streams to standard output; the platform routes and stores them.
• Admin processes: Database migrations and maintenance tasks run as one-off processes using the same release.
• Limitation: The method guides application operation but does not by itself define microservice boundaries, security architecture, or distributed transactions.

IX. Function-Based Computing — Managed Execution Without Server Administration

A. Serverless architectures

Serverless architecture runs code on provider-managed infrastructure, typically through event-driven functions and managed backend services.

• Execution model: AWS Lambda, Azure Functions, or Google Cloud Functions invokes code in response to HTTP requests, queue messages, file uploads, or schedules.
• Scaling: The platform creates concurrent function instances as events arrive and reduces capacity when demand falls.
• Billing: Charges commonly reflect invocation count, execution duration, and allocated resources rather than continuously running servers.
• Example flow: Uploading an image to an object-storage bucket triggers a function that creates a thumbnail and writes it to another bucket.
• Advantages: Teams avoid server patching, gain automatic scaling, and can economically support intermittent workloads.
• Constraints:
  • Cold start: Initializing a new runtime can increase first-request latency.
  • Execution limit: Providers impose maximum duration, memory, payload, and concurrency limits.
  • Statelessness: Durable state must be externalized.
  • Vendor coupling: Event formats, identity systems, and managed-service APIs can hinder migration.
• Best fit: Event processing, scheduled automation, lightweight APIs, and bursty workloads suit functions better than long-running, latency-sensitive processes.

X. Industry Implementations — Cloud-Native Systems at Scale

A. Case studies on Netflix, Amazon, Uber, etc.

Large technology organizations demonstrate how cloud-native practices respond to scale, rapid change, and partial failure.

• Netflix: After major data-center disruption in 2008, Netflix accelerated migration to Amazon Web Services and decomposed its streaming platform into independently deployable services.
  • Resilience engineering: Chaos Monkey deliberately terminates instances to verify that systems tolerate failure.
  • Discovery and routing: Eureka provided service registration, while Ribbon historically supplied client-side load balancing.
  • Delivery: Spinnaker, developed at Netflix and later expanded with Google, supports multi-cloud continuous delivery.
• Amazon: Amazon’s retail platform evolved from a tightly integrated application toward service-oriented teams with APIs and strong ownership.
  • Service boundary: Capabilities such as catalog, ordering, and payments can evolve independently behind interfaces.
  • Cloud platform: AWS services—including EC2, S3, DynamoDB, Lambda, and Elastic Load Balancing—offer reusable infrastructure primitives.
  • Operational lesson: Independent services improve team autonomy but require monitoring, capacity management, and disciplined interface compatibility.
• Uber: Rapid global growth pushed Uber from an early monolithic system toward domain-oriented services supporting trips, pricing, maps, payments, and communications.
  • Real-time workload: Dispatch must connect riders and drivers using continuously changing location data.
  • Event processing: Streaming infrastructure distributes trip and location events to downstream systems.
  • Scaling challenge: Geographic partitioning, low-latency storage, and observability are essential during city-specific demand spikes.
• Shared conclusion: These systems combine service decomposition, automated deployment, discovery, load balancing, autoscaling, and failure isolation; their scale also shows that microservices require substantial platform engineering and organizational discipline.