Unit 3: Cloud Architecture and Economics

I. Orientation — Architectural and Economic Foundations

Cloud computing delivers shared computing resources as metered, network-accessible services. Its architecture connects applications, service platforms, virtualized infrastructure, and physical data centers, while its economics depends on elasticity, resource pooling, automation, scale, and pay-per-use pricing.

• Defining properties:
  • On-demand self-service: Users provision resources such as virtual machines without direct provider intervention.
  • Broad network access: Services are accessed through standard mechanisms such as HTTPS, APIs, and VPNs.
  • Resource pooling: Multi-tenant infrastructure dynamically serves multiple customers while maintaining logical isolation.
  • Rapid elasticity: Capacity can expand or contract with workload demand.
  • Measured service: Consumption is recorded in units such as vCPU-hours, GB-months, requests, or transferred GB.
• Central architectural concern: Workloads must be mapped to compute, storage, and network resources while meeting performance, availability, security, and compliance requirements.
• Central economic concern: Organizations balance capital expenditure, operating expenditure, utilization, risk, and business agility rather than simply choosing the lowest unit price.
• Shared-responsibility principle: The provider secures and operates specified layers; the customer remains responsible for configuration, identities, data, and applications according to the service model.

II. Cloud Computing Stack — Layered Delivery of IT Resources

A. Cloud computing stack

The cloud computing stack organizes capabilities into layers with different levels of provider management and customer control.

• Physical and virtualization layer: Servers, disks, network switches, hypervisors, and container runtimes form the resource foundation.
  • A hypervisor maps virtual CPUs and memory to physical hardware.
  • Software-defined networking creates logical subnets, routes, and firewalls.
• Infrastructure as a Service (IaaS): The provider supplies compute, storage, and networking; the customer manages operating systems, middleware, and applications.
  • Concrete resources: Virtual machines, block volumes, virtual networks, and load balancers.
  • Typical use: Migrating a three-tier application while retaining operating-system control.
• Platform as a Service (PaaS): The provider also manages operating systems, runtimes, scaling mechanisms, and middleware.
  • Concrete resources: Managed databases, application runtimes, message queues, and function platforms.
  • Trade-off: Faster deployment but reduced low-level control and greater portability concerns.
• Software as a Service (SaaS): The provider operates the complete application, while users configure features and manage their data.
  • Examples: Web-based email, customer relationship management, and collaboration systems.
• Management plane: APIs, identity services, monitoring, billing, orchestration, and policy enforcement span every layer.
• Layering implication: Moving from IaaS to SaaS generally decreases customer administration but increases dependence on provider interfaces and service limits.

III. Workload Distribution — Placing and Balancing Demand

A. Workload distribution

Workload distribution assigns requests and computational tasks across resources to improve performance, resilience, and utilization.

• Load-balancing methods:
  1. Static methods: Round-robin and weighted round-robin distribute requests using predetermined rules.
  2. Dynamic methods: Least-connections, observed latency, or queue length reflect current server conditions.
• Distribution levels:
  • Global: DNS or global traffic managers direct users to regions based on latency, health, or geography.
  • Regional: Layer 4 or Layer 7 load balancers distribute traffic among availability zones and instances.
  • Application: Queues allocate asynchronous jobs to worker processes.
• Stateless design: Keeping session state in a shared database or cache allows any healthy instance to process the next request.
• Data locality: Computation should be placed near large datasets because moving 10 TB across regions may cost more and take longer than launching compute beside the data.
• Resilience: Health checks remove failed instances, while multi-zone placement reduces dependence on one facility.
• Limitation: Uneven task sizes can produce stragglers even when each worker receives the same number of jobs.

IV. Capacity Planning — Matching Resources to Demand

A. Capacity planning

Capacity planning estimates the resources required to satisfy present and future workloads at an acceptable cost and service level.

• Demand measurements: Relevant metrics include requests per second, concurrent users, CPU utilization, memory usage, IOPS, throughput, and response-time percentiles.
• Basic capacity model:

Required capacity = Peak demand × (1 + Safety margin)
Instance count = ceil(Required capacity / Capacity per instance)

• Peak demand: Maximum expected workload per measurement interval.
• Safety margin: Fraction reserved for uncertainty or failure.
• Capacity per instance: Tested sustainable workload of one instance.
  • Worked example: For 8,000 requests/second, a 25% margin, and 1,000 requests/second per instance, the requirement is ceil(8,000 × 1.25 / 1,000) = 10 instances.
  • Forecasting inputs: Historical trends, seasonal peaks, product launches, growth rates, and recovery requirements should be considered separately.
  • Scaling choice:
    1. Vertical scaling: Adds CPU or memory to one machine but encounters hardware limits.
    2. Horizontal scaling: Adds instances and improves fault tolerance, provided the workload can be partitioned.
  • Validation: Load tests must identify bottlenecks before forecasts are converted into production capacity.

V. Cloud Bursting — Extending Private Capacity Temporarily

A. Cloud bursting

Cloud bursting moves excess workload from a private environment to a public cloud when local capacity reaches a defined threshold.

• Operating sequence: Monitoring detects saturation, orchestration provisions public resources, traffic is redirected, and temporary capacity is removed after demand falls.
• Trigger example: Bursting may begin when average CPU exceeds 75% for ten minutes, although queue depth or response time is often a better application-level signal.
• Suitable workloads: Batch rendering, simulations, testing, and stateless web processing can be expanded without transferring tightly coupled state.
• Requirements: Compatible images, automated deployment, secure connectivity, synchronized identities, and portable application configurations are essential.
• Economic benefit: An organization can retain predictable baseline demand privately and rent public capacity only for short-lived peaks.
• Constraints:
  • Data gravity: Large datasets are slow or expensive to transfer.
  • Licensing: Software licenses may prohibit or raise the cost of external execution.
  • Latency and compliance: Remote processing may violate response-time or data-location requirements.
• Distinction: Cloud bursting is temporary hybrid scaling, not ordinary disaster recovery or permanent multi-cloud deployment.

VI. Disk Provisioning — Allocating Virtual Storage

A. Disk provisioning

Disk provisioning determines how logical storage capacity is allocated from physical storage pools.

• Thick provisioning: The full logical capacity is reserved when a volume is created.
  • Advantage: Predictable capacity and lower risk of pool exhaustion.
  • Cost: A reserved 1 TB volume consumes 1 TB even if only 200 GB contains data.
• Thin provisioning: Physical blocks are allocated as data is written.
  • Advantage: Higher utilization and rapid creation of large logical volumes.
  • Risk: Overcommitment can exhaust the backing pool unless alerts and expansion policies exist.
• Performance dimensions:
  • IOPS: Number of input/output operations completed each second.
  • Throughput: Data transferred per second, commonly measured in MB/s.
  • Latency: Time required for an operation, commonly measured in milliseconds.
• Disk types: Block storage supports filesystems and databases; object storage uses API-addressed objects; file storage exposes shared directory structures.
• Operational controls: Snapshots, replication, encryption, lifecycle policies, and backup schedules address recovery and governance.
• Limitation: A snapshot is not necessarily an independent backup if it shares the same account, region, or underlying failure domain.

VII. Service-Oriented Architecture — Systems Composed from Services

A. Service-Oriented Architecture (SOA)

SOA structures applications as reusable, network-accessible services with explicit contracts and loosely coupled consumers.

• Service contract: An interface specifies operations, message formats, endpoints, and policies; examples include WSDL-described SOAP services and REST-style HTTP APIs.
• Loose coupling: Consumers depend on the contract rather than the service’s programming language or internal database.
• Core roles: A service provider implements functionality, a consumer invokes it, and a registry or catalog supports discovery and governance.
• Communication styles:
  1. Synchronous: A client waits for an HTTP response, making latency and availability immediately visible.
  2. Asynchronous: A queue or event broker decouples producers from consumers and absorbs temporary demand spikes.
• Governance: Versioning, authentication, authorization, schema management, observability, and retirement policies prevent uncontrolled service proliferation.
• Cloud relationship: Managed APIs, queues, functions, and containers supply practical infrastructure for implementing service compositions.
• Limitations: Distributed calls introduce network failures, tracing complexity, serialization overhead, and cascading-failure risks; timeouts and circuit breakers are therefore required.

VIII. Service Level Agreements — Measurable Service Commitments

A. Service Level Agreements (SLAs)

An SLA is a formal agreement defining measurable service targets, responsibilities, exclusions, and remedies.

• Common service-level indicators: Availability, latency, error rate, durability, recovery time, recovery point, support response, and throughput.
• Availability calculation:

Availability (%) = (Total time − Downtime) / Total time × 100

• Total time: Length of the measurement window.
• Downtime: Time classified as unavailable under the agreement.
  • Concrete interpretation: At 99.9% monthly availability over 30 days, the implied downtime allowance is approximately 43.2 minutes.
  • Agreement components: Scope, measurement source, maintenance exclusions, customer duties, escalation procedures, reporting period, and service credits must be explicit.
  • SLA versus SLO: The SLA is contractual; a service-level objective is the operational target, often set stricter than the contractual threshold.
  • Composite availability: Serial dependencies reduce end-to-end reliability; two independent services each available 99.9% yield approximately 0.999 × 0.999 = 99.8001%.
  • Limitation: Service credits compensate only part of the fee and rarely cover lost revenue, reputation, or data.

IX. Laws of Cloudonomics — Economic Principles of Cloud Value

A. Laws of cloudonomics

The laws of cloudonomics, associated with Joe Weinman, explain why pooled, scalable, on-demand resources can create economic advantage.

• Utility premium: On-demand capacity may have a higher unit price yet cost less overall because customers avoid paying continuously for idle peak capacity.
• Forecasting limitation: Instant provisioning reduces the financial consequences of inaccurate long-term demand forecasts.
• Demand aggregation: The peak of combined workloads is generally no greater than the sum of their separate peaks.
• Statistical smoothing: Diverse customers are unlikely to reach maximum demand simultaneously, improving provider utilization.
• Scale economics: Fixed costs for facilities, automation, security, and expertise are spread across many users.
• Resource pooling: Large fleets improve purchasing power, operational specialization, and failure absorption.
• Latency constraint: Distance still matters; interactive workloads may require edge or regional placement despite centralized scale.
• Geographic dispersion: More distributed facilities can reduce user latency but increase replication and coordination costs.
• Risk diversification: Multiple failure domains reduce concentration risk when architectures avoid shared dependencies.
• Inertia: Data gravity, skills, contracts, and legacy integration create switching costs, so technically movable workloads may remain in place.
• Qualification: These principles describe tendencies, not guarantees; poor governance or persistent high utilization can make owned infrastructure cheaper.

X. Cost Estimation — Quantifying Total Cloud Expenditure

A. Cost estimation

Cost estimation converts workload assumptions and provider prices into a realistic total cost of ownership.

• Core model:

Monthly cost = Compute + Storage + Network + Managed services
             + Software + Operations + Risk allowance

• Compute: Instance-hours, vCPU-seconds, function invocations, or container resources.
• Storage: GB-months, IOPS, requests, snapshots, and backup retention.
• Network: Internet egress, inter-region transfer, gateways, and load balancing.
  • Pricing models: On-demand pricing offers flexibility; reservations or savings commitments reduce rates for predictable usage; spot capacity discounts interruptible work.
  • Utilization effect: A server costing $600 monthly at 30% utilization has an effective utilized-capacity cost of $600 / 0.30 = $2,000 per fully utilized equivalent month.
  • Hidden costs: Migration, data transfer, observability, security tools, support plans, retraining, and application refactoring must be included.
  • Forecasting technique: Estimate baseline, expected, and peak scenarios rather than relying on one deterministic figure.
  • Unit economics: Cost per customer, transaction, API call, or rendered job connects infrastructure spending to business output.

XI. Economic Strategy — Aligning Cloud Spending with Business Value

A. Economic strategy

Economic strategy selects sourcing, architecture, pricing, and governance choices that maximize value under cost, risk, and agility constraints.

• Workload segmentation: Stable workloads may suit commitments or private infrastructure; uncertain and temporary workloads favor elastic on-demand services.
• FinOps cycle: Teams continuously inform, optimize, and operate using tagging, budgets, allocation reports, forecasts, and accountability.
• Optimization controls: Rightsizing, autoscaling, scheduling non-production shutdowns, storage tiering, commitment coverage, and egress-aware placement reduce waste.
• Build-versus-buy decision: Managed services increase unit price but may reduce staffing, patching, downtime, and time to market.
• Portfolio approach: Organizations can combine reserved baseline capacity, on-demand variability, and spot-based batch processing rather than selecting one pricing model.
• Risk-adjusted evaluation:

Expected cost = Direct cost + Σ(Probability of event × Financial impact)

• Direct cost: Normal operating expenditure.
• Event: Outage, breach, migration delay, or vendor price change.
• Financial impact: Estimated loss if that event occurs.
  • Strategic objective: The lowest bill is not always optimal; the preferred design minimizes total economic cost while satisfying SLA, security, compliance, resilience, and growth requirements.