Unit 5: Resiliency, Monitoring, and Automation - Subjective Questions

Amazon CloudWatch is an AWS monitoring and observability service that collects operational data from AWS resources, applications, and on-premises systems.

Its major components are:

• Metrics: Time-ordered numerical data, such as EC2 CPU utilization, network traffic, or Lambda invocation count.
• Namespaces: Containers used to organize related metrics, such as AWS/EC2 and AWS/Lambda.
• Dimensions: Name-value pairs that identify a metric, such as an EC2 instance ID.
• CloudWatch Logs: Collects, stores, searches, and analyzes logs from applications and AWS services.
• Alarms: Monitor a metric or metric-math expression and perform an action when a threshold is crossed.
• Dashboards: Provide customizable visualizations of metrics and alarms.
• CloudWatch Agent: Collects operating-system-level metrics and logs, including memory and disk usage.
• CloudWatch Synthetics and Application Insights: Help monitor endpoint availability and application health.

CloudWatch supports proactive monitoring, troubleshooting, automated recovery, scaling, and operational visibility.

The three CloudWatch features serve different but related purposes:

• CloudWatch metrics: Numerical measurements recorded over time. For example, the CPUUtilization metric shows the percentage of EC2 CPU capacity being used.
• CloudWatch Logs: Text-based records generated by applications, operating systems, and AWS services. For example, an application can send error messages and request records to a CloudWatch log group.
• CloudWatch alarms: Rules that evaluate metrics or metric-math expressions. For example, an alarm can enter the ALARM state when average CPU utilization exceeds 80% for three consecutive periods.

A common workflow is:

1. An application writes error information to CloudWatch Logs.
2. A metric filter converts matching error records into a custom metric.
3. A CloudWatch alarm monitors that metric.
4. The alarm sends an Amazon SNS notification or triggers an automated action.

Thus, logs provide detailed events, metrics provide measurable trends, and alarms enable notification or automation.

A CloudWatch alarm is configured by selecting a metric, statistic, period, threshold, comparison operator, evaluation periods, and action.

Important configuration elements:

• Metric: The value to monitor, such as request latency.
• Statistic: A calculation such as Average, Sum, Minimum, Maximum, or percentile.
• Period: The time interval over which data is aggregated.
• Threshold: The value that separates normal and abnormal behavior.
• Evaluation periods: The number of recent periods examined by the alarm.
• Datapoints to alarm: The number of breaching datapoints required within the evaluation periods.

Alarm states:

• OK: The metric is within the defined limit.
• ALARM: The threshold condition has been met.
• INSUFFICIENT_DATA: There is not enough data to determine the state.

Missing data may be treated as breaching, not breaching, missing, or ignored. The appropriate choice depends on the workload. For example, missing heartbeat data should normally be treated as breaching, while the absence of an error metric may be treated as not breaching.

Alarm actions can notify an SNS topic, modify an Auto Scaling group, invoke automation, or perform supported EC2 recovery actions.

A monitoring strategy for a multi-tier application should cover infrastructure, application behavior, user experience, and business outcomes.

Web and load-balancing tier:

• Monitor request count, response time, HTTP 4xx and 5xx errors, and healthy target count.
• Create alarms for high latency, excessive server errors, and unhealthy targets.

Compute tier:

• Monitor EC2 CPU utilization, network traffic, status-check failures, memory usage, and disk usage.
• Install the CloudWatch Agent because memory and disk utilization are not standard EC2 metrics.

Database tier:

• Monitor database connections, CPU, free storage, read/write latency, and replication lag where applicable.

Application monitoring:

• Send application and operating-system logs to CloudWatch Logs.
• Use structured logs and metric filters to count exceptions or failed transactions.
• Add custom metrics for business indicators such as successful orders.

Visualization and response:

• Build CloudWatch dashboards for centralized visibility.
• Use alarms and Amazon SNS for notifications.
• Connect alarms to Auto Scaling or automated remediation.
• Apply least-privilege IAM permissions and appropriate log-retention policies.

This layered approach helps detect failures, identify their causes, and respond before users are significantly affected.

Amazon EC2 Auto Scaling automatically maintains and adjusts the number of EC2 instances according to workload demand, health status, and configured capacity limits.

Its main elements are:

• Launch template: Defines how new instances are launched. It may specify the AMI, instance type, security groups, storage, IAM role, user data, and purchasing options.
• Auto Scaling group: A logical collection of EC2 instances. It defines the minimum, desired, and maximum capacity and can distribute instances across multiple Availability Zones.
• Scaling policy: Defines when and how capacity changes. It may add instances during high demand and remove them when demand decreases.

For example, an Auto Scaling group may have a minimum capacity of 2, desired capacity of 4, and maximum capacity of 10. A target tracking policy can attempt to maintain average CPU utilization at 50%.

Auto Scaling improves availability by replacing unhealthy instances and improves cost efficiency by matching capacity to demand.

The main Auto Scaling approaches are:

• Target tracking scaling: Maintains a metric near a target value, such as keeping average CPU utilization at 50%. It is simple to configure and resembles a thermostat.
• Step scaling: Changes capacity by different amounts according to the severity of an alarm breach. For example, add two instances at moderate load and four at very high load.
• Simple scaling: Performs one scaling adjustment after an alarm and waits for a cooldown period. It is less responsive than step scaling and is generally used for basic scenarios.
• Scheduled scaling: Adjusts capacity at predefined times. It is suitable for predictable events, such as increasing capacity before office hours.
• Predictive scaling: Uses historical patterns and forecasts to schedule capacity before expected demand arrives.

Target tracking is suitable for continuously changing demand, step scaling offers precise reaction to different load levels, scheduled scaling handles known timing, and predictive scaling handles recurring patterns. These methods may be combined, provided that their interactions and capacity limits are carefully tested.

An Auto Scaling group continuously evaluates the health of its instances and attempts to maintain the desired capacity.

Health-check mechanisms:

• EC2 status checks detect underlying system or instance-level problems.
• Elastic Load Balancing health checks determine whether an instance can correctly serve application requests.
• Custom health information may also be submitted through supported automation.

When an instance is marked unhealthy, Auto Scaling terminates it and launches a replacement according to the launch template.

Health-check grace period:

• Gives a newly launched instance time to initialize before health-check failures are acted upon.
• Prevents premature replacement while software is being installed or the application is starting.

Lifecycle hooks:

• Pause an instance during launch or termination.
• Allow bootstrapping, configuration, registration, log collection, or connection draining to complete.
• Continue when the lifecycle action is completed or when its timeout expires.

Correct health checks should test actual application readiness rather than only verifying that the operating system is running.

The basic number of instances required for the current workload is:

A 25% spare-capacity requirement means provisioning:

Therefore, the Auto Scaling group should have a desired capacity of at least 10 instances for this workload.

A suitable implementation would include:

• Configure the desired capacity as 10 before the known load begins.
• Set the minimum capacity according to the minimum availability requirement, such as 2 or more instances across multiple Availability Zones.
• Set a maximum capacity above 10 so that unexpected demand can be handled.
• Use a target tracking policy based on request count per target, CPU utilization, or another representative application metric.
• Attach the group to a load balancer target group.
• Configure health checks so failed instances are automatically replaced.

The calculation assumes that instances have equal processing capacity and that traffic is distributed reasonably evenly. Real deployments should validate the estimate through load testing and account for startup time, uneven requests, downstream bottlenecks, and Availability Zone failure.

AWS provides several Elastic Load Balancing options:

• Application Load Balancer (ALB): Operates mainly at Layer 7 for HTTP and HTTPS traffic. It supports host-based routing, path-based routing, redirects, WebSockets, and integration with containerized applications.
• Network Load Balancer (NLB): Operates mainly at Layer 4 for TCP, UDP, and TLS traffic. It is designed for very high performance, low latency, static IP requirements, and sudden traffic growth.
• Gateway Load Balancer (GWLB): Helps deploy, scale, and manage virtual network appliances such as firewalls and intrusion-detection systems. It transparently forwards traffic through appliance fleets.
• Classic Load Balancer (CLB): A previous-generation service that supports basic Layer 4 and Layer 7 balancing. It is mainly retained for legacy workloads.

An ALB is preferred for modern web applications requiring content-based routing. An NLB is appropriate for high-performance network traffic. A GWLB is selected for security-appliance insertion. New applications should generally use ALB, NLB, or GWLB instead of CLB.

A highly available design places a load balancer in front of an Auto Scaling group distributed across at least two Availability Zones.

Operation:

1. Clients send requests to the load balancer's DNS name.
2. The load balancer accepts traffic through listeners and forwards it to healthy instances in a target group.
3. The Auto Scaling group launches instances across multiple Availability Zones.
4. Load balancer health checks remove unhealthy targets from traffic distribution.
5. Auto Scaling replaces failed instances and changes capacity as demand varies.

Important configuration decisions:

• Use public subnets for an internet-facing load balancer and private subnets for application instances.
• Configure listener rules, target groups, security groups, and TLS certificates correctly.
• Use application-level health-check paths such as /health.
• Ensure the minimum capacity can tolerate the failure of an instance or Availability Zone.
• Keep application servers stateless or store sessions in a shared service.
• Monitor healthy-host count, response time, rejected connections, and HTTP errors.

This architecture prevents a single instance from becoming a single point of failure and supports both horizontal scaling and fault recovery.

These features influence load-balancer availability and traffic distribution:

• Health checks: Periodically test registered targets using a configured protocol, port, and path. A target receives normal traffic only after meeting the healthy-threshold requirement.
• Cross-zone load balancing: Allows load-balancer nodes to distribute traffic across registered targets in enabled Availability Zones. Its behavior and configuration depend on the load balancer type.
• Connection draining or deregistration delay: Gives in-flight requests time to complete after a target is deregistered or begins termination. This reduces errors during deployments and scale-in operations.
• Session stickiness: Routes requests from a client to the same target for a period, usually by using a cookie for an ALB. It can help stateful applications but may produce uneven load and complicate scaling.

For resilient cloud applications, stateless design is generally preferred over stickiness. Health-check paths should verify application readiness without performing expensive operations, and deregistration delay should be coordinated with Auto Scaling lifecycle actions and application shutdown behavior.

Amazon Route 53 supports several routing policies:

• Simple routing: Returns one or more records without specialized routing logic. It is suitable for a basic single-resource application.
• Weighted routing: Sends configurable percentages of traffic to different resources. It is useful for canary releases, A/B testing, and gradual migration.
• Latency-based routing: Directs users to the AWS Region expected to provide the lowest latency.
• Failover routing: Sends traffic to a primary resource and switches to a secondary resource when the primary is unhealthy.
• Geolocation routing: Routes traffic according to the geographic location of users, such as country or continent.
• Geoproximity routing: Routes according to the location of resources and users, with optional bias to expand or shrink a resource's traffic area.
• Multivalue answer routing: Returns multiple healthy records selected by Route 53, enabling simple DNS-level distribution.
• IP-based routing: Routes according to the client's IP address range, allowing network-origin-specific responses.

The policy should be chosen according to availability, performance, compliance, testing, and traffic-management requirements.

A two-Region failover design can use one Region as the primary site and another as the secondary site.

Design steps:

1. Deploy the application stack in both Regions, including load balancers and sufficient compute capacity.
2. Create a Route 53 health check for the primary endpoint or use supported alias-target health evaluation.
3. Create a primary failover record pointing to the primary Region's load balancer.
4. Create a secondary failover record pointing to the standby Region's load balancer.
5. Associate health evaluation with the primary record so Route 53 stops returning it when it becomes unhealthy.
6. Select a reasonable DNS TTL to balance caching efficiency and failover speed.
7. Replicate application data and configuration according to the required recovery point objective.
8. Test failure and recovery procedures regularly.

DNS failover is not instantaneous because recursive resolvers and clients may cache records. The secondary Region must also have enough capacity and access to current data. Monitoring should alert operators even when failover occurs automatically.

Route 53 health checks test endpoint availability or evaluate calculated health-check conditions. Routing policies can use health status to avoid returning unhealthy resources.

Alias records are Route 53-specific records that can point the zone apex or another name to selected AWS resources, such as an Application Load Balancer or CloudFront distribution. Unlike a traditional CNAME, an alias can be used at the root of a hosted zone and generally does not require a separate DNS query to resolve the AWS target.

Time to Live (TTL) specifies how long a DNS resolver may cache a record. Its impact is:

• A lower TTL can allow clients to obtain updated failover answers sooner.
• A higher TTL reduces DNS query volume and improves caching efficiency.
• A low TTL does not guarantee immediate failover because clients, operating systems, and resolvers may apply their own caching behavior.

Effective failover requires correct health-check thresholds, healthy backup resources, tested routing records, and realistic expectations about DNS caching.

Infrastructure as Code (IaC) is the practice of defining and managing infrastructure through machine-readable files rather than manual console operations. AWS CloudFormation implements IaC by creating and managing AWS resources as stacks.

A CloudFormation template is written in YAML or JSON and may contain:

• AWSTemplateFormatVersion: Identifies the template format version.
• Description: Explains the template's purpose.
• Metadata: Stores additional template information.
• Parameters: Accepts deployment-time input values.
• Mappings: Defines static lookup tables.
• Conditions: Controls whether resources or properties are created.
• Resources: Declares AWS resources and is the only required main section.
• Outputs: Returns useful values, such as a load balancer DNS name.
• Rules: Validates combinations of parameter values in supported scenarios.

CloudFormation provides repeatability, version control, consistency, reviewability, and automated dependency handling. The same template can be reused across development, testing, and production with different parameter values.

A CloudFormation stack is a collection of AWS resources managed as one unit.

Stack lifecycle operations:

• Creation: CloudFormation validates the template, determines resource dependencies, and creates resources in the required order.
• Update: The template or parameter values are changed. Depending on the property, a resource may be updated in place, interrupted, or replaced.
• Rollback: If creation or update fails, CloudFormation normally attempts to return the stack to its previous stable state.
• Deletion: Managed resources are deleted unless a retention policy or deletion policy preserves them.
• Drift detection: Compares actual resource configuration with the expected configuration in the template and reports differences.
• Change sets: Preview proposed additions, modifications, replacements, and deletions before an update is executed.

Operational safeguards include reviewing change sets, protecting critical stacks from accidental deletion, using DeletionPolicy for important data, validating templates, and avoiding manual changes that cause drift.

CloudFormation provides mechanisms for building modular and dynamic templates.

Intrinsic functions calculate values during stack processing. Common examples include:

• Ref to obtain a parameter value or resource identifier.
• Fn::GetAtt to retrieve a resource attribute.
• Fn::Sub to substitute values into a string.
• Fn::Join to combine values.
• Fn::FindInMap to retrieve a mapping value.
• Fn::If to select a value according to a condition.

Resource dependencies:

• CloudFormation automatically detects many dependencies when one resource references another.
• DependsOn can define an explicit creation or deletion order when an implicit reference is insufficient.

Nested stacks:

• Divide a large architecture into reusable child templates.
• Improve modularity and reduce duplication.

Cross-stack references:

• One stack exports an output, and another stack imports it using Fn::ImportValue.
• They are useful for sharing resources such as VPC IDs across stacks.

Overusing cross-stack references can tightly couple stack lifecycles, so interfaces between stacks should be designed carefully.

AWS Quick Starts are automated reference deployments that help users deploy selected workloads and technologies on AWS by following architecture and security practices developed by AWS and AWS Partners.

A Quick Start commonly includes:

• A reference architecture.
• CloudFormation templates.
• A deployment guide.
• Parameters for customization.
• Scripts or configuration artifacts required by the workload.

Deployment process:

1. Review the architecture, prerequisites, costs, and security considerations.
2. Select whether to deploy into a new or existing AWS environment when supported.
3. Provide template parameters.
4. Launch the CloudFormation stacks.
5. Monitor stack events and validate the deployed workload.

Benefits:

• Faster and more consistent deployment.
• Reduced manual configuration.
• Reusable, documented architecture.
• A useful starting point for production designs.

Limitations:

• Templates may require customization for organizational policies.
• Deployed resources incur charges.
• Users remain responsible for security, governance, patching, data protection, and operational validation.
• Updates and compatibility should be reviewed before deployment.

Amazon Q Developer is a generative AI assistant that can support software development and AWS-related operational work. It can assist infrastructure automation by:

• Suggesting CloudFormation, AWS CDK, scripts, and configuration code.
• Explaining unfamiliar templates and resource properties.
• Helping identify syntax errors or deployment failures.
• Proposing tests, documentation, and implementation steps.
• Assisting with refactoring and repetitive coding tasks.
• Providing AWS service guidance within supported development and management environments.

Amazon Q Developer should be treated as an assistant rather than an autonomous authority. Important precautions include:

• Review all generated code before use.
• Validate templates with appropriate tools and test them in a non-production account.
• Preview infrastructure changes with CloudFormation change sets.
• Check IAM permissions for least privilege.
• Scan for security issues, exposed secrets, destructive actions, and unexpected costs.
• Verify service limits, Region support, and organizational policies.
• Maintain human approval for production deployments.

AI assistance can accelerate work, but accountability for correctness, security, and deployment outcomes remains with the user and organization.

A resilient public web application can be designed as follows:

Traffic and DNS layer:

• Use Route 53 to host the application's DNS records.
• Create an alias record pointing to an internet-facing Application Load Balancer.
• For multi-Region resilience, use failover or latency-based routing with health evaluation.

Load-balancing and compute layer:

• Place the Application Load Balancer in public subnets across multiple Availability Zones.
• Place application instances in private subnets.
• Register an EC2 Auto Scaling group with the load balancer's target group.
• Configure minimum, desired, and maximum capacities and use target tracking scaling.
• Use application-level health checks and deregistration delay.

Monitoring and response:

• Send application and system logs to CloudWatch Logs.
• Monitor latency, errors, CPU, request count, and healthy-target count.
• Create dashboards and alarms.
• Send alerts through Amazon SNS and use alarms to support scaling or remediation.

Infrastructure automation:

• Define networking, IAM roles, security groups, load balancers, Auto Scaling, monitoring, and DNS in CloudFormation.
• Use parameters, nested stacks, outputs, change sets, rollback, and drift detection.
• Store templates in version control and deploy them through a reviewed CI/CD process.

Quick Starts and Amazon Q Developer:

• Use a relevant Quick Start as a reviewed reference or deployment foundation where appropriate.
• Use Amazon Q Developer to explain, generate, test, or troubleshoot automation artifacts.
• Validate all generated content and require approval before production execution.

Resilience considerations:

• Avoid storing session state on individual instances.
• Use Multi-AZ or replicated data services.
• Apply least-privilege IAM policies, encryption, backups, and recovery testing.
• Test instance, Availability Zone, and Region failure scenarios.

The resulting architecture combines fault isolation, automatic replacement, elastic capacity, observable operations, DNS-based recovery, and repeatable deployment.