Unit 2: AWS Storage; Amazon Virtual Private Cloud - Subjective Questions

Amazon S3 (Simple Storage Service) is an object storage service with a flat, non-hierarchical architecture.

Key architectural components:

• Buckets: Top-level containers for storing objects. Bucket names must be globally unique across all AWS accounts. Each bucket is created in a specific AWS Region.
• Objects: The fundamental entities stored in S3. An object consists of:
  • Data (the actual file content, up to 5 TB per object)
  • Metadata (key-value pairs describing the object)
  • Key (unique identifier of the object within a bucket)
• Keys: The full path/name used to retrieve an object. Although S3 is flat, prefixes with / simulate a folder structure (e.g., images/2024/photo.jpg).
• Regions: Data is stored in the chosen region and never leaves it unless explicitly transferred, helping with latency and compliance.

Access methods: Objects are accessed via REST APIs, AWS SDKs, CLI, or the Management Console. Each object has a unique URL of the form:

https://bucket-name.s3.region.amazonaws.com/key

Consistency model: S3 now provides strong read-after-write consistency for all operations, meaning any read immediately reflects the latest write.

Durability refers to the protection of data against loss, while Availability refers to the ability to access data when needed.

S3 Durability:

• Amazon S3 Standard provides 99.999999999% (11 nines) durability of objects over a year.
• This means if you store 10 million objects, you can on average expect to lose a single object once every 10,000 years.
• Achieved by automatically replicating data across a minimum of three Availability Zones (AZs) within a region.

S3 Availability (by storage class):

Why S3 is highly durable:

• Redundant storage across multiple AZs and devices.
• Checksums to detect and self-heal corrupted data.
• Versioning to protect against accidental deletion/overwrites.
• Regular integrity checks using MD5 checksums and CRCs.

The S3 Object Lifecycle is a set of rules that automate the management of objects by transitioning them between storage classes or deleting them after a defined period, helping to optimize cost.

Two types of lifecycle actions:

1. Transition actions: Move objects to cheaper storage classes after a specified time.

   • Example: Move objects to S3 Standard-IA after 30 days, then to S3 Glacier after 90 days.

2. Expiration actions: Permanently delete objects after a specified period.

   • Example: Delete objects 365 days after creation.

Example lifecycle rule (JSON):

{
"Rules": [{
"ID": "ArchiveAndDelete",
"Status": "Enabled",
"Transitions": [
{"Days": 30, "StorageClass": "STANDARD_IA"},
{"Days": 90, "StorageClass": "GLACIER"}
],
"Expiration": {"Days": 365}
}]
}

Benefits:

• Cost reduction by moving infrequently accessed data to cheaper tiers.
• Automated cleanup of expired or temporary data.
• Can target specific objects using prefixes and tags.

Ways of accessing S3 objects:

• AWS Management Console: Web-based GUI for uploading/downloading.
• AWS CLI: Command-line tool, e.g., aws s3 cp file.txt s3://my-bucket/.
• AWS SDKs: Programmatic access in languages like Python (boto3), Java, etc.
• REST API: Direct HTTP requests using object URLs.

Access control mechanisms:

• IAM Policies: Attached to users/roles to grant permissions on S3 resources.
• Bucket Policies: JSON-based resource policies attached to a bucket, controlling access at the bucket level.
• Access Control Lists (ACLs): Legacy, grant basic read/write to specific accounts.
• Pre-signed URLs: Time-limited URLs that grant temporary access to private objects without requiring AWS credentials.
• Block Public Access: Account/bucket-level setting to prevent accidental public exposure.

Example of a bucket policy allowing public read:

{
"Effect": "Allow",
"Principal": "",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my-bucket/"
}

Pre-signed URL use case: Allowing a user to download a private file for a limited time (e.g., 1 hour) by generating a signed URL.

Amazon S3 Glacier is a secure, low-cost storage service for data archiving and long-term backup, offering the same 11 nines (99.999999999%) durability as S3 Standard.

Glacier storage classes:

• S3 Glacier Instant Retrieval: Millisecond retrieval, for rarely accessed data needing immediate access.
• S3 Glacier Flexible Retrieval: Retrieval in minutes to hours.
• S3 Glacier Deep Archive: Lowest cost, retrieval in 12-48 hours.

Retrieval options (Flexible Retrieval):

Best suited use cases:

• Regulatory and compliance archives (e.g., financial, healthcare records).
• Long-term backups rarely accessed.
• Media asset preservation and scientific data archives.
• Replacing on-premises tape libraries.

Glacier trades retrieval latency for extremely low storage cost, making it ideal for cold data.

Amazon EFS is a fully managed, scalable network file system (NFS) for use with AWS Cloud services and on-premises resources. It provides a shared file system that can be mounted by multiple EC2 instances simultaneously.

Key features:

• Elastic: Automatically grows and shrinks as files are added/removed (petabyte scale).
• Shared access: Thousands of EC2 instances across multiple AZs can access it concurrently.
• Uses the NFSv4 protocol.
• Storage classes: Standard and Infrequent Access (IA).

Comparison with S3 and EBS:

Use cases: Content management, web serving, home directories, big data analytics requiring shared access.

Amazon FSx is a fully managed service that provides feature-rich, high-performance file systems built on popular file system technologies, removing the operational overhead of managing file servers.

FSx variants:

• Amazon FSx for Windows File Server:

  • Fully managed Windows-native file system built on Windows Server.
  • Supports SMB protocol, Active Directory integration, and NTFS.
  • Use case: Windows-based applications, home directories, enterprise file shares.

• Amazon FSx for Lustre:

  • High-performance file system optimized for HPC, machine learning, and media processing.
  • Delivers sub-millisecond latency and millions of IOPS.
  • Integrates seamlessly with S3.
  • Use case: High performance computing, big data analytics, ML training.

• Amazon FSx for NetApp ONTAP:

  • Fully managed ONTAP file system with multi-protocol support (NFS, SMB, iSCSI).
  • Use case: Migrating NetApp workloads to AWS.

• Amazon FSx for OpenZFS:

  • Managed file storage built on the OpenZFS file system (NFS).
  • Use case: Linux-based workloads needing ZFS features.

Benefits: Fully managed, high availability, automatic backups, and native compatibility with existing applications.

AWS Storage Gateway is a hybrid cloud storage service that connects on-premises environments with AWS cloud storage, providing seamless and secure integration with low-latency local caching.

Gateway types:

1. File Gateway (S3 File Gateway):

   • Presents a file interface (NFS/SMB) to store files as objects in Amazon S3.
   • Use case: File shares, backups, and content distribution to the cloud.

2. Volume Gateway:

   • Presents block storage volumes (iSCSI) backed by S3.
   • Two modes:
     • Cached volumes: Primary data in S3, frequently accessed data cached locally.
     • Stored volumes: Primary data stored locally, asynchronously backed up to S3 as EBS snapshots.
   • Use case: Backup and disaster recovery.

3. Tape Gateway (VTL):

   • Presents a virtual tape library interface to backup applications.
   • Virtual tapes stored in S3 and archived to Glacier.
   • Use case: Replacing physical tape infrastructure.

4. FSx File Gateway:

   • Provides low-latency on-premises access to Amazon FSx for Windows File Server.

Benefits: Low-latency local cache, standard protocols, encrypted data transfer, and cost-effective cloud backup.

A VPC CIDR (Classless Inter-Domain Routing) block defines the range of private IP addresses available within an Amazon VPC. It is specified using CIDR notation during VPC creation.

CIDR notation: Written as IP_address/prefix_length, where the prefix length indicates how many bits represent the network portion.

• Example: 10.0.0.0/16 means the first 16 bits are the network portion, leaving 16 bits for hosts.
• Number of IP addresses =
• For /16: addresses.

Allowed VPC CIDR block sizes:

• Minimum: /28 ( addresses)
• Maximum: /16 ( addresses)

Recommended private ranges (RFC 1918):

• 10.0.0.0/8
• 172.16.0.0/12
• 192.168.0.0/16

AWS reserved IPs: In every subnet, AWS reserves the first 4 and the last 1 IP addresses (5 total), so a /28 subnet usable addresses = .

Example: A VPC with CIDR 10.0.0.0/16 can be divided into subnets like 10.0.1.0/24, 10.0.2.0/24, etc.

Availability Zones (AZs): Physically separate, isolated data centers within an AWS Region, each with independent power, cooling, and networking. Deploying across multiple AZs provides high availability and fault tolerance.

Subnets: A subnet is a sub-range of a VPC's CIDR block that resides within a single Availability Zone. Resources like EC2 instances are launched into subnets.

• A subnet cannot span multiple AZs.
• One VPC can have multiple subnets across different AZs.

Public vs. Private Subnets:

Example design:

• Public subnet 10.0.1.0/24 in us-east-1a for web servers.
• Private subnet 10.0.2.0/24 in us-east-1b for databases.

A subnet becomes public when its route table has a route to an Internet Gateway.

An Elastic Network Interface (ENI) is a virtual network card that can be attached to an EC2 instance within a VPC. It represents a logical networking component.

ENI attributes:

• A primary private IPv4 address.
• One or more secondary private IPv4 addresses.
• One Elastic IP per private IP.
• A public IPv4 address, MAC address, and one or more security groups.

Primary vs. Secondary Private IP Addresses:

• Primary Private IP: Assigned automatically when the ENI is created and cannot be changed or removed for the lifetime of the ENI.
• Secondary Private IP: Additional IPs that can be added, moved, or removed. Useful for hosting multiple applications/websites on a single instance or for failover.

Attaching ENIs:

• A primary ENI (eth0) is created and attached automatically when an instance launches and cannot be detached.
• Additional ENIs can be attached (hot attach to running instances, warm attach to stopped instances, or cold attach at launch).
• Moving an ENI between instances allows failover — rerouting traffic by reassigning the ENI to a standby instance.

Use cases: Management networks, dual-homed instances, low-budget high-availability, and network/security appliances.

Internet Gateway (IGW):

• A horizontally scaled, redundant, and highly available VPC component that allows communication between resources in the VPC and the internet.
• Performs network address translation (NAT) for instances with public IPv4 addresses.
• Only one IGW can be attached to a VPC at a time.

Route Table:

• A set of rules (routes) that determine where network traffic from subnets is directed.
• Each subnet must be associated with a route table; the main route table is used by default.
• Each route specifies a destination (CIDR) and a target (IGW, NAT gateway, ENI, etc.).

How they work together to enable internet access:

1. Attach an Internet Gateway to the VPC.

2. Add a route in the subnet's route table directing internet-bound traffic to the IGW:

   Destination	Target
   10.0.0.0/16	local
   0.0.0.0/0	igw-xxxxxx

3. Ensure the instance has a public IP or Elastic IP.

4. Configure security groups and NACLs to allow the desired traffic.

The 0.0.0.0/0 → IGW route makes the subnet public, enabling bidirectional internet communication.

Both Security Groups and NACLs are virtual firewalls, but they operate at different levels and behave differently.

Security Group: Acts as a firewall at the instance (ENI) level.

NACL: Acts as a firewall at the subnet level.

Key differences:

When to use:

• Security Group: Fine-grained control for individual instances, e.g., allow SSH (port 22) only to a web server.
• NACL: Broad subnet-level rules, e.g., blocking a specific malicious IP range from an entire subnet, which security groups cannot do (no deny rules).

Best practice: Use security groups as the primary defense and NACLs as an additional layer of subnet protection (defense in depth).

An Elastic IP (EIP) is a static, public IPv4 address designed for dynamic cloud computing. It is allocated to your AWS account and remains yours until you release it.

Characteristics:

• Static: The IP address does not change (unlike auto-assigned public IPs).
• Can be remapped to any instance or ENI in your account.
• Associated with an account in a specific region.
• Enables failover by quickly reassigning the EIP to a replacement instance.

Elastic IP vs. Standard Public IP:

Use cases:

• Failover: Remap the EIP from a failed instance to a standby.
• Hosting services needing a consistent public IP (e.g., for DNS or firewall whitelisting).

Cost note: AWS charges for EIPs that are allocated but not associated with a running instance, to encourage efficient use.

AWS Global Accelerator is a networking service that improves the availability and performance of applications for global users by routing traffic through the AWS global network backbone instead of the public internet.

How it works:

• Provides two static anycast IP addresses that serve as a fixed entry point to your application.
• Traffic enters the AWS network at the nearest edge location to the user.
• Routes traffic over AWS's private, optimized backbone to the closest healthy endpoint.

Key features:

• Static anycast IPs: No DNS changes needed; one entry point to multiple regions.
• Automatic failover: Routes around unhealthy endpoints within seconds.
• Health checks: Continuously monitors endpoints (ALB, NLB, EC2, EIP).
• Traffic dials and weights: Control the proportion of traffic to each endpoint.

Performance improvements:

• Reduced latency and jitter by avoiding public internet congestion.
• Faster first-byte and improved throughput.

Global Accelerator vs. CloudFront:

• Global Accelerator: Improves performance for a wide range of applications over TCP/UDP; ideal for non-cacheable, gaming, IoT, VoIP.
• CloudFront: A CDN that caches content at edge locations; ideal for cacheable static/dynamic web content.

Network Address Translation (NAT) allows instances in a private subnet to initiate outbound connections to the internet (e.g., for software updates) while preventing inbound connections from the internet.

How it works:

• Private instances send internet-bound traffic to a NAT device located in a public subnet.
• The NAT device translates the private source IP to its own public IP and forwards the request.
• Return traffic is translated back and routed to the private instance.
• The private subnet route table includes: 0.0.0.0/0 → nat-gateway-id.

NAT Gateway vs. NAT Instance:

Key point: NAT devices must reside in a public subnet and require an Elastic IP. For high availability, deploy a NAT Gateway in each AZ.

AWS recommends NAT Gateway for most use cases due to its managed, scalable, and resilient nature.

VPC Peering is a networking connection between two VPCs that enables routing of traffic between them using private IP addresses, as if they were in the same network.

Characteristics:

• Can connect VPCs in the same or different regions (inter-region peering) and even across different AWS accounts.
• Traffic stays on the AWS private network (not the public internet).
• Requires updating route tables on both VPCs and ensuring non-overlapping CIDR blocks.

Setting up a peering connection:

1. One VPC owner requests a peering connection.
2. The other VPC owner accepts it.
3. Route tables are updated on both sides.
4. Security groups/NACLs are configured to allow traffic.

Limitations:

• No overlapping CIDR blocks allowed.
• Non-transitive: Peering is strictly between two VPCs only.
• No edge-to-edge routing (cannot use a peer's IGW, NAT, or VPN).

Non-transitive peering example:

• If VPC A is peered with VPC B, and VPC B is peered with VPC C, then VPC A cannot communicate with VPC C through VPC B.
• A direct peering connection between A and C must be created.

For many interconnected VPCs, AWS Transit Gateway is preferred to avoid complex full-mesh peering.

Hybrid Cloud Networking refers to connecting on-premises infrastructure (data centers/offices) with the AWS cloud to form a single, integrated network, enabling workloads to span both environments.

Main connectivity options:

1. AWS Site-to-Site VPN:

• Creates an encrypted IPsec tunnel over the public internet between on-premises and a VPC.
• Uses a Virtual Private Gateway (VGW) or Transit Gateway on the AWS side and a Customer Gateway on-premises.
• Pros: Quick to set up, low cost, encrypted.
• Cons: Variable latency and bandwidth (depends on the internet).

2. AWS Direct Connect (DX):

• A dedicated, private physical connection between on-premises and AWS.
• Bypasses the public internet entirely.
• Pros: Consistent low latency, high bandwidth (1-100 Gbps), reduced data transfer cost.
• Cons: Longer setup time, higher cost. Not encrypted by default (can combine with VPN).

3. AWS Transit Gateway:

• A central hub that connects multiple VPCs and on-premises networks in a hub-and-spoke model.
• Simplifies management of large, complex networks and supports transitive routing.

4. AWS VPN CloudHub:

• Connects multiple remote sites to AWS and allows them to communicate with each other via a hub-and-spoke VPN model.

Supporting services:

• AWS Storage Gateway: Hybrid storage integration.
• Route 53 Resolver: Hybrid DNS resolution.
• Direct Connect + VPN: Combines DX performance with VPN encryption for secure, reliable connectivity.

Common use cases:

• Cloud migration (gradual workload movement).
• Disaster recovery and backup to the cloud.
• Bursting on-premises workloads into AWS during peak demand.

Given: VPC CIDR = 10.0.0.0/16, subnet size = /24.

Step 1 — Total IPs in the VPC:

Number of addresses addresses.

Step 2 — Usable IPs in the VPC:

AWS reserves 5 IP addresses per subnet (not per VPC), so VPC-level usable count is discussed at subnet level below. The full /16 provides 65,536 addressable IPs.

Step 3 — Number of /24 subnets:

Bits available for subnetting bits.

Number of subnets subnets.

Examples: 10.0.0.0/24, 10.0.1.0/24, ... , 10.0.255.0/24.

Step 4 — Addresses per /24 subnet:

Total per subnet addresses.

Step 5 — Usable hosts per subnet (AWS reserved IPs):

AWS reserves the first 4 and last 1 IP in each subnet:

• 10.0.0.0 — Network address
• 10.0.0.1 — VPC router
• 10.0.0.2 — DNS server
• 10.0.0.3 — Reserved for future use
• 10.0.0.255 — Broadcast (reserved; broadcast not supported)

Usable hosts per subnet addresses.

Summary:

Amazon S3 offers multiple storage classes optimized for different access patterns, durability, and cost requirements. All classes offer the same 11 nines (99.999999999%) durability.

S3 Storage Classes:

Key cost-optimization principles:

• Hot data (frequently accessed): Use S3 Standard — higher storage cost but no retrieval fees.
• Warm data (infrequent): Use Standard-IA or One Zone-IA — lower storage cost but retrieval charges apply.
• Cold/archival data: Use Glacier classes — lowest storage cost, higher retrieval time/cost.
• Unknown patterns: Use Intelligent-Tiering, which automatically moves objects between tiers based on usage, avoiding manual management.

Lifecycle integration:

Combine storage classes with S3 Lifecycle policies to automatically transition objects as they age:

Standard → Standard-IA (after 30 days) → Glacier (after 90 days) → Deep Archive (after 180 days) → Delete (after 365 days)

One Zone-IA note: Stores data in a single AZ (not three), so it is ~20% cheaper than Standard-IA but less resilient — suitable only for easily reproducible data.

Conclusion: Matching the storage class to the actual access frequency, retrieval requirements, and resilience needs minimizes cost while meeting performance and durability objectives.