Back to Subject
Practice MCQ

Unit 2 - Notes

INT331

Unit 2: DevOps on cloud with basic LINUX commands

1. Essentials of Cloud Computing

Cloud computing is the on-demand delivery of IT resources (compute, storage, databases, networking, software) over the Internet with pay-as-you-go pricing. Instead of buying, owning, and maintaining physical data centers, organizations access technology services from a cloud provider.

Key Characteristics (NIST Model)

  1. On-Demand Self-Service: Users can provision computing capabilities (server time, network storage) automatically without requiring human interaction with each service provider.
  2. Broad Network Access: Capabilities are available over the network and accessed through standard mechanisms (mobile phones, tablets, laptops, and workstations).
  3. Resource Pooling: The provider’s computing resources are pooled to serve multiple consumers (Multi-tenancy) using a multi-tenant model, with different physical and virtual resources dynamically assigned according to demand.
  4. Rapid Elasticity: Capabilities can be elastically provisioned and released, in some cases automatically, to scale rapidly outward and inward commensurate with demand (Scalability).
  5. Measured Service: Cloud systems automatically control and optimize resource use by leveraging a metering capability appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts).

Service Models

  • IaaS (Infrastructure as a Service): Provides fundamental building blocks (virtual machines, networking, storage). The user manages the OS and applications; the provider manages the physical hardware.
    • Examples: AWS EC2, Google Compute Engine, Microsoft Azure Virtual Machines.
  • PaaS (Platform as a Service): Provides a platform allowing customers to develop, run, and manage applications without the complexity of building and maintaining the infrastructure associated with developing and launching an app.
    • Examples: AWS Elastic Beanstalk, Heroku, Google App Engine.
  • SaaS (Software as a Service): Completed software that is run and managed by the service provider. Users access it via a web browser.
    • Examples: Gmail, Salesforce, Dropbox, Slack.

Deployment Models

  • Public Cloud: Resources are owned and operated by a third-party cloud service provider and delivered over the public Internet.
  • Private Cloud: Computing resources used exclusively by a single business or organization. It can be physically located on the company’s on-site data center or hosted by a third-party service provider.
  • Hybrid Cloud: Combines public and private clouds, bound together by technology that allows data and applications to be shared between them.
  • Community Cloud: Shared by several organizations and supports a specific community that has shared concerns (e.g., mission, security requirements, policy).

2. Architecture of Cloud and Virtualization

Cloud Architecture

Cloud architecture is generally divided into two sections:

  1. Front End: The client side (user interface). It includes the client's network and applications used to access the cloud (e.g., a Web Browser).
  2. Back End: The cloud itself. It comprises the servers, storage, operating systems, and management mechanisms that provide the service.
  3. Middleware: Connects the front end and back end. It manages the coordination of the system.

Virtualization

Virtualization is the fundamental technology that powers cloud computing. It creates a simulated, or "virtual," version of something, such as an operating system, a server, a storage device, or network resources.

  • How it works: Software allows a single piece of hardware to run multiple operating systems (virtual machines) simultaneously.
  • The Hypervisor: The software layer that coordinates the VMs. It sits between the hardware and the virtual machine and assigns hardware resources (CPU, RAM) to the VMs.

Types of Hypervisors

  • Type 1 (Bare Metal): Installs directly on the hardware. Highly efficient. Used in enterprise data centers (e.g., VMware ESXi, Microsoft Hyper-V, Xen).
  • Type 2 (Hosted): Runs as an application inside an existing OS. Used for personal desktops (e.g., Oracle VirtualBox, VMware Workstation).

Types of Virtualization

  1. Server Virtualization: Partitioning a physical server into smaller virtual servers.
  2. Network Virtualization: Reproducing a physical network in software (Software-Defined Networking - SDN).
  3. Storage Virtualization: Pooling physical storage from multiple network storage devices into a single storage device.
  4. Desktop Virtualization: Deploying a desktop environment on a remote server (VDI).

3. Different Cloud Providers

While there are many providers, the market is dominated by a few major players known as "Hyperscalers."

  1. Amazon Web Services (AWS):
    • Market Leader. Launched in 2006.
    • Known for the broadest service portfolio and global reach.
  2. Microsoft Azure:
    • Strong integration with Microsoft enterprise tools (Windows Server, Active Directory, SQL Server).
    • Popular among large enterprises already using the Microsoft stack.
  3. Google Cloud Platform (GCP):
    • Strong focus on data analytics, AI, machine learning, and containerization (Kubernetes was born at Google).
  4. Others:
    • IBM Cloud: Focuses on hybrid cloud and AI (Watson).
    • Oracle Cloud: Focuses on databases and ERP.
    • Alibaba Cloud: Dominant in China and parts of Asia.

4. Why We Need DevOps on Cloud

DevOps and Cloud Computing are symbiotic. The cloud provides the necessary infrastructure to implement DevOps practices effectively.

  1. Speed of Deployment: Cloud enables the instant provisioning of servers. DevOps automates the software delivery. Together, they allow deployment in minutes rather than weeks.
  2. Scalability: DevOps practices often involve microservices and containerization. Cloud infrastructure scales these resources automatically based on traffic.
  3. Infrastructure as Code (IaC): Cloud allows infrastructure to be defined by code/scripts (e.g., Terraform, CloudFormation). This aligns with DevOps practices of version controlling infrastructure alongside application code.
  4. Automation: Cloud APIs allow DevOps tools (Jenkins, Ansible) to programmatically control infrastructure, enabling fully automated CI/CD pipelines.
  5. Cost Efficiency: Cloud eliminates the CapEx (Capital Expenditure) of buying hardware for testing environments. DevOps ensures environments are spun down when not in use to save OpEx (Operational Expenditure).

5. Introduction to Amazon Web Services (AWS)

AWS is a secure cloud services platform, offering compute power, database storage, content delivery, and other functionality.

Global Infrastructure

  • Regions: Physical geographic locations (e.g., US-East-1, EU-West-2). Each region is completely independent.
  • Availability Zones (AZs): Discrete data centers within a Region. Each AZ has redundant power, networking, and connectivity.
  • Edge Locations: Endpoints for AWS used for caching content (CloudFront) to reduce latency for end-users.

Core AWS Services

  • Compute:
    • EC2 (Elastic Compute Cloud): Resizable virtual servers.
    • Lambda: Serverless compute (run code without provisioning servers).
  • Storage:
    • S3 (Simple Storage Service): Object storage (files, images, backups).
    • EBS (Elastic Block Store): Hard drive storage for EC2 instances.
  • Database:
    • RDS (Relational Database Service): Managed SQL databases (MySQL, PostgreSQL).
    • DynamoDB: Managed NoSQL database.
  • Networking:
    • VPC (Virtual Private Cloud): Isolated cloud network.
    • Route 53: DNS web service.

6. Continuous Integration, Delivery, Deployment, and Monitoring

These form the "DevOps Pipeline."

1. Continuous Integration (CI)

A development practice where developers integrate code into a shared repository frequently (several times a day).

  • Process: Developer commits code -> Automated Build triggers -> Automated Tests run.
  • Goal: Detect errors quickly (Fail Fast).
  • Tools: Jenkins, GitLab CI, Travis CI, CircleCI.

2. Continuous Delivery (CD)

An extension of CI where the code changes are automatically prepared for a release to production.

  • Process: After CI passes, the build is deployed to a staging environment for testing.
  • Key Distinction: There is a manual approval step before the code goes to the Production environment. The software is always in a deployable state.

3. Continuous Deployment

The next step after Continuous Delivery.

  • Process: Every change that passes all stages of your production pipeline is released to your customers automatically.
  • Key Distinction: No human intervention. If the tests pass, the code goes live.

4. Continuous Monitoring

The process of monitoring every phase of the DevOps and IT operations lifecycles. It involves monitoring the health of the application and the infrastructure.

  • Metrics: CPU usage, memory, latency, error rates, user traffic.
  • Feedback Loop: Monitoring data is fed back to developers to improve the product.
  • Tools: Nagios, Prometheus, Grafana, AWS CloudWatch, Splunk, Datadog.

7. Basic Commands of LINUX

Linux is the dominant operating system in the Cloud/DevOps world because it is open-source, stable, secure, and scriptable.

File and Directory Management

  • pwd: Print Working Directory. Shows the current path.
  • ls: List files and directories.
    • ls -l: Long listing (shows permissions, owner, size).
    • ls -a: Show hidden files.
  • cd [path]: Change Directory.
    • cd ..: Go back one directory level.
    • cd ~: Go to the home directory.
  • mkdir [name]: Make Directory. Creates a new folder.
  • rmdir [name]: Remove an empty directory.
  • rm [file]: Remove a file.
    • rm -rf [directory]: Forcefully remove a directory and its contents (Use with caution!).
  • cp [source] [dest]: Copy file.
  • mv [source] [dest]: Move file (also used to rename files).

File Viewing and Editing

  • touch [filename]: Create an empty file.
  • cat [filename]: Display the content of a file on the screen.
  • head [filename]: View the first 10 lines of a file.
  • tail [filename]: View the last 10 lines of a file.
    • tail -f [filename]: Follow the file in real-time (great for logs).
  • nano [filename] or vi [filename]: Open a text editor to edit the file.
  • grep "text" [filename]: Search for a specific string of text inside a file.

Permissions and Ownership

  • chmod [permissions] [file]: Change file mode (permissions).
    • Example: chmod 755 script.sh (Read/Write/Execute for owner, Read/Execute for others).
    • +x: Make executable.
  • chown [user]:[group] [file]: Change the owner and group of a file.
  • sudo [command]: Run a command with Superuser DO (root) privileges.

System Information and Processes

  • top: Display dynamic real-time view of running processes (Task Manager).
  • ps: Display currently running processes.
    • ps aux: Detailed view of all processes.
  • kill [PID]: Terminate a process by its Process ID.
  • df -h: Display disk space usage in human-readable format.
  • free -m: Display memory (RAM) usage in MB.
  • uname -a: Print system information (kernel version, architecture).

Networking

  • ping [host]: Check connectivity to a host.
  • ip addr: Show IP addresses of the machine.
  • curl [url]: Transfer data from or to a server (great for testing APIs).
  • ssh [user]@[host]: Secure Shell. Connect to a remote Linux server.
  • netstat: Network statistics (ports, connections).

Package Management (Ubuntu/Debian)

  • apt update: Update the list of available packages.
  • apt install [package_name]: Install a software package.