Unit 4 - Notes

CSC104

Unit 4: Advanced Bash Scripting & Automation

1. Pipes vs. Redirection

Understanding the manipulation of data streams is fundamental to advanced scripting. While often confused, pipes and redirection serve distinct purposes regarding Input/Output (I/O) streams.

Standard Streams

Linux assigns file descriptors to data streams:

0: Standard Input (stdin) – Data fed into the program (default: keyboard).
1: Standard Output (stdout) – Data printed by the program (default: terminal screen).
2: Standard Error (stderr) – Error messages (default: terminal screen).

Pipes (`|`)

A pipe connects the stdout of one command to the stdin of another command. It allows for the chaining of utilities to process data.

Syntax: command1 | command2
Example: ls -l | grep ".txt"
- ls -l lists files.
- The output is passed directly to grep.
- grep filters for lines containing ".txt".

Redirection (`>`, `>>`, `<`)

Redirection changes the source or destination of I/O, typically involving files rather than other commands.

Output Redirection (>): Overwrites a file with stdout.
- echo "Hello" > file.txt
Append Redirection (>>): Appends stdout to the end of a file.
- echo "World" >> file.txt
Input Redirection (<): Feeds a file contents into a command's stdin.
- mysql -u root -p < database_backup.sql
Error Redirection (2>): Redirects stderr specifically.
- ls /non_existent_folder 2> error_log.txt
Redirecting stdout and stderr:
- command > log.txt 2>&1 (Sends both success and error messages to log.txt).

2. Multi-Function Scripting

Advanced scripts should be modular. Instead of writing linear code, use functions to group logic, improve readability, and enable code reuse.

Defining Functions

Functions can be defined in two ways:

BASH

function my_func {
    # commands
}

# OR

my_func() {
    # commands
}

Scope and Arguments

Local Variables: Use the local keyword inside functions to prevent variables from leaking into the global scope.
Arguments: Functions process arguments similarly to scripts ( $1`, `$ 2, $@), but these are independent of the script's main arguments.

Example Structure

BASH

#!/bin/bash

# Global variable
LOG_FILE="/var/log/myscript.log"

log_msg() {
    local timestamp=$(date +"%Y-%m-%d %H:%M:%S")
    echo "[$timestamp] $1" >> "$LOG_FILE"
}

check_root() {
    if [[ $EUID -ne 0 ]]; then
        echo "This script must be run as root" 
        exit 1
    fi
}

# Main execution flow
check_root
log_msg "Script started."

3. Interactive Menu with Colors

User interfaces in CLI scripts are enhanced using read, case statements, and ANSI escape codes for coloring.

ANSI Color Codes

Colors are defined by specific escape sequences echoed to the terminal.

Red: \e[31m
Green: \e[32m
Yellow: \e[33m
Reset: \e[0m (Crucial to stop coloring).

Script Example: Interactive Menu

BASH

#!/bin/bash

# Color definitions
RED='\e[31m'
GREEN='\e[32m'
NC='\e[0m' # No Color

show_menu() {
    echo -e "${GREEN}--- SYSTEM UTILITY MENU ---${NC}"
    echo "1. Show Disk Usage"
    echo "2. Show Uptime"
    echo "3. Exit"
}

while true; do
    show_menu
    read -p "Select an option: " choice
    
    case $choice in
        1)
            df -h
            ;;
        2)
            uptime
            ;;
        3)
            echo -e "${RED}Exiting...${NC}"
            break
            ;;
        *)
            echo "Invalid option."
            ;;
    esac
    echo "" # Empty line for formatting
done

4. Working with JSON via `jq`

Bash cannot natively parse JSON efficiently. jq is a lightweight command-line JSON processor used to slice, filter, and map JSON data.

Basic Usage

Assuming data.json contains {"name": "Server1", "status": "active", "load": [0.1, 0.5, 0.2]}:

Pretty Print: jq . data.json
Extract Value: jq -r '.status' data.json (Output: active)
- -r: Raw output (removes quotes).
Extract Array Item: jq '.load[0]' data.json (Output: 0.1)

Constructing JSON

jq can also build JSON objects from variables:

BASH

HOST="web01"
IP="192.168.1.5"
jq -n --arg h "$HOST" --arg i "$IP" '{hostname: $h, ip_address: $i}'

5. Cloudflare API Integration

Automating DNS or caching via APIs requires curl for HTTP requests and jq for parsing responses.

Prerequisites

API Token: Generated in the Cloudflare dashboard.
Zone ID: The unique identifier for the domain zone.

Example: Purge Cache

This script sends a POST request to the Cloudflare API to purge the cache for a specific zone.

BASH

#!/bin/bash

ZONE_ID="your_zone_id"
API_TOKEN="your_api_token"

response=$(curl -s -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/purge_cache" \
     -H "Authorization: Bearer $API_TOKEN" \
     -H "Content-Type: application/json" \
     --data '{"purge_everything":true}')

# Check success using jq
success=$(echo $response | jq -r '.success')

if [ "$success" == "true" ]; then
    echo "Cache purged successfully."
else
    echo "Failed to purge cache."
    echo $response | jq .errors
fi

6. Access Log Summarizer

Analyzing web server logs (Apache/Nginx) is a common automation task using awk, sort, and uniq.

Logic Flow

Read the log file.
Use awk to extract a specific field (e.g., Client IP address is usually field $1).
sort the IPs (required before uniq).
uniq -c to count occurrences.
sort -nr to sort numerically in reverse (highest traffic first).

The Script

BASH

#!/bin/bash

LOG_FILE="/var/log/apache2/access.log"

echo "Top 5 IP Addresses by Traffic:"
echo "------------------------------"

# awk '{print $1}' : Print first column (IP)
# sort             : Group IPs
# uniq -c          : Count unique lines
# sort -nr         : Sort by count (descending)
# head -n 5        : Show only top 5

awk '{print $1}' "$LOG_FILE" | sort | uniq -c | sort -nr | head -n 5

7. Sending Emails with SSMTP

SSMTP is a simple utility to send mail from a local computer to a configured mail hub (like Gmail or an SMTP server) without running a full mail server (like Postfix).

Configuration (`/etc/ssmtp/ssmtp.conf`)

INI

root=postmaster
mailhub=smtp.gmail.com:587
AuthUser=your_email@gmail.com
AuthPass=your_app_password
UseSTARTTLS=YES

Scripting Email Alerts

This allows scripts to notify admins of failures.

BASH

#!/bin/bash

SUBJECT="Backup Alert"
TO="admin@example.com"
MESSAGE="/tmp/email_body.txt"

# Create email body
echo "Subject: $SUBJECT" > $MESSAGE
echo "To: $TO" >> $MESSAGE
echo "" >> $MESSAGE
echo "The backup failed on $(hostname) at $(date)." >> $MESSAGE

# Send via SSMTP
ssmtp "$TO" < $MESSAGE

8. Password Generator

Generating random, secure strings is useful for auto-provisioning users.

Methods

Using openssl (Preferred):
BASH
```
    openssl rand -base64 12
    
```
Using /dev/urandom and tr:
- /dev/urandom: Source of random bytes.
- tr -dc: Delete content not in the specified set (keep only A-Z, a-z, 0-9).
- fold -w 16: Wrap lines at 16 characters.
- head -n 1: Take the first line.

Script Implementation

BASH

#!/bin/bash

generate_password() {
    local length=$1
    if [ -z "$length" ]; then length=16; fi
    
    tr -dc 'A-Za-z0-9!@#%^' < /dev/urandom | head -c "$length"
    echo "" # Newline
}

echo "Generated Password: $(generate_password 12)"

9. Remote Script Execution

Sysadmins often need to execute logic on remote servers without manually logging in. This relies on SSH.

Prerequisites

SSH access enabled.
SSH Keys: ssh-copy-id user@host should be set up to allow passwordless execution for automation.

Techniques

Running a Single Command:
BASH
```
    ssh user@remote-server "df -h"
    
```
Running a Local Script Remotely:
This reads the local script file and sends it to the remote bash process via stdin. The script does not need to exist on the remote server.
BASH
```
    ssh user@remote-server "bash -s" < ./local_script.sh
    
```

Using Heredoc for Blocks of Code:

BASH

    ssh user@remote-server << 'EOF'
        echo "Updating system..."
        sudo apt-get update -y
        echo "Done."
    EOF

10. Automated WordPress Setup on LAMP

This capstone topic combines variables, functions, I/O redirection, and package management to automate a complex deployment.

Workflow

Update System & Install LAMP Stack: Apache, MySQL (MariaDB), PHP.
Configure Database: Create DB and User via SQL commands.
Deploy WordPress: Download, extract, and configure.
Set Permissions: Ensure Apache owns the files.

Comprehensive Script

BASH

#!/bin/bash

# Configuration Variables
DB_NAME="wordpress_db"
DB_USER="wp_user"
DB_PASS="$(openssl rand -base64 12)"
WP_URL="https://wordpress.org/latest.tar.gz"

echo "--- Starting WordPress Automation ---"

# 1. Install Dependencies
echo "Installing LAMP stack..."
sudo apt-get update -q
sudo apt-get install -y apache2 mariadb-server php php-mysql libapache2-mod-php wget tar -q

# 2. Configure Database
echo "Configuring Database..."
# Determine query to create DB and User
SQL_QUERY="CREATE DATABASE IF NOT EXISTS $DB_NAME;
CREATE USER IF NOT EXISTS '$DB_USER'@'localhost' IDENTIFIED BY '$DB_PASS';
GRANT ALL PRIVILEGES ON $DB_NAME.* TO '$DB_USER'@'localhost';
FLUSH PRIVILEGES;"

# Execute SQL
sudo mysql -e "$SQL_QUERY"
echo "Database created. Password: $DB_PASS" > wp_credentials.txt

# 3. Download and Install WordPress
echo "Downloading WordPress..."
cd /var/www/html
sudo wget -q $WP_URL
sudo tar -xzf latest.tar.gz
sudo mv wordpress/* .
sudo rmdir wordpress
sudo rm latest.tar.gz
sudo rm index.html # Remove default Apache page

# 4. Configure wp-config
echo "Configuring wp-config.php..."
sudo cp wp-config-sample.php wp-config.php
# Use sed to replace placeholder strings with actual variables
sudo sed -i "s/database_name_here/$DB_NAME/" wp-config.php
sudo sed -i "s/username_here/$DB_USER/" wp-config.php
# Escape special chars in password for sed
SAFE_PASS=$(echo $DB_PASS | sed 's/[\/&]/\\&/g')
sudo sed -i "s/password_here/$SAFE_PASS/" wp-config.php

# 5. Permissions
echo "Setting permissions..."
sudo chown -R www-data:www-data /var/www/html
sudo chmod -R 755 /var/www/html

echo "--- WordPress Installed Successfully ---"
echo "Database credentials saved to wp_credentials.txt"

Unit 4 - Notes

Table of Contents

Unit 4: Advanced Bash Scripting & Automation

1. Pipes vs. Redirection

Standard Streams

Pipes (|)

Redirection (>, >>, <)

2. Multi-Function Scripting

Defining Functions

Scope and Arguments

Example Structure

3. Interactive Menu with Colors

ANSI Color Codes

Script Example: Interactive Menu

4. Working with JSON via jq

Basic Usage

Constructing JSON

5. Cloudflare API Integration

Prerequisites

Example: Purge Cache

6. Access Log Summarizer

Logic Flow

The Script

7. Sending Emails with SSMTP

Configuration (/etc/ssmtp/ssmtp.conf)

Scripting Email Alerts

8. Password Generator

Methods

Script Implementation

9. Remote Script Execution

Prerequisites

Techniques

10. Automated WordPress Setup on LAMP

Workflow

Comprehensive Script

Send Feedback

Thank You!

Pipes (`|`)

Redirection (`>`, `>>`, `<`)

4. Working with JSON via `jq`

Configuration (`/etc/ssmtp/ssmtp.conf`)