Unit 4: Advanced Bash Scripting & Automation

I. Foundations of Advanced Bash Automation

Advanced Bash scripting uses the Unix shell as an automation language for coordinating commands, files, services, APIs, and remote systems. Bash scripts are most reliable when they use clear functions, controlled inputs, explicit error handling, and commands designed for machine-readable output.

• Interpreter: A script commonly begins with #!/usr/bin/env bash, which locates Bash through the current environment.
• Execution model: Commands normally run sequentially; conditionals, loops, functions, pipelines, and background jobs alter that flow.
• Exit status: 0 means success, while values from 1 to 255 indicate failure or another defined condition. $? contains the latest status.
• Defensive options: set -Eeuo pipefail stops on unhandled errors, unset variables, and failed pipeline components; -E propagates ERR traps into functions.
• Quoting convention: Variables should normally be written as "$variable" to prevent unwanted word splitting and filename expansion.
• Input validation: Arguments, files, API responses, and user selections must be checked before they affect a system.
• Least privilege: Scripts should request root access only for commands that require it and should never embed passwords or API secrets.
• Idempotency: Repeated execution should preserve the intended state rather than duplicate users, configuration entries, databases, or installations.
• Observability: Useful scripts report meaningful errors and may log timestamps, actions, and command outcomes.

II. Modular Script Design

A. Multi-function scripting

Multi-function scripting divides an automation task into named, reusable operations with explicit inputs and return statuses.

• Function syntax: A Bash function is declared as name() { commands; } and invoked by writing its name.
• Arguments: Inside a function, $1, $2, and $@ represent its positional arguments; local prevents temporary variables from altering global state.
• Return values: return 0 reports success and return 1 reports failure. Textual data should be emitted with printf and captured through command substitution.
• Program entry point: A main function makes control flow visible and passes the script’s original arguments with main "$@".
• Example:

#!/usr/bin/env bash
set -Eeuo pipefail

log() { printf '[%s] %s\n' "$(date '+%F %T')" "$*"; }

require_file() {
    local file=$1
    [[ -f "$file" ]] || { log "Missing file: $file" >&2; return 1; }
}

main() {
    require_file "$1"
    log "Processing $1"
}

main "$@"

• Cleanup: trap cleanup EXIT can remove temporary files regardless of whether the script succeeds or fails.

III. Terminal User Interfaces

A. Interactive menu with colors

An interactive menu presents controlled choices while ANSI escape sequences visually distinguish prompts, success messages, and errors.

• Menu mechanism: select automatically displays numbered options, while read -r gives greater control over prompts and validation.
• Color codes: \033[31m selects red, \033[32m green, and \033[0m resets formatting.
• Safe output: printf interprets escape sequences consistently and is preferable to implementation-dependent echo -e.
• Input handling: A case statement maps exact choices to functions and rejects unexpected input.

red='\033[31m'
green='\033[32m'
reset='\033[0m'

while true; do
    printf '1. Status\n2. Backup\n3. Exit\nChoice: '
    read -r choice
    case $choice in
        1) systemctl --no-pager status nginx ;;
        2) printf '%bBackup started%b\n' "$green" "$reset"; run_backup ;;
        3) break ;;
        *) printf '%bInvalid selection%b\n' "$red" "$reset" >&2 ;;
    esac
done

• Terminal awareness: Colors should be disabled when standard output is not a terminal, testable with [[ -t 1 ]], so redirected logs remain readable.

IV. Administration Across Hosts

A. Remote script execution

Remote script execution uses SSH to authenticate, encrypt traffic, and run commands through a shell on another host.

• Direct command: ssh admin@web01 'uptime' executes uptime remotely; single quotes prevent the local shell from expanding remote expressions.
• Script streaming: A local script can become the remote shell’s standard input:

ssh admin@web01 'bash -s -- --check' < maintenance.sh

• Arguments: Values after -- become the remote script’s positional parameters, beginning with $1.
• Authentication: Public-key login, restricted keys, known_hosts verification, and an SSH agent are safer than automated password entry.
• Privilege: Remote sudo may require a terminal or password. Automation should grant narrowly scoped sudoers permissions instead of unrestricted root access.
• Failure detection: SSH returns the remote command’s exit status, but connection failures commonly use status 255; scripts must test the result.
• Parallel operation: Tools such as Ansible are more suitable when inventory management, concurrency, repeatability, and configuration state span many hosts.

V. Structured Data Processing

A. Working with JSON via jq

jq parses JSON structurally, allowing scripts to select, transform, validate, and construct data without fragile text matching.

• Field selection: jq -r '.result[0].id' response.json selects nested values; -r outputs strings without JSON quotation marks.
• Array iteration: .items[] | select(.enabled == true) | .name filters objects and returns matching names.
• Variables: --arg safely supplies shell strings without manually escaping JSON.
• Constructing JSON:

payload=$(jq -n \
  --arg type "A" \
  --arg name "www.example.com" \
  --arg content "203.0.113.10" \
  '{type: $type, name: $name, content: $content, ttl: 300}')

• Validation: jq -e '.success == true' response.json returns a failing status when the expression is false or null.
• Error handling: Network status and JSON meaning are separate concerns; scripts should check both curl --fail-with-body and the API’s success field.

VI. DNS Automation

A. Cloudflare API integration

Cloudflare API integration automates DNS and zone operations through HTTPS requests authenticated by a scoped API token.

• Endpoint model: DNS records are managed beneath /client/v4/zones/{zone_id}/dns_records; record updates require both zone and record identifiers.
• Authentication: Send Authorization: Bearer $CF_API_TOKEN. Tokens should be stored in protected environment or secret-manager storage.
• Permissions: A DNS updater normally needs only Zone:DNS:Edit for specified zones, not account-wide access.
• Request example:

curl --fail-with-body --silent --show-error \
  -X POST "https://api.cloudflare.com/client/v4/zones/$zone_id/dns_records" \
  -H "Authorization: Bearer $CF_API_TOKEN" \
  -H "Content-Type: application/json" \
  --data "$payload" |
jq -e '.success == true'

• Record lifecycle: Query existing records before creation to avoid duplicates; use PUT for a complete replacement or PATCH for supplied fields.
• Operational limits: Handle rate limits, non-2xx responses, and Cloudflare’s JSON errors array. Never print tokens in debug logs or process arguments.

VII. Web Traffic Reporting

A. Access log summarizer

An access log summarizer aggregates web-server request records into operational measures such as popular paths, status counts, clients, and transferred bytes.

• Common fields: In a typical combined log, $1 is the client address, $7 the requested path, $9 the status, and $10 the response size.
• Status summary:

awk '$9 ~ /^[0-9][0-9][0-9]$/ {count[$9]++}
     END {for (code in count) print code, count[code]}' access.log |
sort -k2,2nr

• Top clients: awk '{print $1}' access.log | sort | uniq -c | sort -nr | head ranks client addresses by request count.
• Byte handling: A size value of - is not numeric and should be treated as zero before summation.
• Rotation: Reports may need both active and compressed rotated logs, processed with zcat -f.
• Limitations: Fixed field positions can fail with custom formats or quoted values containing spaces. For production analytics, define the log format precisely or use a dedicated parser.

VIII. Automated Notifications

A. Sending emails with SSMTP

SSMTP sends locally generated mail through an external SMTP relay, but it is unmaintained and should generally be replaced by msmtp, Postfix, or a provider API.

• Role: SSMTP is a send-only mail transfer agent; it does not receive mail or maintain a local delivery queue.
• Configuration: Typical settings identify the relay host, port, authenticated user, TLS use, and sender rewriting in /etc/ssmtp/ssmtp.conf.
• Message format: Headers are separated from the body by one blank line.

{
    printf 'To: admin@example.com\n'
    printf 'From: monitor@example.com\n'
    printf 'Subject: Backup failure\n\n'
    printf 'Backup failed on %s at %s.\n' "$(hostname)" "$(date)"
} | sendmail -t

• Secret protection: SMTP credentials must have restrictive file permissions and should be supplied through a managed secret mechanism where possible.
• Reliability: A successful local command does not guarantee final delivery; queued MTAs provide retries and clearer delivery logs.

IX. Credential Utilities

A. Password generator

A password generator must use a cryptographically secure random source and avoid predictable seeds, biased selection, and accidental disclosure.

• Entropy source: /dev/urandom is appropriate for password generation on modern Unix-like systems.
• Generation:

LC_ALL=C tr -dc 'A-Za-z0-9!@#%_+=' < /dev/urandom | head -c 24
printf '\n'

• Length: With an alphabet of 70 symbols and 24 independent characters, ideal entropy is approximately 24 × log2(70), or 147 bits.
• Pipeline status: Under set -o pipefail, tr may report failure because head deliberately closes the pipe early; this behavior must be handled intentionally.
• Safer alternative: openssl rand -base64 24 produces secure random text, although Base64 output may include /, +, and padding =.
• Handling: Generated passwords should not enter shell history, logs, command-line arguments, or broadly readable temporary files.

X. Shell Data Flow

A. Pipes versus redirection

Pipes connect commands to commands, whereas redirection connects a command’s file descriptors to files, devices, or other streams.

1. Pipes: producer | consumer sends standard output of producer to standard input of consumer.

   • Example: journalctl -u nginx | grep -F 'failed' filters generated output.
   • Exit behavior: Without pipefail, the pipeline status is normally that of the final command.

2. Redirection: Operators change where a command reads or writes.

   • Overwrite: command > output.log sends descriptor 1, standard output, to a truncated file.
   • Append: command >> output.log adds output to the file.
   • Errors: command 2> errors.log redirects descriptor 2, standard error.
   • Combined output: command > all.log 2>&1 sends both streams to the same opened destination.
   • Input: command < input.txt supplies file contents as standard input.

• Ordering: Redirections are processed left to right; 2>&1 >file leaves standard error pointing at the original standard output.
• Pipeline logging: command | tee output.log both displays and stores output.

XI. Web Platform Provisioning

A. Automated WordPress setup on LAMP stack

Automated WordPress setup provisions Linux, Apache, MySQL or MariaDB, PHP, WordPress files, a database, and web-server configuration as one repeatable workflow.

• Packages: Install Apache, a database server, PHP, and required extensions such as php-mysql, php-curl, php-gd, php-mbstring, and php-xml.
• Database: Create a dedicated database and least-privileged user using SQL, with credentials obtained from protected variables rather than embedded literals.
• WordPress CLI: WP-CLI provides structured installation commands:

wp core download --path=/var/www/example
wp config create --path=/var/www/example \
  --dbname="$db_name" --dbuser="$db_user" --dbpass="$db_pass"
wp core install --path=/var/www/example \
  --url="$site_url" --title="$site_title" \
  --admin_user="$admin_user" --admin_password="$admin_pass" \
  --admin_email="$admin_email"

• Apache configuration: A virtual host should set DocumentRoot, hostname, directory permissions, and suitable rewrite support; enable it with a2ensite and validate using apachectl configtest.
• Ownership: Application files need carefully limited ownership and modes; making the entire document root globally writable is unsafe.
• Idempotency: Check whether packages, databases, configuration files, and the WordPress installation already exist before creating them.
• Completion: Reload Apache only after validation, enable HTTPS, remove temporary artifacts, and verify the site with an HTTP request and WP-CLI health checks.