Unit 6: Image Processing Using Python

I. Foundations of Digital Images

Digital image processing uses computer algorithms to acquire, represent, transform, analyze, and improve images. A digital image is commonly modeled as a two-dimensional function (f(x,y)), sampled into a rectangular grid of pixels; each pixel stores one or more numerical intensity values.

• Pixel representation: A grayscale pixel usually stores one intensity value, while a color pixel commonly stores three channels such as red, green, and blue.
• Coordinate convention: In Pillow and OpenCV, the origin is normally the top-left corner; (x) increases to the right and (y) increases downward.
• Resolution: An image of width (W) and height (H) contains (W \times H) pixels; for example, (1920 \times 1080) contains 2,073,600 pixels.
• Bit depth: An 8-bit channel represents values from 0 to 255. A three-channel RGB image can represent approximately (2^{24}) colors.
• Processing model: Operations may change geometry, pixel intensity, color channels, or local relationships among neighboring pixels.
• Common libraries: Pillow provides accessible image-editing tools, while OpenCV supplies extensive computer-vision and numerical-processing functions.

A. Introduction to digital image processing

Digital image processing converts an input image into a modified image or extracts useful information from it.

• Image formation: Sampling divides a scene into pixels, while quantization assigns each sampled intensity a finite numerical value.
• Mathematical model: A grayscale image can be represented as:

I = f(x, y)

• (I): pixel intensity.
• (x,y): horizontal and vertical pixel coordinates.
  • Color model: In RGB, each pixel is ((R,G,B)); pure red is ((255,0,0)), white is ((255,255,255)), and black is ((0,0,0)).
  • Processing levels:
• Low-level: Noise removal, resizing, and contrast enhancement.
• Mid-level: Segmentation, edge detection, and feature extraction.
• High-level: Object recognition and scene interpretation.
  • File formats: JPEG uses lossy compression; PNG supports lossless compression and transparency; TIFF is common in high-quality imaging.

II. Geometric Image Transformations — Changing Spatial Arrangement

Geometric transformations alter pixel positions or image dimensions without necessarily changing the underlying color values.

A. Cropping

Cropping extracts a rectangular region of interest and discards pixels outside its boundaries.

• Crop box: A rectangle is commonly specified as ((left, top, right, bottom)).
• Resulting dimensions:

new_width  = right - left
new_height = bottom - top

• Boundary condition: Valid coordinates normally satisfy (0 \le left < right \le W) and (0 \le top < bottom \le H).
• Pillow operation:

from PIL import Image

image = Image.open("input.jpg")
cropped = image.crop((100, 50, 500, 350))
cropped.save("cropped.jpg")

• Applications: Cropping removes unwanted borders, focuses on a subject, or prepares a region for recognition.

B. Resizing

Resizing changes image width and height by mapping source pixels onto a new grid.

• Scaling factors:

sx = W_new / W_old
sy = H_new / H_old

• (s_x,s_y): horizontal and vertical scale factors.
• (W,H): image width and height.
  • Aspect ratio: Preserving (W/H) prevents stretching; a (1200 \times 800) image resized to width 600 should have height 400.
  • Interpolation:
• Nearest-neighbor: Fast but produces blocky edges.
• Bilinear: Uses four nearby pixels and gives smoother results.
• Bicubic/Lanczos: Uses larger neighborhoods and generally improves photographic downscaling.
  • Pillow operation:

resized = image.resize((600, 400), Image.Resampling.LANCZOS)

• Limitation: Enlarging creates estimated pixels; it cannot recover detail absent from the original.

C. Rotation

Rotation turns an image around a center point through an angle (\theta).

• Coordinate transformation:

x' = x cos(θ) - y sin(θ)
y' = x sin(θ) + y cos(θ)

• (x,y): coordinates relative to the rotation center.
• (x',y'): rotated coordinates.
• (\theta): rotation angle.
  • Canvas handling: Arbitrary rotation may clip corners unless the output canvas is expanded.
  • Pillow operation:

rotated = image.rotate(45, expand=True)

• Interpolation effect: Rotations other than multiples of (90^\circ) require interpolation and may slightly soften the image.
• Applications: Rotation corrects camera orientation, deskews documents, and augments training data.

D. Flipping

Flipping reflects pixels across a horizontal or vertical axis.

1. Horizontal flip: Reverses left and right using (x' = W-1-x).
2. Vertical flip: Reverses top and bottom using (y' = H-1-y).

• Pillow operation:

from PIL import ImageOps

horizontal = ImageOps.mirror(image)
vertical = ImageOps.flip(image)

• Properties: Flipping preserves dimensions and intensity values but changes spatial orientation.
• Applications: Reflection-based augmentation is useful when object orientation does not change its class.

III. Tonal and Color Transformations — Changing Pixel Values

Tonal and color transformations modify individual pixels or channels to improve visibility, appearance, or consistency.

A. Brightness adjustment

Brightness adjustment makes an image uniformly lighter or darker by transforming its intensity values.

• Additive model:

I' = clip(I + β, 0, 255)

• (I): original intensity.
• (I'): adjusted intensity.
• (\beta): brightness offset.
• clip: restricts values to the valid range.
  • Multiplicative model: (I'=cI), where (c>1) brightens and (0<c<1) darkens.
  • Pillow operation:

from PIL import ImageEnhance

bright = ImageEnhance.Brightness(image).enhance(1.4)

• Interpretation: A factor of 1.0 preserves brightness, 0.0 produces black, and 1.4 increases brightness by the library’s factor convention.
• Limitation: Excessive brightening clips highlights at 255 and destroys detail.

B. Contrast adjustment

Contrast adjustment increases or decreases differences between dark and bright regions.

• Linear transformation:

I' = clip(α(I - m) + m, 0, 255)

• (\alpha): contrast factor.
• (m): reference midpoint, often 128 for 8-bit data.
  • Effect: (\alpha>1) expands intensity differences; (0<\alpha<1) compresses them.
  • Pillow operation:

contrasted = ImageEnhance.Contrast(image).enhance(1.5)

• Histogram methods: Histogram equalization redistributes grayscale intensities; CLAHE enhances local regions while limiting noise amplification.
• Limitation: Strong contrast can remove shadow or highlight information through clipping.

C. Color adjustment

Color adjustment changes saturation, channel balance, hue, or the color space used to represent an image.

• Saturation: Increasing saturation strengthens differences between chromatic channels; reducing it toward zero produces grayscale.
• Channel balance: Multiplying red, green, or blue independently can correct a color cast.
• Pillow operation:

colorful = ImageEnhance.Color(image).enhance(1.3)

• Color spaces:
  • RGB: Suited to display and general editing.
  • HSV: Separates hue, saturation, and brightness-like value.
  • Grayscale: A common luminance approximation is (Y=0.299R+0.587G+0.114B).
• Caution: OpenCV loads standard color images in BGR order, whereas Pillow normally uses RGB.

IV. Spatial Filtering — Neighborhood-Based Processing

Spatial filtering calculates each output pixel from a neighborhood around the corresponding input location, commonly by applying a kernel.

A. Filtering and enhancement

Filtering and enhancement suppress unwanted information or emphasize features important for viewing and analysis.

• Convolution model:

g(x,y) = Σi Σj K(i,j) f(x-i, y-j)

• (f): input image.
• (g): filtered image.
• (K): kernel or filter mask.
• (i,j): offsets within the kernel.
  • Filter classes: Low-pass filters smooth rapid changes; high-pass filters emphasize edges and fine detail.
  • Kernel properties: A smoothing kernel often sums to 1, preserving approximate overall brightness.
  • Boundary handling: Padding may use zeros, reflected pixels, replicated border values, or wrapped coordinates.
  • Enhancement purpose: The best result depends on the task; visual attractiveness and machine-analysis accuracy are not always identical.

B. Blurring

Blurring reduces noise and fine detail by combining neighboring pixel values.

• Box blur: Replaces each pixel with the arithmetic mean of a neighborhood.
• Gaussian blur: Assigns greater weight to nearby pixels using a Gaussian distribution; kernel size and standard deviation (\sigma) control smoothing.
• Median blur: Replaces a pixel with the neighborhood median and is especially effective against salt-and-pepper noise.
• OpenCV operation:

import cv2

blurred = cv2.GaussianBlur(image, (5, 5), sigmaX=0)

• Kernel constraint: Gaussian kernel dimensions such as (5, 5) are normally positive odd numbers.
• Trade-off: More blur suppresses more noise but also removes texture and weak edges.

C. Sharpening

Sharpening emphasizes intensity transitions so that edges and fine structures appear clearer.

• Sharpening kernel:

 0 -1  0
-1  5 -1
 0 -1  0

• OpenCV operation:

import numpy as np

kernel = np.array([[0, -1, 0],
                   [-1, 5, -1],
                   [0, -1, 0]])
sharpened = cv2.filter2D(image, -1, kernel)

• Unsharp masking: A blurred image is subtracted from the original to obtain detail, which is then added back with a chosen strength.
• Kernel sum: The displayed kernel sums to 1, helping retain overall brightness while strengthening local differences.
• Limitation: Sharpening cannot create genuine detail and may amplify noise, halos, and compression artifacts.

D. Edge detection

Edge detection identifies locations where image intensity changes sharply, often corresponding to object boundaries.

• Gradient principle: The gradient magnitude is:

G = √(Gx² + Gy²)

• (G_x): horizontal intensity derivative.
• (G_y): vertical intensity derivative.
• (G): overall edge strength.
  • Sobel method: Convolves the image with horizontal and vertical derivative kernels to estimate (G_x) and (G_y).
  • Canny method: Applies Gaussian smoothing, gradient calculation, non-maximum suppression, and double-threshold edge tracking.
  • OpenCV operation:

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
edges = cv2.Canny(gray, 100, 200)

• Thresholds: Pixels above 200 are strong edges; pixels between 100 and 200 survive when connected to strong edges.
• Limitation: Noise creates false edges, while excessive smoothing or high thresholds may erase weak boundaries.

V. OpenCV — Computer-Vision Toolkit

OpenCV is an open-source library that combines image processing, video analysis, numerical operations, feature detection, and machine-learning utilities.

A. OpenCV basics

OpenCV basics center on loading images as NumPy arrays, applying functions, displaying results, and saving outputs.

• Installation: The standard Python package is installed with pip install opencv-python.
• Image loading:

import cv2

image = cv2.imread("input.jpg")
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
cv2.imwrite("gray.jpg", gray)

• Array shape: A color image generally has shape (height, width, channels); a grayscale image has shape (height, width).
• Reading failure: cv2.imread() returns None if the path is invalid or the file cannot be decoded.
• Display sequence:

cv2.imshow("Image", image)
cv2.waitKey(0)
cv2.destroyAllWindows()

• Strengths: OpenCV supports cameras, video frames, contours, morphology, feature matching, object detection, and efficient array-based processing.
• Data caution: Arithmetic on uint8 arrays can overflow unless OpenCV saturation functions or wider numeric types are used.

VI. Pillow — Python Imaging Library

Pillow is the actively maintained successor to PIL and offers a high-level interface for opening, editing, converting, drawing on, and saving raster images.

A. PIL (Pillow)

PIL (Pillow) represents images through Image objects and provides modules specialized for enhancement, filtering, drawing, and file handling.

• Installation and import:

from PIL import Image, ImageFilter, ImageEnhance

image = Image.open("input.png")
print(image.size, image.mode)

• Core attributes: size gives (width, height), mode gives representations such as "RGB", "RGBA", or "L", and format identifies formats such as PNG.
• Mode conversion:

gray = image.convert("L")
smooth = gray.filter(ImageFilter.GaussianBlur(radius=2))
smooth.save("output.png")

• Editing model: Methods such as crop(), resize(), rotate(), convert(), and filter() commonly return new Image objects.
• Resource management: A context manager closes file resources reliably:

with Image.open("input.jpg") as image:
    image.convert("RGB").save("copy.png")

• Strengths and limits: Pillow is convenient for file conversion and routine editing; OpenCV is generally better suited to real-time video, advanced vision algorithms, and large numerical pipelines.