Unit 1: Fundamentals of Image Processing, Image Enhancement, and Noise removal and filtering

1. Introduction to Image Processing

Image processing is a method to perform some operations on an image, in order to get an enhanced image or to extract some useful information from it. It is a type of signal processing in which the input is an image (such as a photograph or video frame) and the output may be an image or characteristics/features associated with that image.

Core Concepts:

• Analog Image Processing: Used for hard copies like printouts and photographs. Image analysts use various fundamentals of interpretation while using these visual techniques.
• Digital Image Processing (DIP): The use of computer algorithms to perform image processing on digital images. It involves:
  • Sampling: Digitizing the coordinate values (spatial resolution).
  • Quantization: Digitizing the amplitude values (pixel intensity/color depth).
• Pixel: The smallest element of an image, containing a specific value representing brightness or color.

2. Key Differences Between Image Processing and Computer Vision

While often used interchangeably, these two fields have distinct goals and methodologies.

3. Applications of Image Processing and Computer Vision

• Medical Imaging: X-ray enhancement, MRI analysis, tumor detection, and 3D organ reconstruction (IP + CV).
• Autonomous Vehicles: Lane detection, pedestrian tracking, reading traffic signs, and obstacle avoidance (CV).
• Security and Surveillance: Biometric authentication (facial, iris, fingerprint recognition), crowd monitoring, and anomaly detection.
• Industrial Automation / Quality Inspection: Defect detection in manufacturing pipelines, barcode reading, and robotic guidance.
• Agriculture: Crop monitoring using satellite/drone imagery, weed detection, and yield prediction.
• Entertainment and Media: CGI, augmented reality (AR) filters (e.g., Snapchat/Instagram), image restoration for old movies.

4. Image File Formats

Different file formats are optimized for different uses based on compression algorithms and color depth.

• JPEG (Joint Photographic Experts Group): Uses lossy compression. Best for photographs and web images where small file size is crucial, but suffers from compression artifacts.
• PNG (Portable Network Graphics): Uses lossless compression. Supports transparency (alpha channel). Ideal for web graphics, logos, and images requiring sharp contrast.
• TIFF (Tagged Image File Format): Uncompressed or losslessly compressed. Extremely high quality, widely used in professional photography, publishing, and medical imaging.
• BMP (Bitmap): Uncompressed, raster graphics image file format. Files are very large; primarily used in Windows environments.
• RAW: Contains minimally processed data directly from the camera's image sensor. Often called "digital negatives," offering the highest flexibility for post-processing.

5. Contrast Enhancement

Contrast enhancement improves the visual quality of an image by stretching or modifying the range of intensity values.

Techniques:

• Linear Contrast Stretching: Expands the original dynamic range of the image to the full dynamic range (e.g., 0-255).
  • Formula:
  • Where and are the minimum and maximum pixel values in the original image.
• Non-Linear Contrast Enhancement (Power-Law / Gamma Correction): Used to correct images that are washed out or too dark by applying a non-linear curve.
  • Formula:
  • If , dark regions are brightened. If , bright regions are darkened.
• Logarithmic Transformation: Expands the values of dark pixels while compressing higher-level values. Useful for viewing Fourier transform magnitude spectra.

6. Histogram Equalization and Specification

A Histogram represents the frequency distribution of pixel intensities in an image.

Histogram Equalization (HE)

HE is a technique for adjusting image intensities to enhance contrast globally. It spreads out the most frequent intensity values, flattening the histogram.

• Process:
  1. Calculate the probability mass function (PMF) of the image.
  2. Compute the cumulative distribution function (CDF).
  3. Multiply the CDF by the maximum pixel value (e.g., 255) and round to the nearest integer to map old pixels to new pixels.
• Limitation: It can amplify noise and produce unnatural looking images because it applies the same transformation globally.

Histogram Specification (Matching)

Instead of flattening the histogram, Histogram Specification transforms the image so that its histogram matches a specified target histogram.

• Process:
  1. Equalize the original image.
  2. Equalize the target/reference histogram to find its transformation function.
  3. Apply the inverse of the target's transformation function to the equalized original image.

7. Image Noise and Its Types

Noise is a random variation of image intensity or color information, usually introduced during image acquisition or transmission.

• Gaussian Noise (Normal Noise): Statistical noise having a probability density function equal to the normal distribution. Often caused by poor illumination or high temperature in sensors.
• Salt and Pepper Noise (Impulse Noise): Appears as randomly occurring white and black pixels. Usually caused by sharp, sudden disturbances in the image signal (e.g., analog-to-digital converter errors, bit errors in transmission).
• Poisson Noise (Photon Noise): Arises from the discrete nature of electric charge. It is prevalent in low-light conditions where photon counting statistics become significant.
• Speckle Noise: Granular noise that inherently degrades synthetic aperture radar (SAR), medical ultrasound, and optical coherence tomography images. It is multiplicative noise.

8. Spatial Domain Filtering Techniques

Spatial domain filtering operates directly on the pixels of an image using a "kernel" or "mask" (a small matrix).

Linear Filters

• Mean Filter (Average Filter): Replaces each pixel with the average value of its neighbors.
  • Effect: Smooths the image, reduces Gaussian noise, but blurs sharp edges.
• Gaussian Filter: Replaces pixels with a weighted average, where the central pixel has the highest weight, following a Gaussian curve.
  • Effect: Excellent for removing Gaussian noise while preserving edges better than a mean filter.

Non-Linear Filters

• Median Filter: Replaces each pixel with the median value of its neighbors.
  • Effect: Highly effective at removing Salt & Pepper noise while preserving sharp edges.
• Bilateral Filter: A non-linear, edge-preserving, and noise-reducing smoothing filter. It weighs pixels based on both spatial distance and intensity difference.

# Example: Applying Spatial Filters using OpenCV in Python
import cv2

image = cv2.imread('noisy_image.jpg')

# 1. Mean Filter
mean_filtered = cv2.blur(image, (5, 5))

# 2. Gaussian Filter
gaussian_filtered = cv2.GaussianBlur(image, (5, 5), 0)

# 3. Median Filter (Best for Salt & Pepper)
median_filtered = cv2.medianBlur(image, 5)

# 4. Bilateral Filter (Edge preserving)
bilateral_filtered = cv2.bilateralFilter(image, 9, 75, 75)

9. Frequency Domain Filtering

Frequency domain filtering involves transforming an image from the spatial domain to the frequency domain using the Discrete Fourier Transform (DFT), multiplying it with a filter mask, and then applying the Inverse Discrete Fourier Transform (IDFT) to return to the spatial domain.

• Low Frequencies: Correspond to smooth regions and gradual changes (backgrounds).
• High Frequencies: Correspond to sharp changes, edges, and noise.

Filter Types in Frequency Domain:

• Low-Pass Filter (LPF): Attenuates high frequencies and retains low frequencies.
  • Effect: Blurs the image and removes noise.
  • Examples: Ideal LPF, Butterworth LPF, Gaussian LPF.
• High-Pass Filter (HPF): Attenuates low frequencies and retains high frequencies.
  • Effect: Sharpens the image, highlights edges and fine details, but amplifies noise.
  • Examples: Ideal HPF, Butterworth HPF, Gaussian HPF.
• Band-Pass Filter: Allows a specific range of frequencies to pass while blocking lower and higher frequencies. Useful for isolating specific periodic patterns.
• Band-Reject / Notch Filter: Blocks a specific frequency or narrow range of frequencies. Ideal for removing periodic noise (like moiré patterns or electrical interference).