Unit 1: Foundations of Computer Vision and Deep Learning

I. Orientation — Vision as Learning from Images

Computer vision is the branch of artificial intelligence that enables machines to extract useful information from images and video. Modern systems commonly learn visual representations directly from data, connecting image formation, numerical preprocessing, neural-network computation, and optimization.

• Core objective: Map visual input (x), such as an image tensor, to an output (y), such as a class label, bounding box, segmentation mask, or generated image.
• Data convention: A digital image is represented as an array with spatial dimensions—height (H) and width (W)—and usually a channel dimension (C).
• Learning assumption: Training examples are sampled from a distribution sufficiently similar to the data encountered after deployment.
• Model principle: A neural network forms predictions through parameterized transformations and learns parameters that reduce a specified loss.
• Pipeline: A typical workflow is data collection, preprocessing, model inference, loss calculation, backpropagation, parameter optimization, and evaluation.
• Evaluation principle: Generalization to unseen validation or test data matters more than memorization of training examples.

II. Computer Vision in Modern AI — Purpose and Paradigm Shift

A. Role of computer vision in modern AI applications

Computer vision converts visual measurements into decisions, descriptions, or actions across a wide range of AI systems.

• Classification: Assigns an image a category, such as identifying a radiograph as normal or abnormal.
• Object detection: Predicts both object classes and locations, commonly using bounding boxes ((x, y, w, h)).
• Semantic segmentation: Assigns a class to every pixel, supporting tasks such as road-scene understanding and tumor delineation.
• Identity and behavior analysis: Includes face verification, pose estimation, gesture recognition, and activity recognition.
• Autonomous systems: Cameras help vehicles and robots recognize lanes, obstacles, pedestrians, tools, and navigable space.
• Industrial and scientific use: Vision detects manufacturing defects, analyzes satellite imagery, monitors crops, and measures microscopic structures.
• Generative applications: Vision-language and diffusion models can caption, retrieve, edit, and synthesize images.
• Operational constraint: Accuracy must be balanced against latency, memory use, energy consumption, privacy, and robustness.

B. Traditional versus deep-learning-based vision approaches

Traditional vision depends heavily on manually designed features, whereas deep learning learns task-relevant representations from examples.

1. Traditional approach:

   • Pipeline: Preprocessing is followed by handcrafted feature extraction and a separate classifier.
   • Named techniques: Edge detectors, SIFT, HOG, color histograms, support vector machines, and decision trees are typical components.
   • Strength: Methods can work with limited data and may provide interpretable intermediate features.
   • Limitation: Features designed for one setting often fail under changes in illumination, viewpoint, scale, background, or object appearance.

2. Deep-learning approach:

   • Pipeline: A network learns hierarchical features and predictions jointly through end-to-end optimization.
   • Representation: Early layers often respond to edges and textures, while deeper layers combine them into parts and semantic objects.
   • Strength: Convolutional networks and vision transformers achieve strong performance on large, varied datasets.
   • Limitation: Training may require substantial labeled data, computation, and controls for bias and distribution shift.

III. Image Representation — From Light to Numerical Tensors

A. Image fundamentals

A digital image is a sampled and quantized representation of a continuous visual scene.

• Sampling: Spatial sampling divides the image plane into an (H \times W) grid; higher resolution records more spatial samples.
• Quantization: Continuous intensity is mapped to discrete levels; an 8-bit channel has (2^8=256) possible values, usually (0) through (255).
• Tensor shape: A color image may be stored as (H \times W \times C) or, in many deep-learning libraries, (C \times H \times W).
• Spatial coordinates: The origin is usually the top-left corner, with row position increasing downward and column position increasing rightward.
• Image quality: Resolution, dynamic range, noise, blur, exposure, and compression artifacts affect the information available to a model.

B. Pixels

A pixel is the smallest addressable spatial sample in a raster image and stores one or more numerical intensity values.

• Grayscale value: One number represents brightness; for 8-bit data, (0) conventionally denotes black and (255) white.
• Color value: An RGB pixel contains three components, such as ((255,0,0)) for saturated red in an 8-bit representation.
• Neighborhood: Adjacent pixels carry local structure; edges correspond to strong intensity changes over nearby coordinates.
• Physical interpretation: A pixel is a measurement area rather than an inherently square object in the real scene.
• Precision: Images may use unsigned integers or floating-point values, with medical and scientific images often exceeding 8-bit precision.

C. Channels

Channels are aligned two-dimensional planes that store distinct components of image information.

• RGB channels: Red, green, and blue planes combine additively to represent visible colors.
• Grayscale channel: A single plane stores luminance or intensity, producing a tensor with (C=1).
• Alpha channel: RGBA images include alpha for opacity; it is not ordinarily a visual color measurement.
• Domain-specific channels: Satellite data may contain infrared bands, while medical scans may combine multiple imaging sequences.
• Model compatibility: A network expecting (C=3) cannot directly accept a one-channel tensor without adapting the data or first layer.
• Ordering convention: RGB and BGR contain the same components in different orders; confusing them systematically changes model input.

D. Color spaces

A color space defines the numerical coordinates used to represent color and brightness.

• RGB: Matches additive display channels and is the standard input format for many pretrained vision networks.
• HSV/HSL: Separates hue from saturation and brightness-like components, which can simplify color-based selection.
• YCbCr: Separates luma (Y) from chroma components (Cb) and (Cr), supporting image and video compression.
• CIE Lab: Represents lightness and approximately perceptual color-opponent dimensions, making it useful for color comparison.
• Conversion requirement: Transformations must follow the library’s scale and channel conventions; hue ranges, for example, vary across implementations.
• Practical consequence: The best color space depends on the task, but pretrained models must receive the space used during their original training.

IV. Image Preprocessing — Producing Consistent Model Inputs

A. Resizing

Resizing changes an image’s spatial dimensions so that samples can be processed consistently or within computational limits.

• Interpolation: Nearest-neighbor copies nearby values, bilinear uses four neighbors, and bicubic uses a larger neighborhood for smoother estimates.
• Aspect ratio: Directly changing (W:H) can distort object geometry; padding or aspect-preserving scaling avoids this.
• Downsampling: Reducing dimensions can remove fine details and cause aliasing, so low-pass filtering may be applied first.
• Computational effect: Doubling both height and width produces four times as many pixels and substantially increases model cost.
• Label handling: Segmentation masks normally require nearest-neighbor resizing so interpolation does not create invalid class identifiers.

B. Cropping

Cropping selects a rectangular image region to control composition, resolution, or data variation.

• Center crop: Retains a fixed central region and gives deterministic input during evaluation.
• Random crop: Selects different regions during training, acting as data augmentation and reducing dependence on exact object position.
• Information risk: A crop may remove the target or important context, particularly when objects are small.
• Coordinate consistency: Bounding boxes, landmarks, and masks must be shifted and clipped using the same crop boundaries.
• Common sequence: An image may be resized on its shorter side and then cropped to the network’s required (H \times W).

C. Normalization

Normalization transforms pixel values to a scale and distribution suitable for stable numerical learning.

• Range scaling: Dividing 8-bit values by (255) maps ([0,255]) to ([0,1]).
• Standardization: Each channel can be centered and scaled:

x' = (x - μ) / σ

• Symbol definitions: (x) is the original channel value, (\mu) is the training-set channel mean, (\sigma) is its standard deviation, and (x') is normalized output.
• Optimization benefit: Comparable feature scales generally improve gradient behavior and accelerate convergence.
• Consistency rule: Validation and deployment images must use the same statistics and channel order as training data.
• Data leakage: Means and standard deviations should be estimated from training data, not from the test set.

V. Neural Network Architecture — Building Learnable Functions

A. Neural network basics

A neural network is a parameterized composition of layers that transforms an input tensor into a prediction.

• Neuron computation: A unit calculates:

z = Σ(w_i x_i) + b
a = φ(z)

• Symbol definitions: (x_i) are inputs, (w_i) are weights, (b) is bias, (z) is the pre-activation, (\phi) is an activation function, and (a) is output.
• Parameters: Weights and biases store learned behavior and are updated during training.
• Forward pass: Input travels layer by layer to produce predicted output (\hat{y}).
• Backpropagation: The chain rule computes each parameter’s contribution to the loss.
• Generalization controls: Validation, weight decay, dropout, augmentation, and early stopping help limit overfitting.

B. Perceptron

The perceptron is a linear binary classifier that applies a threshold to a weighted sum.

• Decision rule:

ŷ = 1 if wᵀx + b ≥ 0; otherwise ŷ = 0

• Symbol definitions: (x) is the feature vector, (w) is the weight vector, (b) is bias, and (\hat{y}) is the predicted class.
• Learning rule: For learning rate (\eta), an error can update weights as (w \leftarrow w+\eta(y-\hat{y})x).
• Geometric meaning: (w^Tx+b=0) defines a separating hyperplane.
• Capability: It represents linearly separable operations such as AND and OR.
• Limitation: A single perceptron cannot represent XOR because no single line separates its two classes.

C. Feedforward networks

A feedforward neural network connects layers without cycles, allowing information to move from input to output.

• Layer equation:

h^(l) = φ(W^(l)h^(l-1) + b^(l))

• Symbol definitions: (h^{(l)}) is layer (l)’s output, (W^{(l)}) and (b^{(l)}) are its parameters, and (h^{(l-1)}) is the previous layer’s output.
• Hidden layers: Multiple transformations learn nonlinear combinations of input features.
• Output layer: Its form depends on the task—one value for regression, sigmoid for binary classification, or softmax probabilities for multiple classes.
• Vision limitation: Fully connected layers ignore spatial locality and require many parameters for large images; convolutional layers address these issues through local connectivity and shared weights.

VI. Learning Components — Nonlinearity, Objectives, and Updates

A. Activation functions

Activation functions introduce nonlinearity, enabling networks to model relationships beyond a single linear transformation.

• ReLU: (\phi(z)=\max(0,z)) is inexpensive and widely used, but units can become inactive when inputs remain negative.
• Sigmoid: (\sigma(z)=1/(1+e^{-z})) maps values to ((0,1)), making it suitable for binary-output probabilities.
• Tanh: (\tanh(z)) maps values to ((-1,1)) but may produce vanishing gradients at large magnitudes.
• Softmax: For class logit (z_k),

p_k = exp(z_k) / Σ_j exp(z_j)

• Symbol definitions: (p_k) is the probability of class (k), and the denominator sums exponentials over all classes (j).
• Modern variants: Leaky ReLU, GELU, and SiLU preserve useful gradient behavior in architectures where standard ReLU may be restrictive.

B. Loss functions

A loss function assigns a numerical penalty to disagreement between predictions and target values.

• Mean squared error: Regression commonly uses:

MSE = (1/N) Σ_i (y_i - ŷ_i)²

• Symbol definitions: (N) is the sample count, (y_i) is the target, and (\hat{y}_i) is the prediction.
• Cross-entropy: Multiclass classification uses (L=-\sum_k y_k\log(p_k)), where (y_k) is the target indicator and (p_k) the predicted probability.
• Binary cross-entropy: Sigmoid outputs use (-[y\log(p)+(1-y)\log(1-p)]).
• Vision-specific objectives: Dice or IoU-based losses support segmentation, while detection combines classification and box-regression losses.
• Design principle: The loss should match the output representation and operational goal; class weighting can reduce the impact of imbalance.

C. Optimization methods

Optimization methods update network parameters to minimize the loss calculated over training data.

• Gradient descent:

θ ← θ - η∇_θL

• Symbol definitions: (\theta) represents parameters, (\eta) is the learning rate, (L) is loss, and (\nabla_\theta L) is its parameter gradient.
• Stochastic mini-batches: SGD estimates the gradient from a subset of examples, reducing computation per update and introducing useful noise.
• Momentum: Accumulates a moving direction from previous gradients, accelerating progress and reducing oscillation.
• Adam: Adapts step sizes using moving estimates of gradient means and squared gradients; it often converges quickly.
• Learning-rate control: Warm-up, step decay, cosine schedules, and plateau-based reduction change (\eta) during training.
• Regularization: Weight decay penalizes large parameters, while gradient clipping limits unstable updates.
• Training cycle: Each mini-batch follows forward pass, loss computation, gradient reset, backpropagation, and optimizer step.