Unit 1: Foundations of Computer Vision and Deep Learning - Subjective Questions

Computer vision is a field of artificial intelligence that enables machines to acquire, process, and interpret information from images and videos.

Its major roles include:

• Image classification: Assigning a label to an image, such as identifying whether an image contains a car or a pedestrian.
• Object detection: Locating and classifying multiple objects within an image.
• Image segmentation: Assigning a class label to individual pixels.
• Face recognition: Identifying or verifying individuals from facial images.
• Visual inspection: Detecting defects in manufactured products.
• Medical diagnosis: Analyzing X-rays, MRI scans, and pathology images.
• Autonomous navigation: Helping vehicles and robots understand their surroundings.

Computer vision transforms raw visual data into meaningful predictions or decisions. Deep learning has significantly improved its accuracy by allowing models to learn useful visual features directly from large datasets.

Traditional and deep-learning-based vision systems differ mainly in how they obtain image features.

Traditional computer vision:

• Uses manually designed feature extractors such as edges, corners, SIFT, HOG, and texture descriptors.
• Separates feature extraction from classification.
• Commonly uses classifiers such as support vector machines, decision trees, or k-nearest neighbors.
• Can work well with small datasets and limited computing resources.
• Depends strongly on domain expertise and careful feature engineering.

Deep-learning-based computer vision:

• Learns hierarchical features automatically from training data.
• Usually combines feature extraction and prediction in an end-to-end model.
• Uses architectures such as convolutional neural networks and vision transformers.
• Generally requires larger datasets and greater computational power.
• Often achieves better performance on complex, high-dimensional visual tasks.

A traditional pipeline may be written as image handcrafted features classifier, whereas a deep learning pipeline learns the mapping image output directly through optimization.

Advantages:

• Automatic feature learning: Deep networks learn edges, textures, shapes, and semantic concepts directly from data.
• High accuracy: They perform well on classification, detection, segmentation, and recognition tasks.
• End-to-end training: A single model can learn both feature extraction and prediction.
• Scalability: Performance often improves with more data and computing power.
• Transfer learning: Models trained on large datasets can be adapted to new tasks.

Limitations:

• They may require large labeled datasets.
• Training can be computationally expensive.
• Their decisions may be difficult to interpret.
• Performance can degrade under distribution shifts, poor lighting, occlusion, or adversarial perturbations.
• Biased training data can produce biased predictions.

Therefore, deep learning is powerful, but its use must consider data quality, computational cost, robustness, fairness, and explainability.

A digital image is represented as a numerical array composed of pixels. A pixel is the smallest addressable element of an image.

• A grayscale image is commonly represented by a matrix of shape , where is height and is width.
• A color image is generally represented by a tensor of shape , where is the number of channels.
• A grayscale image usually has one channel, while an RGB image has three channels: red, green, and blue.
• For an 8-bit image, each channel value is typically an integer in the range .
• The position of a pixel identifies its spatial location, while its numerical value represents brightness or color intensity.

For example, an RGB image of size is commonly represented by a tensor of shape . Deep learning frameworks may instead use the channel-first arrangement .

An image channel is a two-dimensional array containing one component of the information stored at every pixel.

• Grayscale image: Contains one channel representing brightness. Smaller values usually indicate darker pixels, while larger values indicate brighter pixels.
• RGB image: Contains three channels representing red, green, and blue intensities. A pixel color is formed by combining the three values .
• RGBA image: Contains the RGB channels and an additional alpha channel. The alpha value controls transparency or opacity.

For an 8-bit RGB image:

• Black is approximately .
• White is approximately .
• Pure red is approximately .

Some libraries store colors in BGR order rather than RGB order. The channel ordering must therefore be checked when images are transferred between libraries or supplied to a neural network.

A color space defines how colors are represented numerically.

• RGB: Represents a color as a combination of red, green, and blue. It is commonly used by displays and neural network inputs.
• Grayscale: Represents only intensity or luminance. It reduces data dimensionality when color is unnecessary.
• HSV: Represents hue, saturation, and value. Hue identifies the color type, saturation indicates color purity, and value indicates brightness.
• YCbCr: Separates luminance from chrominance components and . It is widely used in image and video compression.

Color-space conversion can make a task easier. HSV can help isolate objects by color under moderate illumination changes, while YCbCr can support skin-color analysis or compression. Grayscale conversion reduces computational cost for tasks based mainly on shape or texture. However, conversion may discard information, so the selected color space should match the requirements of the application.

Image resizing changes the spatial dimensions of an image so that it matches a required input size or computational budget. Since new output coordinates may not align with original pixels, interpolation is used to estimate their values.

• Nearest-neighbor interpolation: Selects the value of the closest source pixel. It is fast but can produce blocky edges.
• Bilinear interpolation: Uses a weighted average of the four nearest source pixels. It gives smoother results at moderate computational cost.
• Bicubic interpolation: Uses a larger neighborhood, commonly pixels. It often produces smoother and sharper results but is more expensive.

Resizing can distort objects if the original aspect ratio is not preserved. Common solutions include proportional resizing followed by padding, or resizing the shorter side and then cropping. For segmentation masks containing class IDs, nearest-neighbor interpolation is preferred because averaging class labels would create invalid categories.

Cropping selects a rectangular subregion of an image and removes the surrounding content.

• Center cropping: Selects a region from the image center. It is deterministic and is commonly used during validation or testing.
• Random cropping: Selects a region from a randomly chosen position. It is often used as data augmentation during training.
• Region-of-interest cropping: Selects a meaningful area, such as a detected face, object, or medical abnormality.

Cropping can reduce computation, standardize input dimensions, and focus the model on relevant content. Random cropping also improves generalization by exposing the model to different object positions. Its main risk is that an important object may be partially or completely removed. Crop parameters should therefore be chosen according to object size and task requirements.

Image normalization transforms pixel values into a scale or distribution that is more suitable for neural network training.

For 8-bit images, min-max scaling to is commonly performed as:

A general min-max transformation is:

Standardization transforms a pixel or channel using its mean and standard deviation :

Normalization provides several benefits:

• It prevents large input magnitudes from dominating computations.
• It can improve numerical stability.
• It can make optimization faster and more consistent.
• Per-channel standardization accounts for differences among color-channel distributions.

The same statistics and preprocessing rules used during training must also be applied during validation, testing, and deployment.

A suitable preprocessing pipeline may contain the following stages:

1. Decode the image: Read the file and convert it into a numerical tensor.
2. Correct orientation: Apply metadata-based rotation if required.
3. Convert the color format: Convert images consistently to RGB and remove or process alpha channels.
4. Resize: Resize while preserving aspect ratio, or resize to a fixed size if controlled distortion is acceptable.
5. Crop or pad: Produce the exact spatial dimensions expected by the model.
6. Augment training data: Apply random crops, flips, rotations, or color changes when appropriate.
7. Convert the data type: Change integer pixels to floating-point values.
8. Normalize: Scale values to or standardize each channel using training-set statistics.
9. Arrange dimensions: Convert to the framework's expected or format.
10. Batch the inputs: Combine examples into a batch tensor.

Validation and test pipelines should normally avoid random augmentation. Preprocessing must remain consistent with the procedure used to train the model.

A perceptron is a basic artificial neuron used for binary classification. Given inputs , weights , and bias , it first computes a weighted sum:

It then applies an activation function. In the original perceptron, a step function is used:

The weights determine the influence of each input, while the bias shifts the decision boundary. The decision boundary is the hyperplane:

A single perceptron can solve only linearly separable classification problems. It cannot represent non-linearly separable relationships such as the XOR function without additional layers.

The perceptron learning algorithm updates model parameters when an example is misclassified. For an input with target and predicted class , the update rule is:

where is the learning rate.

The procedure is:

• Initialize the weights and bias.
• Compute .
• Apply the step function to obtain .
• Calculate the prediction error .
• Update the weights and bias.
• Repeat over the training examples for multiple epochs.

If the classes are linearly separable, the perceptron convergence theorem states that the algorithm will find a separating hyperplane in a finite number of updates. For non-linearly separable data, it may fail to converge.

A feedforward neural network transfers information from the input layer through one or more hidden layers to the output layer without feedback connections.

For layer , the forward computation is:

Here, is the weight matrix, is the bias vector, and is an activation function.

• The input layer receives features or pixel values.
• Hidden layers learn intermediate representations.
• The output layer produces predictions, such as class probabilities.

During training, a loss function measures prediction error. Backpropagation computes gradients, and an optimizer updates the parameters. Multiple hidden layers enable the model to learn complex non-linear mappings, unlike a single linear perceptron.

Activation functions introduce non-linearity. Without them, a stack of linear layers would still be equivalent to a single linear transformation and could not learn complex decision boundaries.

• Sigmoid:
  
  It maps values to and is useful for binary output probabilities. It can saturate and produce vanishing gradients.

• Tanh:
  
  It maps values to and is zero-centered, but it can also suffer from saturation.

• ReLU:
  
  It is computationally efficient and widely used in hidden layers. Negative inputs have zero gradient, which can produce inactive neurons.

• Leaky ReLU:
  
  It retains a small gradient for negative inputs and reduces the inactive-neuron problem.

Softmax converts a vector of logits into normalized positive values. For class among classes:

Each exponential is positive, so:

The sum of the outputs is:

Therefore, every output lies between zero and one, and all outputs sum to one. They can consequently be interpreted as a categorical probability distribution.

For numerical stability, implementations usually subtract the largest logit before exponentiation:

This transformation does not change the result but reduces the risk of numerical overflow.

A loss function measures the difference between a model's prediction and the correct target. Training aims to minimize the average loss over the dataset.

Mean squared error:

It is commonly used for regression and strongly penalizes large errors.

Binary cross-entropy:

It is used for binary classification or independent multi-label classification, usually with sigmoid outputs.

Categorical cross-entropy:

It is used for mutually exclusive multi-class classification, usually with softmax outputs. For a one-hot target, it reduces to the negative logarithm of the probability assigned to the correct class.

Categorical cross-entropy for one example is:

Substituting the target and predicted probabilities:

Therefore:

Only the predicted probability of the correct class contributes because the target is one-hot encoded. A loss of approximately indicates that the model assigns a reasonably high probability to the correct class, but it is not completely confident.

If the correct-class probability approached , the loss would approach . If that probability approached , the loss would become very large. Cross-entropy therefore strongly penalizes confident incorrect predictions.

Gradient descent is an iterative optimization method that minimizes a loss function , where represents all trainable parameters.

The gradient:

points in the direction of the steepest increase in loss. Therefore, parameters are updated in the opposite direction:

where is the learning rate.

The training process consists of:

• Performing a forward pass to compute predictions.
• Calculating the loss.
• Using backpropagation to compute gradients through the chain rule.
• Updating parameters using the optimizer.
• Repeating the process over many batches and epochs.

A learning rate that is too large can cause divergence or oscillation, while a very small learning rate can make convergence unnecessarily slow.

The three methods differ in the number of training examples used to estimate a gradient before each parameter update.

• Batch gradient descent: Uses the entire training dataset. It provides a stable gradient but can require substantial memory and computation.
• Stochastic gradient descent: Uses one training example per update. It performs frequent, noisy updates and can explore the loss surface effectively, but training may fluctuate.
• Mini-batch gradient descent: Uses a small subset of examples, such as 32, 64, or 128. It balances gradient stability, memory use, update frequency, and hardware efficiency.

For a mini-batch , the gradient estimate is:

Mini-batch training is the standard approach for deep learning because matrix operations can be parallelized efficiently on GPUs while retaining useful stochasticity.

SGD with momentum accumulates a velocity from previous gradients:

It reduces oscillation and accelerates movement in consistent directions.

RMSProp maintains an exponentially weighted average of squared gradients:

It adapts the update size separately for each parameter.

Adam combines momentum-like first moments with RMSProp-like second moments. It generally converges quickly and works well with limited tuning.

SGD with momentum may provide strong final generalization in vision models, while Adam is often convenient for rapid training and noisy or sparse gradients. Optimizer performance still depends on the learning rate, schedule, batch size, architecture, and dataset.