Unit 6: Generative Adversarial Networks - Subjective Questions

Generative models learn the underlying probability distribution of observed data. Given training samples , they attempt to learn a distribution from which new, realistic samples can be generated.

Discriminative models learn a decision boundary or the conditional probability to predict a label for an input .

Key differences:

• A generative model learns how data is distributed, while a discriminative model learns how to separate or classify data.
• Generative models can create new samples; discriminative models primarily make predictions.
• GANs, VAEs, and autoregressive models are generative models.
• Logistic regression, support vector machines, and standard classifiers are discriminative models.

For example, a generative model can create a new face image, whereas a discriminative model can determine whether a given image contains a face.

A Generative Adversarial Network (GAN) consists of two neural networks trained in competition:

• Generator : Converts a random latent vector into a synthetic sample .
• Discriminator : Receives either a real sample or a generated sample and predicts the probability that the sample is real.

The networks play a two-player minimax game:

During training:

• The discriminator learns to distinguish real samples from generated samples.
• The generator learns to produce samples that fool the discriminator.
• Their parameters are updated alternately.

At ideal equilibrium, the generated distribution matches the real-data distribution, and the discriminator outputs approximately for both real and generated samples.

The discriminator is a binary classifier that estimates whether an input is real or generated. Its output is commonly interpreted as the probability that came from the training data.

A typical image discriminator contains:

• Convolutional layers for extracting spatial features.
• Downsampling through strides or pooling.
• Leaky ReLU activation functions.
• A final sigmoid output for binary classification.

For fixed generator parameters, the discriminator maximizes:

Training involves:

• Assigning real samples the target label .
• Assigning generated samples the target label .
• Computing binary cross-entropy loss.
• Updating only the discriminator parameters during its training step.

A discriminator that becomes too strong may provide weak gradients to the generator, so its training must remain balanced with generator training.

The generator learns a mapping from a latent space to the data space:

Here, is sampled from a simple prior distribution such as a Gaussian or uniform distribution, and is a generated sample.

For image generation, its architecture may include:

• A dense layer to project the latent vector.
• Reshaping into low-resolution feature maps.
• Transposed convolutions or upsampling layers.
• Batch normalization and ReLU activations.
• A final sigmoid or tanh activation matching the image normalization range.

The generator does not receive direct labels describing how to construct a real sample. Instead, it receives gradients propagated through the discriminator. A common non-saturating generator loss is:

Minimizing this loss encourages the generator to create samples that the discriminator classifies as real.

For a fixed generator, let denote the generated-data distribution. The GAN objective can be written pointwise as:

For each , maximize:

Differentiating and setting the result to zero gives:

Therefore, the optimal discriminator is:

Substitution into the value function shows that generator training is related to minimizing the Jensen-Shannon divergence between and .

At the ideal equilibrium:

• .
• for every supported input.
• The discriminator can no longer reliably distinguish generated samples from real samples.
• The objective reaches the value .

The major steps are:

1. Prepare the dataset: Resize images, convert them to tensors, and normalize pixel values to the range expected by the generator output, such as for tanh.
2. Define the latent space: Select a latent dimension and sample noise vectors from a Gaussian or uniform distribution.
3. Build the generator: Transform latent vectors into images using dense, reshaping, upsampling, or transposed-convolution layers.
4. Build the discriminator: Use convolutional layers to classify images as real or generated.
5. Choose losses and optimizers: Binary cross-entropy and Adam are common choices.
6. Train the discriminator: Use a batch containing real images and detached generated images.
7. Train the generator: Freeze discriminator updates, generate images, and optimize the generator so that generated images receive real targets.
8. Repeat alternately: Balance discriminator and generator update frequencies.
9. Monitor training: Save generated samples, losses, and checkpoints at regular intervals.
10. Evaluate results: Inspect sample quality, diversity, and quantitative measures such as FID.

GAN training alternates between two optimization steps for each mini-batch.

Discriminator update:

• Sample real data .
• Sample latent vectors .
• Generate fake samples and detach them from the generator computation graph.
• Minimize the discriminator loss:

Generator update:

• Sample a new batch of latent vectors.
• Pass generated samples through the discriminator.
• Keep discriminator parameters fixed during this optimizer step.
• Minimize the non-saturating loss:

The generator gradient flows through the discriminator, but only generator parameters are updated. This alternating process continues until generated quality and diversity become satisfactory or a chosen convergence criterion is met.

Mode collapse occurs when a generator maps many different latent vectors to the same output or to a small set of similar outputs. It therefore represents only a few modes of the real-data distribution.

Symptoms:

• Generated images look nearly identical.
• Different noise vectors produce limited variation.
• Samples may look realistic individually but lack diversity.

Possible causes:

• The generator discovers a small set of outputs that consistently fool the discriminator.
• The discriminator fails to penalize missing modes.
• Generator and discriminator learning rates are poorly balanced.
• Unstable gradients cause oscillation between modes.

Remedies:

• Use Wasserstein GAN with gradient penalty.
• Apply minibatch discrimination or feature matching.
• Use spectral normalization in the discriminator.
• Tune learning rates and discriminator update frequency.
• Use unrolled GAN training or multiple generators.
• Monitor both sample quality and diversity rather than loss alone.

GAN optimization is difficult because two networks learn simultaneously in a non-stationary adversarial game.

Major problems:

• Mode collapse: The generator creates only a limited variety of samples.
• Vanishing gradients: A highly accurate discriminator may give the generator almost no useful gradient.
• Non-convergence: The networks may oscillate instead of reaching equilibrium.
• Training imbalance: One network may improve much faster than the other.
• Sensitivity to hyperparameters: Architecture, normalization, optimizer, and learning-rate choices strongly affect results.
• Evaluation difficulty: Low loss does not necessarily imply high-quality or diverse samples.

Practical techniques:

• Use the non-saturating generator loss.
• Apply Wasserstein loss with gradient penalty.
• Use spectral normalization and suitable weight initialization.
• Tune separate learning rates using techniques such as TTUR.
• Add label smoothing or limited noise to discriminator inputs.
• Use stable architectural practices from DCGAN.
• Track generated samples, FID, and diversity metrics.

These techniques improve stability but do not guarantee convergence in every application.

The original GAN uses a discriminator that outputs a probability and optimizes a binary cross-entropy-based minimax objective. When real and generated distributions have little overlap, its divergence measure may saturate and provide weak generator gradients.

A Wasserstein GAN (WGAN) uses a critic that outputs an unrestricted real-valued score rather than a probability. Its objective is approximately:

Here, is the set of 1-Lipschitz functions.

Reasons for improved stability:

• The Wasserstein distance can provide meaningful gradients even when the distributions do not overlap.
• Critic loss tends to correlate better with generated sample quality.
• The critic does not use a sigmoid output.
• The critic can be trained multiple times per generator update.

The Lipschitz constraint was originally enforced by weight clipping. WGAN-GP instead adds a gradient penalty, which is generally more reliable and avoids capacity problems caused by clipping.

CycleGAN is a GAN architecture for translating images between two domains when corresponding paired examples are unavailable. For domains and , it uses:

• A generator .
• A generator .
• A discriminator that distinguishes real samples from .
• A discriminator that distinguishes real samples from .

The adversarial losses make translated outputs resemble images from the target domains. However, adversarial loss alone does not guarantee that an input's content is preserved. CycleGAN therefore applies cycle consistency:

Thus, an image translated to the other domain should be recoverable when translated back. This makes unpaired tasks such as horse-to-zebra, summer-to-winter, and painting-to-photograph translation possible.

For generators and , CycleGAN uses adversarial losses in both directions. One such loss is:

A corresponding loss is used for the reverse direction.

The cycle-consistency loss is:

An optional identity loss encourages generators not to change samples already belonging to the target domain:

The complete objective is:

The generators minimize this objective while the discriminators maximize their adversarial terms. The coefficients control the trade-off between realism, content preservation, and color or style preservation.

Paired translation GAN:

• Requires aligned input-output pairs, such as an edge map and its corresponding photograph.
• Can compare a generated output directly with the correct target using a reconstruction loss.
• Usually learns a more constrained and direct mapping.
• Pix2Pix is a common example.

CycleGAN:

• Uses two unpaired sets of images from domains and .
• Does not know the exact target image corresponding to an input.
• Uses two generators and two discriminators.
• Relies on cycle consistency to preserve input content.
• Is useful when paired data are expensive or impossible to obtain.

CycleGAN offers greater data flexibility, but the mapping is less constrained. It can produce incorrect semantic changes and may struggle when the required transformation involves major geometric alterations.

An adversarial example is an input intentionally modified by a small perturbation so that a trained model makes an incorrect prediction, while the modified input may still appear unchanged to a human.

FGSM is a one-step white-box attack that uses the gradient of the model loss with respect to the input. For an input , true label , model parameters , loss , and perturbation size , it constructs:

Procedure:

• Perform a forward pass and calculate the loss.
• Compute the gradient of the loss with respect to the input.
• Take the sign of each gradient component.
• Multiply by and add the perturbation to the input.
• Clip the result to the valid input range.

The sign operation selects the direction that rapidly increases the loss under an perturbation constraint.

FGSM seeks a perturbation that maximizes the loss while satisfying :

Using a first-order Taylor approximation around :

To maximize the inner product under the element-wise constraint , each component must have the same sign as its corresponding gradient component:

Therefore:

Effect of :

• A small gives a less visible perturbation but may fail to change the prediction.
• A large increases attack success but makes the perturbation easier to detect.
• The numerical meaning of depends on input normalization.

For a targeted attack, the gradient direction is reversed to reduce the loss for the chosen target class.

Adversarial training augments normal training data with adversarial examples. For each batch, FGSM examples are generated using the current model, and the model is trained on a mixture of clean and perturbed inputs.

A combined objective can be written as:

where controls the balance between clean accuracy and robustness.

Benefits:

• Exposes the model to worst-direction perturbations during training.
• Encourages smoother decision boundaries near training samples.
• Can improve resistance to FGSM and related attacks.

Limitations:

• May reduce accuracy on clean inputs.
• Increases training cost because input gradients must be computed.
• Robustness against FGSM does not guarantee robustness against stronger iterative attacks.
• Poor implementation may cause label leaking or gradient masking.
• The selected must match the threat model and input scale.

Robustness should therefore be evaluated with multiple attacks, perturbation budgets, and clean-data tests.

Docker packages an application with its runtime and dependencies into a portable container image. It reduces differences between development and production environments.

Benefits for deep learning deployment:

• Reproducible Python, framework, and library versions.
• Isolation from host-system packages.
• Easier deployment on local machines, cloud systems, and NVIDIA servers.
• Consistent startup commands and network configuration.
• Versioned images that support rollback and testing.

A suitable image normally contains:

• A trusted base image, such as a slim Python image for CPU or an NVIDIA CUDA runtime image for GPU execution.
• System libraries required by image-processing packages.
• A dependency file installed in a cache-efficient layer.
• Application source code and trained-model loading logic.
• A non-root runtime user where practical.
• An exposed Streamlit port, commonly .
• A command that starts the Streamlit application.

Model weights may be included in the image, mounted as a volume, or downloaded securely at startup.

A containerization workflow includes the following steps:

• Create a Streamlit application that loads the trained generator once, accepts user input, performs preprocessing and inference, and displays the result.
• Pin Python dependencies in a requirements or lock file.
• Select a base image compatible with the intended CPU or CUDA environment.
• Copy and install dependencies before copying frequently changed source files to improve build caching.
• Copy the application code into the image.
• Configure Streamlit to listen on all interfaces using --server.address=0.0.0.0.
• Expose container port 8501.
• Define the startup command as streamlit run app.py --server.address=0.0.0.0 --server.port=8501.
• Build the image using docker build -t gan-streamlit ..
• Run a CPU container using docker run -p 8501:8501 gan-streamlit.

The container should be tested for model-loading errors, missing native libraries, input validation, inference latency, and correct port accessibility.

An NVIDIA GPU deployment requires compatible hardware and software across the host, container runtime, CUDA runtime, and deep learning framework.

Host requirements:

• An NVIDIA GPU supported by the installed driver.
• A suitable NVIDIA driver installed on the server.
• Docker Engine.
• NVIDIA Container Toolkit configured for Docker.
• Network and firewall access to the application port.

Procedure:

1. Verify the host GPU using nvidia-smi.
2. Build or obtain an image based on an appropriate NVIDIA CUDA runtime.
3. Install a GPU-enabled version of PyTorch or TensorFlow compatible with the image runtime.
4. Build the application image.
5. Start it with GPU access, for example docker run --gpus all -p 8501:8501 gan-streamlit.
6. Confirm GPU visibility inside the container using framework-specific checks.
7. Load the model onto a CUDA device and place it in inference mode.
8. Test inference, memory use, logs, and external connectivity.

The host driver must support the CUDA runtime used by the container. Production deployments should also use restart policies, health checks, restricted privileges, controlled secrets, and monitoring.

A Streamlit GAN application can follow this inference flow:

1. Initialize the application: Select cuda when available and fall back to cpu only if supported.
2. Load the model: Reconstruct the generator architecture, load trusted weights, move the model to the selected device, and call evaluation mode.
3. Cache resources: Use Streamlit resource caching so the model is not reloaded on every interaction.
4. Collect input: Accept a random seed, latent-space controls, or a validated source image for translation models.
5. Preprocess: Resize, normalize, and batch the input exactly as required during training.
6. Run inference: Disable gradient calculation, execute the generator, and optionally use mixed precision after correctness testing.
7. Postprocess: Convert tensors to displayable images, reverse normalization, clip values, and transfer results to CPU.
8. Present output: Display the generated image and provide an appropriate download action.

Operational considerations:

• Limit upload types and sizes and never execute uploaded content.
• Do not expose arbitrary checkpoint paths or unsafe deserialization controls.
• Add error handling, health checks, request limits, and structured logs.
• Warm up the model and monitor GPU memory and latency.
• Place a reverse proxy with TLS and authentication in front of Streamlit when public access is required.
• Avoid displaying internal stack traces or sensitive server information to users.