Back to Subject

Unit 3: Probability Theory and Loss Functions for Machine Learning - Practice Quiz

INT255 — Mathematics Behind Machine Learning 60 Questions
0 Correct 0 Wrong 60 Left
0/60

1 In machine learning, what does a random variable represent?

Random variables in ML models Easy
A. A variable whose value is a numerical outcome of a random phenomenon.
B. A variable that is always zero.
C. A fixed hyperparameter of a model, like the learning rate.
D. The name of the machine learning algorithm.

2 In a dataset for classifying handwritten digits (0-9), the digit's label is an example of a...

Random variables in ML models Easy
A. Negative random variable
B. Discrete random variable
C. Constant variable
D. Continuous random variable

3 Which of the following is a clear example of a continuous random variable in a machine learning model?

Random variables in ML models Easy
A. Whether an email is 'spam' or 'not spam'.
B. The temperature in Celsius predicted by a weather model.
C. The number of 'likes' on a social media post.
D. The star rating (1 to 5) of a product.

4 A random variable provides a numerical summary of a...

Random variables in ML models Easy
A. Dataset's size
B. Random outcome
C. Fixed algorithm
D. Model's complexity

5 Which probability distribution is commonly used to model binary outcomes, such as whether a customer will click on an ad or not?

Probability distributions used in learning algorithms Easy
A. Poisson distribution
B. Gaussian (Normal) distribution
C. Bernoulli distribution
D. Exponential distribution

6 The Gaussian distribution, often called the 'bell curve', is primarily defined by which two parameters?

Probability distributions used in learning algorithms Easy
A. Minimum and maximum
B. Mean and variance
C. Rate and time
D. Number of trials and probability of success

7 If you are modeling the number of typographical errors on a page of a book, which discrete probability distribution is most appropriate?

Probability distributions used in learning algorithms Easy
A. Bernoulli distribution
B. Poisson distribution
C. Uniform distribution
D. Gaussian distribution

8 What does a probability distribution fundamentally describe?

Probability distributions used in learning algorithms Easy
A. The exact value a variable will take.
B. The speed of a learning algorithm.
C. The probability of each possible outcome of a random variable.
D. The total number of data points in a set.

9 What is the primary goal of the Maximum Likelihood Estimation (MLE) method?

Likelihood estimation in ML context Easy
A. To calculate the prior probability of the parameters.
B. To minimize the number of features in the dataset.
C. To prove that a model is 100% correct.
D. To find the model parameters that maximize the probability of observing the given data.

10 The likelihood function is the probability of the observed data treated as a function of the...

Likelihood estimation in ML context Easy
A. Loss function
B. Prior probability
C. Data
D. Parameters

11 Why is the log-likelihood often maximized instead of the likelihood itself?

Likelihood estimation in ML context Easy
A. It is mathematically more convenient, turning products into sums.
B. It is the only way to handle data with negative values.
C. It is a requirement for all regression models.
D. It gives a completely different and better result.

12 In a linear regression model, assuming the errors are normally distributed, MLE is equivalent to minimizing which loss function?

Likelihood estimation in ML context Easy
A. Hinge Loss
B. Absolute Error Loss
C. Squared Error Loss
D. Logistic Loss

13 The Mean Squared Error (MSE) loss function is most suitable for which type of machine learning problem?

Loss functions: squared error, logistic loss Easy
A. Classification
B. Regression
C. Clustering
D. Reinforcement Learning

14 If a model predicts a house price of $300,000 and the actual price is $310,000, what is the squared error for this single prediction?

Loss functions: squared error, logistic loss Easy
A.
B.
C.
D.

15 Logistic Loss, also known as Binary Cross-Entropy, is primarily used in which type of task?

Loss functions: squared error, logistic loss Easy
A. Binary Classification
B. Time Series Forecasting
C. Linear Regression
D. Anomaly Detection

16 What is the fundamental role of a loss function during the training of a machine learning model?

Loss functions: squared error, logistic loss Easy
A. To count the number of data points in the training set.
B. To quantify the model's error, which the training process tries to minimize.
C. To select which algorithm to use.
D. To preprocess and clean the input data.

17 In the Bayesian formula , what does the term represent?

Bayesian interpretation of learning models Easy
A. The Prior: our belief about the parameters before seeing data .
B. The Likelihood: the probability of the data given the parameters.
C. The Posterior: our belief about the parameters after seeing data .
D. The Evidence: the probability of the data.

18 What is the 'posterior' probability in the context of Bayesian learning?

Bayesian interpretation of learning models Easy
A. The updated probability of a hypothesis after considering new evidence.
B. A type of loss function.
C. The probability that the evidence is correct.
D. The initial guess about a hypothesis before seeing any data.

19 A key difference between the Bayesian and frequentist approaches is that the Bayesian approach treats model parameters as...

Bayesian interpretation of learning models Easy
A. Always positive numbers
B. Fixed constants
C. Random variables
D. Unnecessary for prediction

20 What does Maximum a Posteriori (MAP) estimation do?

Bayesian interpretation of learning models Easy
A. It finds the parameter values that maximize the posterior probability.
B. It finds the parameter values that maximize only the prior.
C. It calculates the mean of the posterior distribution.
D. It finds the parameter values that maximize only the likelihood.

21 In a standard linear regression model , which components are treated as random variables from a modeling perspective?

Random variables in ML models Medium
A. Only the parameters
B. The target variable and the error term
C. Only the target variable
D. The feature matrix and the parameters

22 Which probability distribution is most suitable for modeling the class labels in a multi-class classification problem with classes for a single data point?

Probability distributions used in learning algorithms Medium
A. Categorical distribution
B. Poisson distribution
C. Gaussian distribution
D. Binomial distribution

23 Assuming a dataset of i.i.d. data points and a model that gives , the likelihood function is defined as:

Likelihood estimation in ML context Medium
A.
B.
C. without the i.i.d. assumption
D.

24 A binary classifier predicts a probability of for a data point whose true label is . What is the logistic loss (cross-entropy) for this single prediction?

Loss functions: squared error, logistic loss Medium
A.
B.
C.
D.

25 In a Bayesian framework, what is the relationship between the posterior, prior, and likelihood?

Bayesian interpretation of learning models Medium
A. Likelihood Posterior Prior
B. Posterior Likelihood + Prior
C. Prior Likelihood Posterior
D. Posterior Likelihood Prior

26 You are building a model to predict the number of customer support emails your company will receive in a one-hour period. Which probability distribution is a common and suitable choice for this task?

Probability distributions used in learning algorithms Medium
A. Uniform distribution
B. Poisson distribution
C. Gaussian distribution
D. Bernoulli distribution

27 Why is squared error loss generally not a good choice for classification problems?

Loss functions: squared error, logistic loss Medium
A. It penalizes confident, incorrect predictions too little.
B. It assumes the target variable is continuous.
C. Its derivative is always zero for classification.
D. It cannot be optimized using gradient descent.

28 Maximizing the log-likelihood is equivalent to maximizing the likelihood itself. What is the primary practical advantage of optimizing the log-likelihood?

Likelihood estimation in ML context Medium
A. It always results in a convex optimization problem.
B. It is computationally faster to compute a single logarithm than a single product.
C. It incorporates a prior belief about the parameters.
D. It converts products into sums, which are numerically more stable and analytically simpler.

29 When we say a machine learning model provides a 'probabilistic prediction', what does this imply about its output?

Random variables in ML models Medium
A. The model's output is a probability distribution over possible outcomes, not just a single point estimate.
B. The model's output is guaranteed to be correct with a certain probability.
C. The model's parameters are updated randomly during training.
D. The model uses a random number generator to make predictions.

30 What is the primary conceptual difference between Maximum a Posteriori (MAP) and Maximum Likelihood Estimation (MLE)?

Bayesian interpretation of learning models Medium
A. MAP incorporates a prior belief about the model parameters, while MLE does not.
B. MAP is a Bayesian method, while MLE is a frequentist method that cannot be interpreted probabilistically.
C. MAP maximizes the probability of the data given the parameters, while MLE maximizes the posterior.
D. MLE is used for regression while MAP is used for classification.

31 If we assume that the target variable in a regression problem follows a Gaussian distribution with mean and constant variance , maximizing the likelihood of the model parameters is equivalent to minimizing which loss function?

Likelihood estimation in ML context Medium
A. Sum of Absolute Errors:
B. Logistic Loss
C. Hinge Loss
D. Sum of Squared Errors:

32 In logistic regression, the sigmoid function is used to model the probability of the positive class. This probability is the parameter of which underlying probability distribution for the binary target variable?

Probability distributions used in learning algorithms Medium
A. Gaussian distribution
B. Binomial distribution
C. Categorical distribution
D. Bernoulli distribution

33 A regression model predicts a value of 150 for a data point with a true value of 100. Another model predicts 200 for a true value of 150. How do their squared error losses compare?

Loss functions: squared error, logistic loss Medium
A. The comparison is impossible without knowing the model.
B. The second prediction has a higher loss.
C. The loss is the same for both predictions.
D. The first prediction has a higher loss.

34 Performing MAP estimation with a Gaussian prior on the model weights is equivalent to performing MLE with which type of regularization?

Bayesian interpretation of learning models Medium
A. L2 Regularization (Ridge)
B. Dropout
C. No regularization
D. L1 Regularization (Lasso)

35 In a classification model, the output for a given input is a vector of probabilities for classes A, B, and C respectively. This vector can be interpreted as the parameters of which random variable?

Random variables in ML models Medium
A. A Gaussian random variable representing the prediction error
B. A Categorical random variable representing the predicted class
C. A set of three independent Bernoulli random variables
D. A Binomial random variable representing the count of correct predictions

36 You have a biased coin where the probability of heads, , is unknown. You flip it 5 times and observe the sequence H, T, H, H, T. What is the Maximum Likelihood Estimate (MLE) for ?

Likelihood estimation in ML context Medium
A.
B. It cannot be determined from this data.
C.
D.

37 Consider two regression models. Model A has a Root Mean Squared Error (RMSE) of 10. Model B has a Mean Absolute Error (MAE) of 10. What can we definitively conclude?

Loss functions: squared error, logistic loss Medium
A. We cannot directly compare the models as they use different error metrics.
B. Model A is better than Model B.
C. Both models have the same predictive accuracy.
D. Model B is better than Model A.

38 A key assumption of the Naive Bayes algorithm is that the features are conditionally independent given the class label. How does this assumption simplify the model's probability calculations?

Probability distributions used in learning algorithms Medium
A. It forces all features to follow a Gaussian distribution.
B. It ensures that the posterior probability is always greater than the prior.
C. It allows the joint probability of features to be calculated as the product of individual probabilities .
D. It eliminates the need to calculate the likelihood.

39 Under what condition does the Maximum a Posteriori (MAP) estimate for a parameter become exactly the same as the Maximum Likelihood Estimate (MLE)?

Bayesian interpretation of learning models Medium
A. When the posterior distribution is symmetric.
B. When the prior distribution is a uniform distribution.
C. When the likelihood function is Gaussian.
D. When the dataset is very small.

40 The logistic loss function for binary classification is derived from which principle?

Loss functions: squared error, logistic loss Medium
A. Maximizing the likelihood of the data under a Bernoulli distribution assumption.
B. Finding the maximum margin hyperplane between classes.
C. Minimizing the absolute error of the predicted probabilities.
D. Minimizing the squared distance between the prediction and the true label.

41 In Maximum Likelihood Estimation (MLE), finding parameters that maximize the likelihood is equivalent to minimizing a specific Kullback-Leibler (KL) divergence. Given the empirical data distribution and the model distribution , which KL divergence is minimized?

Likelihood estimation in ML context Hard
A.
B. The symmetrized KL Divergence:
C.
D. , where is a standard normal prior

42 Consider a binary classification problem using a sigmoid activation function . While squared error loss, , can be used, logistic loss (binary cross-entropy) is strongly preferred. From an optimization standpoint, what is the primary deficiency of using squared error loss in this context?

Loss functions: squared error, logistic loss Hard
A. It produces vanishingly small gradients for confidently misclassified samples, slowing learning.
B. It is an unbounded loss function, unlike logistic loss.
C. Its corresponding loss surface is non-convex, potentially having multiple local minima.
D. It is not differentiable with respect to the model weights.

43 In Maximum A Posteriori (MAP) estimation, the choice of prior distribution over the weights corresponds to a specific type of regularization. If the prior is a zero-mean Laplace distribution, , what form of regularization does this induce when minimizing the negative log-posterior?

Bayesian interpretation of learning models Hard
A. Elastic Net Regularization
B. L1 Regularization (Lasso)
C. Dropout
D. L2 Regularization (Ridge)

44 In topic modeling with Latent Dirichlet Allocation (LDA), a symmetric Dirichlet prior, , is placed on the per-document topic distributions. How does the hyperparameter influence the characteristics of the topic mixtures learned for documents?

Probability distributions used in learning algorithms Hard
A. controls the number of topics in the model.
B. Values of encourage dense, uniform-like mixtures, while encourages sparse topic mixtures.
C. Values of encourage sparse topic mixtures (few topics per document), while encourages dense, uniform-like mixtures.
D. only controls the variance of the topic distribution, not its sparsity.

45 In a Naive Bayes classifier, the 'naive' assumption concerns the conditional independence of feature random variables given the class random variable . Which mathematical statement correctly represents this core assumption?

Random variables in ML models Hard
A.
B.
C.
D.

46 Suppose your model is misspecified, e.g., you use MLE to fit a Gaussian model to data that was truly generated from a Laplace distribution . In the limit of infinite data, the MLE parameters will converge to the parameters of the Gaussian that is 'closest' to the true Laplace distribution. What measure of 'closeness' does MLE implicitly minimize?

Likelihood estimation in ML context Hard
A. The L2 distance between the probability density functions
B. The KL divergence
C. The Total Variation distance
D. The KL divergence

47 Comparing Huber loss, Logistic loss, and Squared Error loss in a classification context (), how would you rank their robustness to outliers, from most robust to least robust? An outlier is a point with a large error, e.g., a mislabeled point far from the boundary.

Loss functions: squared error, logistic loss Hard
A. Squared Error Loss > Logistic Loss > Huber Loss
B. Huber Loss > Logistic Loss > Squared Error Loss
C. All three have similar robustness as they are all convex.
D. Logistic Loss > Huber Loss > Squared Error Loss

48 In Bayesian modeling, a prior distribution is 'conjugate' to a likelihood if the resulting posterior is in the same probability distribution family as the prior. What is the primary computational advantage of using a conjugate prior?

Bayesian interpretation of learning models Hard
A. It guarantees that the MAP estimate will be identical to the MLE.
B. It simplifies the calculation of model gradients for optimization.
C. It ensures the posterior distribution is symmetric and unimodal.
D. It yields a closed-form analytical expression for the posterior, avoiding the need for numerical approximation methods like MCMC.

49 The reparameterization trick is essential for training Variational Autoencoders (VAEs). For a latent variable modeled by a diagonal Gaussian , how does this trick allow gradients to flow through the sampling step?

Probability distributions used in learning algorithms Hard
A. By replacing the sampling step with the mode of the distribution, .
B. By analytically integrating out the random variable from the loss function.
C. By using the score function estimator (REINFORCE) to estimate the gradient of the stochastic node.
D. By expressing as a deterministic function of parameters and a parameter-free random variable: , where .

50 Assuming a linear regression model where the target variable is modeled as a deterministic function of inputs plus Gaussian noise, , with . Maximizing the likelihood of this model with respect to the weights is equivalent to minimizing which loss function?

Loss functions: squared error, logistic loss Hard
A. Mean Squared Error (MSE)
B. Log-Cosh Loss
C. Mean Absolute Error (MAE)
D. Hinge Loss

51 Given i.i.d. samples from an Exponential distribution with PDF for , what is the Maximum Likelihood Estimate (MLE) for the rate parameter ?

Likelihood estimation in ML context Hard
A. The sample variance
B. The reciprocal of the sample variance
C. The sample mean,
D. The reciprocal of the sample mean,

52 The Maximum A Posteriori (MAP) estimate incorporates a prior belief about the parameters, while the Maximum Likelihood Estimate (MLE) does not. Under which specific condition on the prior distribution do the MAP and MLE estimates become identical?

Bayesian interpretation of learning models Hard
A. When the dataset size approaches infinity, for any valid prior.
B. When the prior is a uniform distribution over the parameter space.
C. When the posterior distribution is Gaussian.
D. When the likelihood function is from the exponential family.

53 A 2D random variable follows a multivariate Gaussian distribution with covariance matrix . Which statement about the relationship between the random variables and is correct?

Probability distributions used in learning algorithms Hard
A. The marginal distribution of is not Gaussian.
B. and are negatively correlated.
C. and are positively correlated and are not independent.
D. and are independent because the distribution is Gaussian.

54 The bias of an estimator for a true function is defined as , where the expectation is over all possible training datasets . What is the direct implication of an estimator being 'unbiased'?

Random variables in ML models Hard
A. The estimator is guaranteed to have the lowest possible Mean Squared Error.
B. The estimator has zero variance.
C. The average of the estimator's predictions over all possible training datasets is equal to the true value.
D. The estimator's prediction for any single training dataset is equal to the true value.

55 Minimizing the cross-entropy loss is a standard approach in classification. This is often described as being equivalent to minimizing the KL divergence . Why is this equivalence valid in the context of training a machine learning model?

Loss functions: squared error, logistic loss Hard
A. Because the entropy of the true data distribution, , is a constant with respect to the model's parameters that determine .
B. Because KL divergence is simply another name for cross-entropy.
C. The equivalence only holds if the model output is a Gaussian distribution.
D. Because cross-entropy is symmetric, so , just like KL divergence.

56 Early stopping is a regularization technique where training of an iterative algorithm (like a neural network) is halted when validation performance stops improving. What is the common Bayesian interpretation of this procedure?

Bayesian interpretation of learning models Hard
A. It is equivalent to using a Laplace (L1) prior for inducing sparsity in the weights.
B. It is an approximation of performing MAP estimation with a Gaussian prior on the weights, where the prior's variance is implicitly controlled by the stopping time.
C. It has no valid Bayesian interpretation and is considered a purely heuristic method.
D. It corresponds to maximizing the marginal likelihood (model evidence) instead of the posterior.

57 The invariance property of Maximum Likelihood Estimators (MLEs) states that if is the MLE for a parameter , and is a function, then the MLE for the transformed parameter is simply . Which of the following is a key condition for this property to hold?

Likelihood estimation in ML context Hard
A. The property only holds if the function is linear.
B. The property only holds if the function is bijective (one-to-one and onto).
C. The property only holds if the likelihood belongs to the exponential family.
D. The property holds for any function , it is not restricted to being one-to-one.

58 A Poisson process describes the number of events occurring in a fixed time interval, with an average rate of events per unit time. What is the probability distribution that models the waiting time between consecutive events in this process?

Probability distributions used in learning algorithms Hard
A. Normal distribution with mean .
B. Exponential distribution with rate parameter .
C. Poisson distribution with mean .
D. Gamma distribution with shape parameter 2.

59 The Fisher Information Matrix, , quantifies the information a random variable carries about an unknown parameter . It has a deep connection to the geometry of the likelihood surface. What is its relationship to the Hessian matrix, , of the negative log-likelihood function?

Loss functions: squared error, logistic loss Hard
A. The Fisher Information is always the identity matrix when the Hessian is positive definite.
B. The Fisher Information is the expectation of the Hessian of the negative log-likelihood.
C. The Fisher Information is the determinant of the Hessian.
D. The Fisher Information is the inverse of the Hessian.

60 In the evidence framework for Bayesian model selection, one maximizes the marginal likelihood to choose a model . How does this process naturally implement Occam's Razor?

Bayesian interpretation of learning models Hard
A. It penalizes overly complex models because they must spread their predictive probability over a larger space of possible datasets, reducing the probability assigned to the observed data.
B. It integrates out nuisance parameters, which is equivalent to L0 regularization.
C. It sets the prior probability of complex models, , to be exponentially lower.
D. It forces the posterior distribution to be unimodal, favoring simpler explanations.
Previous MCQ
Unit 2
Next MCQ
Unit 4