Unit 3 - Practice Quiz

1 In machine learning, what does a random variable represent?

2 In a dataset for classifying handwritten digits (0-9), the digit's label is an example of a...

3 Which of the following is a clear example of a continuous random variable in a machine learning model?

4 A random variable provides a numerical summary of a...

5 Which probability distribution is commonly used to model binary outcomes, such as whether a customer will click on an ad or not?

6 The Gaussian distribution, often called the 'bell curve', is primarily defined by which two parameters?

7 If you are modeling the number of typographical errors on a page of a book, which discrete probability distribution is most appropriate?

8 What does a probability distribution fundamentally describe?

9 What is the primary goal of the Maximum Likelihood Estimation (MLE) method?

10 The likelihood function is the probability of the observed data treated as a function of the...

11 Why is the log-likelihood often maximized instead of the likelihood itself?

12 In a linear regression model, assuming the errors are normally distributed, MLE is equivalent to minimizing which loss function?

13 The Mean Squared Error (MSE) loss function is most suitable for which type of machine learning problem?

14 If a model predicts a house price of $300,000 and the actual price is $310,000, what is the squared error for this single prediction?

15 Logistic Loss, also known as Binary Cross-Entropy, is primarily used in which type of task?

16 What is the fundamental role of a loss function during the training of a machine learning model?

17 In the Bayesian formula , what does the term represent?

18 What is the 'posterior' probability in the context of Bayesian learning?

19 A key difference between the Bayesian and frequentist approaches is that the Bayesian approach treats model parameters as...

20 What does Maximum a Posteriori (MAP) estimation do?

21 In a standard linear regression model , which components are treated as random variables from a modeling perspective?

22 Which probability distribution is most suitable for modeling the class labels in a multi-class classification problem with classes for a single data point?

23 Assuming a dataset of i.i.d. data points and a model that gives , the likelihood function is defined as:

24 A binary classifier predicts a probability of for a data point whose true label is . What is the logistic loss (cross-entropy) for this single prediction?

25 In a Bayesian framework, what is the relationship between the posterior, prior, and likelihood?

26 You are building a model to predict the number of customer support emails your company will receive in a one-hour period. Which probability distribution is a common and suitable choice for this task?

27 Why is squared error loss generally not a good choice for classification problems?

28 Maximizing the log-likelihood is equivalent to maximizing the likelihood itself. What is the primary practical advantage of optimizing the log-likelihood?

29 When we say a machine learning model provides a 'probabilistic prediction', what does this imply about its output?

30 What is the primary conceptual difference between Maximum a Posteriori (MAP) and Maximum Likelihood Estimation (MLE)?

31 If we assume that the target variable in a regression problem follows a Gaussian distribution with mean and constant variance , maximizing the likelihood of the model parameters is equivalent to minimizing which loss function?

32 In logistic regression, the sigmoid function is used to model the probability of the positive class. This probability is the parameter of which underlying probability distribution for the binary target variable?

33 A regression model predicts a value of 150 for a data point with a true value of 100. Another model predicts 200 for a true value of 150. How do their squared error losses compare?

34 Performing MAP estimation with a Gaussian prior on the model weights is equivalent to performing MLE with which type of regularization?

35 In a classification model, the output for a given input is a vector of probabilities for classes A, B, and C respectively. This vector can be interpreted as the parameters of which random variable?

36 You have a biased coin where the probability of heads, , is unknown. You flip it 5 times and observe the sequence H, T, H, H, T. What is the Maximum Likelihood Estimate (MLE) for ?

37 Consider two regression models. Model A has a Root Mean Squared Error (RMSE) of 10. Model B has a Mean Absolute Error (MAE) of 10. What can we definitively conclude?

38 A key assumption of the Naive Bayes algorithm is that the features are conditionally independent given the class label. How does this assumption simplify the model's probability calculations?

39 Under what condition does the Maximum a Posteriori (MAP) estimate for a parameter become exactly the same as the Maximum Likelihood Estimate (MLE)?

40 The logistic loss function for binary classification is derived from which principle?

41 In Maximum Likelihood Estimation (MLE), finding parameters that maximize the likelihood is equivalent to minimizing a specific Kullback-Leibler (KL) divergence. Given the empirical data distribution and the model distribution , which KL divergence is minimized?

42 Consider a binary classification problem using a sigmoid activation function . While squared error loss, , can be used, logistic loss (binary cross-entropy) is strongly preferred. From an optimization standpoint, what is the primary deficiency of using squared error loss in this context?

43 In Maximum A Posteriori (MAP) estimation, the choice of prior distribution over the weights corresponds to a specific type of regularization. If the prior is a zero-mean Laplace distribution, , what form of regularization does this induce when minimizing the negative log-posterior?

44 In topic modeling with Latent Dirichlet Allocation (LDA), a symmetric Dirichlet prior, , is placed on the per-document topic distributions. How does the hyperparameter influence the characteristics of the topic mixtures learned for documents?

45 In a Naive Bayes classifier, the 'naive' assumption concerns the conditional independence of feature random variables given the class random variable . Which mathematical statement correctly represents this core assumption?

46 Suppose your model is misspecified, e.g., you use MLE to fit a Gaussian model to data that was truly generated from a Laplace distribution . In the limit of infinite data, the MLE parameters will converge to the parameters of the Gaussian that is 'closest' to the true Laplace distribution. What measure of 'closeness' does MLE implicitly minimize?

47 Comparing Huber loss, Logistic loss, and Squared Error loss in a classification context (), how would you rank their robustness to outliers, from most robust to least robust? An outlier is a point with a large error, e.g., a mislabeled point far from the boundary.

48 In Bayesian modeling, a prior distribution is 'conjugate' to a likelihood if the resulting posterior is in the same probability distribution family as the prior. What is the primary computational advantage of using a conjugate prior?

49 The reparameterization trick is essential for training Variational Autoencoders (VAEs). For a latent variable modeled by a diagonal Gaussian , how does this trick allow gradients to flow through the sampling step?

50 Assuming a linear regression model where the target variable is modeled as a deterministic function of inputs plus Gaussian noise, , with . Maximizing the likelihood of this model with respect to the weights is equivalent to minimizing which loss function?

51 Given i.i.d. samples from an Exponential distribution with PDF for , what is the Maximum Likelihood Estimate (MLE) for the rate parameter ?

52 The Maximum A Posteriori (MAP) estimate incorporates a prior belief about the parameters, while the Maximum Likelihood Estimate (MLE) does not. Under which specific condition on the prior distribution do the MAP and MLE estimates become identical?

53 A 2D random variable follows a multivariate Gaussian distribution with covariance matrix . Which statement about the relationship between the random variables and is correct?

54 The bias of an estimator for a true function is defined as , where the expectation is over all possible training datasets . What is the direct implication of an estimator being 'unbiased'?

55 Minimizing the cross-entropy loss is a standard approach in classification. This is often described as being equivalent to minimizing the KL divergence . Why is this equivalence valid in the context of training a machine learning model?

56 Early stopping is a regularization technique where training of an iterative algorithm (like a neural network) is halted when validation performance stops improving. What is the common Bayesian interpretation of this procedure?

57 The invariance property of Maximum Likelihood Estimators (MLEs) states that if is the MLE for a parameter , and is a function, then the MLE for the transformed parameter is simply . Which of the following is a key condition for this property to hold?

58 A Poisson process describes the number of events occurring in a fixed time interval, with an average rate of events per unit time. What is the probability distribution that models the waiting time between consecutive events in this process?

59 The Fisher Information Matrix, , quantifies the information a random variable carries about an unknown parameter . It has a deep connection to the geometry of the likelihood surface. What is its relationship to the Hessian matrix, , of the negative log-likelihood function?

60 In the evidence framework for Bayesian model selection, one maximizes the marginal likelihood to choose a model . How does this process naturally implement Occam's Razor?