Back to Subject

Unit 3: Ensemble Methods and Hyperparameter Tuning - Practice Quiz

INT395 — Supervised Learning 50 Questions
0 Correct 0 Wrong 50 Left
0/50

1 What is the primary motivation behind using Ensemble Methods in machine learning?

A. To reduce the number of features in the dataset
B. To increase the computational speed of training models
C. To combine multiple weak models to improve overall performance and generalization
D. To eliminate the need for hyperparameter tuning

2 Which of the following statements best explains how Ensemble methods reduce error according to the bias-variance decomposition?

A. Bagging primarily reduces bias, while Boosting primarily reduces variance.
B. They always reduce bias without affecting variance.
C. They always reduce variance without affecting bias.
D. Bagging primarily reduces variance, while Boosting primarily reduces bias.

3 In the context of ensemble learning, what is a Weak Learner?

A. A model that has too many parameters
B. A model that performs slightly better than random guessing
C. A model that overfits the data significantly
D. A model that has 0% training error

4 What does Bagging stand for?

A. Bootstrap Aggregating
B. Basic Aggregating
C. Binary Aggregating
D. Backward Aggregating

5 Which statistical technique involves sampling data subsets with replacement?

A. Stratification
B. Jackknife
C. Cross-Validation
D. Bootstrapping

6 In a Random Forest, which two randomization techniques are combined?

A. Boosting and Bagging
B. L1 and L2 regularization
C. Grid Search and Random Search
D. Bootstrap sampling and random feature selection

7 If you are training a Bagging ensemble with samples, approximately what fraction of samples are left out of a single bootstrap sample (Out-Of-Bag)?

A.
B.
C.
D.

8 Which of the following is true regarding the parallelization of Bagging and Boosting?

A. Both Bagging and Boosting can be easily parallelized.
B. Boosting is easy to parallelize, whereas Bagging is inherently sequential.
C. Neither can be parallelized.
D. Bagging is easy to parallelize, whereas Boosting is inherently sequential.

9 In AdaBoost, how are the weights of training instances updated after each iteration?

A. Weights are assigned randomly.
B. Misclassified instances are given higher weights.
C. Weights are kept constant throughout training.
D. Correctly classified instances are given higher weights.

10 What is the main difference between AdaBoost and Gradient Boosting?

A. AdaBoost minimizes the loss function using gradient descent, while Gradient Boosting uses weighted voting.
B. AdaBoost changes sample weights, while Gradient Boosting fits the new predictor to the residual errors of the previous predictor.
C. AdaBoost cannot be used for regression, while Gradient Boosting can.
D. There is no difference; they are synonyms.

11 In the context of Stacking, what is a Meta-Learner?

A. A model that learns how to combine the predictions of the base models
B. The first layer of base models
C. A specific type of Deep Neural Network
D. A model used for hyperparameter tuning

12 Which ensemble method is mathematically represented by , where is the learning rate?

A. Stacking
B. Hard Voting
C. Gradient Boosting
D. Random Forest

13 What is the primary risk when using Boosting with a large number of iterations (trees)?

A. Vanishing gradients
B. Underfitting
C. Overfitting
D. High bias

14 What is Hard Voting in ensemble classifiers?

A. Using a meta-model to decide the vote
B. Weighting votes based on classifier confidence
C. Averaging the probabilities of all classifiers
D. Taking the majority class prediction as the final output

15 What is Soft Voting?

A. Predicting the class with the highest summed predicted probability across classifiers
B. Predicting the class with the most votes
C. Randomly selecting a classifier's output
D. Using a soft-margin SVM as the ensemble

16 Why is diversity important in an ensemble?

A. It increases the bias of the ensemble.
B. It ensures all models are identical.
C. It allows models to make independent errors, which cancel out when aggregated.
D. It simplifies the hyperparameter tuning process.

17 Which of the following is NOT a Hyperparameter?

A. The learning rate in Gradient Descent
B. The number of neighbors () in KNN
C. The depth of a decision tree
D. The weights learned by a linear regression model

18 What is the primary purpose of Hyperparameter Tuning?

A. To visualize the results
B. To select the optimal configuration for the learning algorithm to maximize performance
C. To clean the dataset
D. To train the model parameters like weights and biases

19 How does Grid Search work?

A. It manually asks the user to input values during training.
B. It exhaustively tries every combination of a specified list of values for hyperparameters.
C. It uses gradient descent to find optimal hyperparameters.
D. It randomly samples hyperparameters from a distribution.

20 What is the major drawback of Grid Search?

A. It is difficult to implement.
B. It does not find the optimal parameters.
C. It suffers from the Curse of Dimensionality (computationally expensive with many parameters).
D. It only works for Decision Trees.

21 How does Random Search differ from Grid Search?

A. It checks more combinations than Grid Search.
B. It is always slower than Grid Search.
C. It samples a fixed number of parameter settings from specified distributions.
D. It guarantees finding the global optimum.

22 According to Bergstra and Bengio, why is Random Search often more efficient than Grid Search?

A. Because random numbers are faster to generate.
B. Because usually only a few hyperparameters are actually important for model performance.
C. Because Grid Search introduces bias.
D. Because Random Search uses deep learning.

23 What happens if we tune hyperparameters on the Test Set?

A. Information leakage occurs, leading to an optimistic bias in performance estimation.
B. Nothing; this is standard practice.
C. The training time decreases.
D. The model will generalize better.

24 Which technique is commonly used alongside Grid Search to evaluate the performance of each parameter combination?

A. Clustering
B. Standardization
C. Principal Component Analysis
D. K-Fold Cross-Validation

25 In a Bagging classifier, if the base models are unstable (e.g., fully grown Decision Trees), what is the expected outcome?

A. The ensemble will perform worse than a single model.
B. The ensemble will increase bias significantly.
C. Bagging cannot be used with unstable models.
D. The ensemble will significantly reduce variance and improve accuracy.

26 If you perform a Grid Search with: Parameter A = [1, 2, 3], Parameter B = [10, 20], and 5-fold Cross-Validation, how many total training runs are executed?

A.
B.
C.
D.

27 What is Stacking usually vulnerable to if not implemented correctly with cross-validation?

A. Convergence failure
B. Underfitting
C. Data leakage / Overfitting on the training data
D. High bias

28 In Gradient Boosting, what is the role of the Learning Rate (shrinkage)?

A. It sets the random seed.
B. It scales the contribution of each tree; lower values require more trees but improve generalization.
C. It determines the number of features to select.
D. It controls the size of the tree.

29 Which of the following ensemble methods uses Decision Stumps as the default base estimator?

A. AdaBoost
B. Random Forest
C. Bagging
D. Stacking

30 What is the key difference between Stacking and Blending?

A. Stacking typically uses cross-validated predictions for the meta-learner; Blending uses a hold-out validation set.
B. Stacking uses regression; Blending uses classification.
C. Blending is an older name for Bagging.
D. Stacking is parallel; Blending is sequential.

31 When performing hyperparameter tuning for a Decision Tree, which parameter typically controls overfitting?

A. Splitter (Best/Random)
B. Criterion (Gini/Entropy)
C. Random State
D. Max Depth

32 Which theoretical theorem states that if individual classifiers are independent and better than random guessing, the ensemble accuracy approaches 1 as the number of classifiers increases?

A. Condorcet's Jury Theorem
B. Central Limit Theorem
C. No Free Lunch Theorem
D. Bayes Theorem

33 In Random Forest, increasing the number of trees () typically:

A. Causes overfitting.
B. Decreases the variance up to a point without significantly increasing overfitting.
C. Increases the bias significantly.
D. Makes the model faster to train.

34 Which method is best suited if you have high-variance models (e.g., unpruned decision trees)?

A. Linear Regression
B. Boosting
C. Bagging
D. Logistic Regression

35 Which method is best suited if you have high-bias models (e.g., shallow trees)?

A. Naive Bayes
B. Clustering
C. Bagging
D. Boosting

36 What is the OOB (Out-Of-Bag) Error used for?

A. To select features in Grid Search.
B. To calculate the gradient in boosting.
C. To stop the training early.
D. To estimate the generalization error of a Bagging ensemble without needing a separate validation set.

37 In the context of hyperparameter tuning, what is a continuous hyperparameter?

A. Learning rate ()
B. Number of trees
C. Depth of a tree
D. Number of features

38 Why might one choose XGBoost over standard Gradient Boosting?

A. XGBoost includes regularization (L1/L2) and is optimized for speed/scalability.
B. XGBoost does not support regression.
C. XGBoost is slower.
D. XGBoost is a bagging technique.

39 What is the Base Estimator in a heterogeneous Stacking ensemble?

A. It must be the same algorithm with different hyperparameters.
B. It must be a Neural Network.
C. It can be any supervised learning algorithm (SVM, KNN, Tree, etc.).
D. It must be a Decision Tree.

40 Which search strategy uses probability to choose the next set of hyperparameters based on past results (e.g., using Gaussian Processes)?

A. Exhaustive Search
B. Grid Search
C. Bayesian Optimization
D. Random Search

41 In Grid Search, if the optimal value lies between two grid points, the method will:

A. Fail.
B. Automatically interpolate to find it.
C. Select the closest defined grid point.
D. Switch to Random Search.

42 Which of the following is an advantage of Ensemble Methods?

A. Low training time.
B. Interpretability (easy to explain distinct rules).
C. Compactness (small model size).
D. Robustness and Stability.

43 In a Voting Classifier, what requirement must be met to use Soft Voting?

A. The data must be linearly separable.
B. There must be an odd number of classifiers.
C. The base classifiers must support the predict_proba method.
D. The base classifiers must be Decision Trees.

44 When defining a parameter grid for SVM, which parameters are commonly tuned?

A. and (Gamma)
B. Learning rate and momentum
C. and
D. and distance metric

45 What is the concept of Feature Subsampling in Gradient Boosting?

A. Removing features that are not important.
B. Using PCA before training.
C. Using only a random fraction of features at each split or tree construction to reduce variance.
D. Manually selecting features.

46 A Random Forest with features total. For classification, what is the recommended number of features to search at each split?

A.
B.
C.
D.

47 Why is Accuracy sometimes a poor metric to optimize during hyperparameter tuning?

A. In imbalanced datasets, it can be misleading (e.g., predicting the majority class exclusively).
B. It is computationally expensive to calculate.
C. It is not differentiable.
D. Grid search does not support accuracy.

48 In Stacking, the Level-0 models are:

A. The models used for feature selection.
B. The meta-learners.
C. The base models trained on the original dataset.
D. The final output layer.

49 Which component of the error does Random Forest specifically aim to keep low compared to a single Decision Tree?

A. Noise
B. Computation time
C. Variance
D. Bias

50 When using Random Search, if you increase the number of iterations:

A. The probability of finding the optimal parameters decreases.
B. The search space shrinks.
C. The probability of finding a near-optimal combination increases.
D. The computational cost decreases.
Previous MCQ
Unit 2
Next MCQ
Unit 4