Back to Subject

Unit 3: Ensemble Methods and Hyperparameter Tuning - Practice Quiz

INT395 — Supervised Learning 50 Questions
0 Correct 0 Wrong 50 Left
0/50

1 What is the primary motivation behind using Ensemble Methods in machine learning?

A. To combine multiple weak models to improve overall performance and generalization
B. To eliminate the need for hyperparameter tuning
C. To increase the computational speed of training models
D. To reduce the number of features in the dataset

2 Which of the following statements best explains how Ensemble methods reduce error according to the bias-variance decomposition?

A. They always reduce bias without affecting variance.
B. They always reduce variance without affecting bias.
C. Bagging primarily reduces bias, while Boosting primarily reduces variance.
D. Bagging primarily reduces variance, while Boosting primarily reduces bias.

3 In the context of ensemble learning, what is a Weak Learner?

A. A model that performs slightly better than random guessing
B. A model that has 0% training error
C. A model that overfits the data significantly
D. A model that has too many parameters

4 What does Bagging stand for?

A. Binary Aggregating
B. Basic Aggregating
C. Bootstrap Aggregating
D. Backward Aggregating

5 Which statistical technique involves sampling data subsets with replacement?

A. Cross-Validation
B. Jackknife
C. Bootstrapping
D. Stratification

6 In a Random Forest, which two randomization techniques are combined?

A. L1 and L2 regularization
B. Bootstrap sampling and random feature selection
C. Grid Search and Random Search
D. Boosting and Bagging

7 If you are training a Bagging ensemble with samples, approximately what fraction of samples are left out of a single bootstrap sample (Out-Of-Bag)?

A.
B.
C.
D.

8 Which of the following is true regarding the parallelization of Bagging and Boosting?

A. Boosting is easy to parallelize, whereas Bagging is inherently sequential.
B. Both Bagging and Boosting can be easily parallelized.
C. Neither can be parallelized.
D. Bagging is easy to parallelize, whereas Boosting is inherently sequential.

9 In AdaBoost, how are the weights of training instances updated after each iteration?

A. Weights are kept constant throughout training.
B. Weights are assigned randomly.
C. Correctly classified instances are given higher weights.
D. Misclassified instances are given higher weights.

10 What is the main difference between AdaBoost and Gradient Boosting?

A. AdaBoost minimizes the loss function using gradient descent, while Gradient Boosting uses weighted voting.
B. AdaBoost cannot be used for regression, while Gradient Boosting can.
C. AdaBoost changes sample weights, while Gradient Boosting fits the new predictor to the residual errors of the previous predictor.
D. There is no difference; they are synonyms.

11 In the context of Stacking, what is a Meta-Learner?

A. A model used for hyperparameter tuning
B. A specific type of Deep Neural Network
C. The first layer of base models
D. A model that learns how to combine the predictions of the base models

12 Which ensemble method is mathematically represented by , where is the learning rate?

A. Hard Voting
B. Random Forest
C. Gradient Boosting
D. Stacking

13 What is the primary risk when using Boosting with a large number of iterations (trees)?

A. Overfitting
B. Underfitting
C. High bias
D. Vanishing gradients

14 What is Hard Voting in ensemble classifiers?

A. Weighting votes based on classifier confidence
B. Taking the majority class prediction as the final output
C. Averaging the probabilities of all classifiers
D. Using a meta-model to decide the vote

15 What is Soft Voting?

A. Randomly selecting a classifier's output
B. Predicting the class with the highest summed predicted probability across classifiers
C. Predicting the class with the most votes
D. Using a soft-margin SVM as the ensemble

16 Why is diversity important in an ensemble?

A. It simplifies the hyperparameter tuning process.
B. It allows models to make independent errors, which cancel out when aggregated.
C. It ensures all models are identical.
D. It increases the bias of the ensemble.

17 Which of the following is NOT a Hyperparameter?

A. The depth of a decision tree
B. The learning rate in Gradient Descent
C. The weights learned by a linear regression model
D. The number of neighbors () in KNN

18 What is the primary purpose of Hyperparameter Tuning?

A. To visualize the results
B. To select the optimal configuration for the learning algorithm to maximize performance
C. To train the model parameters like weights and biases
D. To clean the dataset

19 How does Grid Search work?

A. It randomly samples hyperparameters from a distribution.
B. It exhaustively tries every combination of a specified list of values for hyperparameters.
C. It manually asks the user to input values during training.
D. It uses gradient descent to find optimal hyperparameters.

20 What is the major drawback of Grid Search?

A. It is difficult to implement.
B. It suffers from the Curse of Dimensionality (computationally expensive with many parameters).
C. It only works for Decision Trees.
D. It does not find the optimal parameters.

21 How does Random Search differ from Grid Search?

A. It samples a fixed number of parameter settings from specified distributions.
B. It checks more combinations than Grid Search.
C. It is always slower than Grid Search.
D. It guarantees finding the global optimum.

22 According to Bergstra and Bengio, why is Random Search often more efficient than Grid Search?

A. Because usually only a few hyperparameters are actually important for model performance.
B. Because random numbers are faster to generate.
C. Because Random Search uses deep learning.
D. Because Grid Search introduces bias.

23 What happens if we tune hyperparameters on the Test Set?

A. The model will generalize better.
B. The training time decreases.
C. Information leakage occurs, leading to an optimistic bias in performance estimation.
D. Nothing; this is standard practice.

24 Which technique is commonly used alongside Grid Search to evaluate the performance of each parameter combination?

A. K-Fold Cross-Validation
B. Principal Component Analysis
C. Standardization
D. Clustering

25 In a Bagging classifier, if the base models are unstable (e.g., fully grown Decision Trees), what is the expected outcome?

A. Bagging cannot be used with unstable models.
B. The ensemble will significantly reduce variance and improve accuracy.
C. The ensemble will increase bias significantly.
D. The ensemble will perform worse than a single model.

26 If you perform a Grid Search with: Parameter A = [1, 2, 3], Parameter B = [10, 20], and 5-fold Cross-Validation, how many total training runs are executed?

A.
B.
C.
D.

27 What is Stacking usually vulnerable to if not implemented correctly with cross-validation?

A. Underfitting
B. Convergence failure
C. Data leakage / Overfitting on the training data
D. High bias

28 In Gradient Boosting, what is the role of the Learning Rate (shrinkage)?

A. It scales the contribution of each tree; lower values require more trees but improve generalization.
B. It determines the number of features to select.
C. It controls the size of the tree.
D. It sets the random seed.

29 Which of the following ensemble methods uses Decision Stumps as the default base estimator?

A. Stacking
B. Random Forest
C. AdaBoost
D. Bagging

30 What is the key difference between Stacking and Blending?

A. Stacking uses regression; Blending uses classification.
B. Stacking is parallel; Blending is sequential.
C. Stacking typically uses cross-validated predictions for the meta-learner; Blending uses a hold-out validation set.
D. Blending is an older name for Bagging.

31 When performing hyperparameter tuning for a Decision Tree, which parameter typically controls overfitting?

A. Random State
B. Max Depth
C. Criterion (Gini/Entropy)
D. Splitter (Best/Random)

32 Which theoretical theorem states that if individual classifiers are independent and better than random guessing, the ensemble accuracy approaches 1 as the number of classifiers increases?

A. No Free Lunch Theorem
B. Condorcet's Jury Theorem
C. Bayes Theorem
D. Central Limit Theorem

33 In Random Forest, increasing the number of trees () typically:

A. Causes overfitting.
B. Decreases the variance up to a point without significantly increasing overfitting.
C. Makes the model faster to train.
D. Increases the bias significantly.

34 Which method is best suited if you have high-variance models (e.g., unpruned decision trees)?

A. Bagging
B. Linear Regression
C. Logistic Regression
D. Boosting

35 Which method is best suited if you have high-bias models (e.g., shallow trees)?

A. Naive Bayes
B. Clustering
C. Bagging
D. Boosting

36 What is the OOB (Out-Of-Bag) Error used for?

A. To calculate the gradient in boosting.
B. To stop the training early.
C. To select features in Grid Search.
D. To estimate the generalization error of a Bagging ensemble without needing a separate validation set.

37 In the context of hyperparameter tuning, what is a continuous hyperparameter?

A. Learning rate ()
B. Number of features
C. Depth of a tree
D. Number of trees

38 Why might one choose XGBoost over standard Gradient Boosting?

A. XGBoost does not support regression.
B. XGBoost is a bagging technique.
C. XGBoost is slower.
D. XGBoost includes regularization (L1/L2) and is optimized for speed/scalability.

39 What is the Base Estimator in a heterogeneous Stacking ensemble?

A. It must be the same algorithm with different hyperparameters.
B. It can be any supervised learning algorithm (SVM, KNN, Tree, etc.).
C. It must be a Decision Tree.
D. It must be a Neural Network.

40 Which search strategy uses probability to choose the next set of hyperparameters based on past results (e.g., using Gaussian Processes)?

A. Random Search
B. Grid Search
C. Exhaustive Search
D. Bayesian Optimization

41 In Grid Search, if the optimal value lies between two grid points, the method will:

A. Select the closest defined grid point.
B. Fail.
C. Automatically interpolate to find it.
D. Switch to Random Search.

42 Which of the following is an advantage of Ensemble Methods?

A. Robustness and Stability.
B. Interpretability (easy to explain distinct rules).
C. Compactness (small model size).
D. Low training time.

43 In a Voting Classifier, what requirement must be met to use Soft Voting?

A. The base classifiers must support the predict_proba method.
B. The data must be linearly separable.
C. There must be an odd number of classifiers.
D. The base classifiers must be Decision Trees.

44 When defining a parameter grid for SVM, which parameters are commonly tuned?

A. and (Gamma)
B. and
C. Learning rate and momentum
D. and distance metric

45 What is the concept of Feature Subsampling in Gradient Boosting?

A. Using only a random fraction of features at each split or tree construction to reduce variance.
B. Using PCA before training.
C. Removing features that are not important.
D. Manually selecting features.

46 A Random Forest with features total. For classification, what is the recommended number of features to search at each split?

A.
B.
C.
D.

47 Why is Accuracy sometimes a poor metric to optimize during hyperparameter tuning?

A. It is computationally expensive to calculate.
B. It is not differentiable.
C. In imbalanced datasets, it can be misleading (e.g., predicting the majority class exclusively).
D. Grid search does not support accuracy.

48 In Stacking, the Level-0 models are:

A. The final output layer.
B. The models used for feature selection.
C. The meta-learners.
D. The base models trained on the original dataset.

49 Which component of the error does Random Forest specifically aim to keep low compared to a single Decision Tree?

A. Computation time
B. Variance
C. Noise
D. Bias

50 When using Random Search, if you increase the number of iterations:

A. The computational cost decreases.
B. The probability of finding a near-optimal combination increases.
C. The probability of finding the optimal parameters decreases.
D. The search space shrinks.
Previous MCQ
Unit 2
Next MCQ
Unit 4