Back to Subject

Unit 6: Spark ML Programming and PySpark - Practice Quiz

INT315 — Cluster Computing 60 Questions
0 Correct 0 Wrong 60 Left
0/60

1 What is machine learning primarily used for?

Introduction to machine learning and PySpark Easy
A. Managing network cables
B. Learning patterns from data
C. Creating database tables
D. Designing computer hardware

2 What is PySpark?

Introduction to machine learning and PySpark Easy
A. A Python tool used only for drawing charts and generating detailed graphical reports
B. A Python API for Apache Spark
C. A Java database management system
D. A web server for Python applications

3 Which machine learning technique uses labeled training data?

Techniques of machine learning Easy
A. Data compression
B. Supervised learning
C. Unsupervised learning
D. Random processing

4 Which technique finds groups or patterns in data without known labels?

Techniques of machine learning Easy
A. Unsupervised learning
B. Supervised learning
C. Database indexing
D. Rule-based sorting

5 What is the main purpose of Spark MLlib?

Introduction to Spark MLlib Easy
A. Designing web page layouts
B. Creating operating system drivers
C. Providing scalable machine learning tools
D. Managing only relational database transactions across several independent servers

6 In Spark ML, what does an Estimator produce when its fit() method is called?

Key concepts of Spark ML Easy
A. A Python package
B. A database schema
C. A fitted Model
D. A raw DataFrame

7 What is a Spark ML Pipeline used for?

Key concepts of Spark ML Easy
A. Starting a web application
B. Combining multiple ML stages
C. Replacing all DataFrame columns with a single manually calculated numeric value
D. Storing files in local folders

8 Which PySpark package commonly contains classification and regression algorithms?

Spark ML algorithms using PySpark Easy
A. pyspark.html
B. pyspark.net
C. pyspark.sql
D. pyspark.ml

9 What type of value does linear regression usually predict?

Linear regression Easy
A. A sequence of database commands
B. A continuous numeric value
C. A class probability only
D. A cluster identifier

10 Logistic regression is commonly used for which task?

Logistic regression Easy
A. Database normalization
B. Classification
C. Clustering
D. File compression

11 How does a decision tree generally make a prediction?

Decision tree Easy
A. By following feature-based branches
B. By checking every possible model and permanently storing all results in memory
C. By calculating only the average value
D. By assigning points to random clusters

12 What does the value represent in K-means clustering?

K-means Easy
A. The number of labels
B. The number of features
C. The number of test records
D. The number of clusters

13 What is the main goal of a Support Vector Machine classifier?

SVM Easy
A. Constructing a complete decision tree with one separate branch for every training record
B. Finding a separating hyperplane
C. Finding cluster centroids
D. Estimating a numeric average

14 Naive Bayes is based on which mathematical idea?

Naive Bayes Easy
A. Pythagoras' theorem
B. Linear matrix inversion
C. Bayes' theorem
D. Euclidean clustering

15 Why is a test dataset used when evaluating a machine learning model?

Evaluation and performance matrix Easy
A. To replace every incorrect prediction with the known correct label before scoring
B. To measure performance on unseen data
C. To increase the number of model features
D. To rename the columns in the dataset

16 In a confusion matrix, what is a true positive?

Confusion matrix Easy
A. A positive case predicted as negative
B. A positive case predicted as positive
C. A negative case predicted as negative
D. A negative case predicted as positive

17 What does the metric indicate in regression?

R2 Easy
A. The proportion of variance explained
B. The exact percentage of records that must always be removed from the dataset
C. The number of prediction classes
D. The total number of training rows

18 What does RMSE stand for?

RMSE Easy
A. Root Mean Squared Error
B. Relative Matrix Scaling Error
C. Regression Mean Sampling Equation
D. Random Model Scoring Estimate

19 What does MAE measure?

MAE Easy
A. The average absolute prediction error
B. The mean of all target values after incorrect predictions have been removed
C. The average number of model features
D. The maximum error in the dataset

20 What does a correlation heat map usually display?

Correlation heat map Easy
A. Detailed source code for every machine learning model used in an application
B. The execution order of Python statements
C. Locations of files in a cluster
D. Relationships between numeric variables

21 A PySpark pipeline must train a model on a dataset that is too large for a single machine. Which capability of Spark is most relevant to this task?

Introduction to machine learning and PySpark Medium
A. Distributed processing across cluster nodes
B. Replacement of training data with metadata
C. Conversion of every feature into text
D. Automatic removal of all noisy records

22 A company has customer records without labels and wants to group customers according to purchasing behavior. Which machine learning technique is most appropriate?

Techniques of machine learning Medium
A. Supervised regression
B. Reinforcement learning
C. Unsupervised clustering
D. Supervised classification

23 Which statement best describes the purpose of Spark MLlib?

Introduction to Spark MLlib Medium
A. It stores only unstructured files
B. It provides distributed machine learning algorithms
C. It manages network addresses for clusters
D. It replaces the Spark execution engine

24 In Spark ML, what is the main purpose of a Pipeline?

Key concepts of Spark ML Medium
A. To remove the need for model evaluation
B. To chain transformers and estimators
C. To increase the number of cluster nodes
D. To convert labels into file names

25 A Spark ML model expects all predictor values in one column of vector type. Which component is commonly used to create this column from several feature columns?

Key concepts of Spark ML Medium
A. VectorAssembler
B. StringIndexer
C. CrossValidator
D. RegressionEvaluator

26 A PySpark workflow uses StringIndexer on a categorical column before training a classifier. What problem does this step primarily address?

Spark ML algorithms using PySpark Medium
A. It removes highly correlated features
B. It calculates the model's prediction error
C. It converts categorical values into numeric indices
D. It groups observations into clusters

27 A linear regression model predicts house prices using area and number of rooms. If the coefficient for area is positive, what does it indicate when other features remain fixed?

Linear regression Medium
A. Area has no effect on price
B. Price tends to increase with area
C. Price tends to decrease with area
D. Area must be the target variable

28 A regression model has an RMSE of when predicting monthly sales. What is the most appropriate interpretation?

Linear regression Medium
A. Exactly predictions are incorrect
B. The model has an accuracy of
C. Typical prediction error is about sales units
D. The model explains of sales variance

29 A logistic regression model returns a probability of for the positive class. Using a threshold of , what class will normally be predicted?

Logistic regression Medium
A. No class until clustering
B. Both classes equally
C. The positive class
D. The negative class

30 A binary classifier produces many false negatives in a medical screening task. Which change may improve detection of positive cases?

Logistic regression Medium
A. Remove the positive training labels
B. Lower the classification threshold
C. Replace probabilities with cluster IDs
D. Increase the classification threshold

31 A decision tree classifies loan applications by repeatedly splitting records according to feature values. What is the main benefit of selecting a split with high information gain?

Decision tree Medium
A. It guarantees zero test error
B. It removes every numerical feature
C. It creates purer child nodes
D. It forces all leaves to have equal size

32 In K-means clustering, a data point is reassigned during an iteration when:

K-means Medium
A. The evaluation metric reaches exactly one
B. Another centroid becomes closer
C. Its label becomes a text value
D. The number of features becomes zero

33 A K-means model is trained with . What does the value represent?

K-means Medium
A. The number of evaluation metrics
B. The number of training iterations
C. The number of requested clusters
D. The number of input features

34 What objective is central to a linear Support Vector Machine classifier?

SVM Medium
A. Finding a boundary with a large margin
B. Creating centroids for every class
C. Counting the frequency of every word
D. Estimating a continuous target mean

35 The Naive Bayes classifier is called naive because it assumes that features are:

Naive Bayes Medium
A. Conditionally independent given the class
B. Independent of the predicted class
C. Numerically distributed around zero
D. Always identical for every record

36 A model achieves accuracy on a dataset where of records belong to one class. Why might accuracy be misleading?

Evaluation and performance matrix Medium
A. Accuracy cannot be calculated for classifiers
B. A majority-class predictor can achieve similar accuracy
C. Accuracy uses only continuous target values
D. Accuracy always equals the RMSE value

37 A fraud detection model labels a legitimate transaction as fraudulent. Which confusion matrix category describes this result?

Confusion matrix Medium
A. True negative
B. False negative
C. False positive
D. True positive

38 A regression model has . Which interpretation is most appropriate?

R2 Medium
A. The model makes errors of exactly units
B. The model explains about of target variance
C. The model correctly classifies of records
D. The model predicts of targets as positive

39 Two regression models have MAE values of and on the same test set. Which model has better average absolute performance?

MAE Medium
A. The model with MAE
B. Both models have equal performance
C. The model with the larger target mean
D. The model with MAE

40 A correlation heat map shows a correlation of between two predictor variables. What concern should be investigated before fitting a linear model?

Correlation heat map Medium
A. Immediate proof that the model is unbiased
B. Possible multicollinearity between predictors
C. Automatic conversion to categorical labels
D. Guaranteed absence of a linear relationship

41 A PySpark classification workflow imputes missing values, standardizes features, and tunes a logistic regression model with CrossValidator. Which design best prevents information leakage while estimating generalization performance?

Introduction to machine learning and PySpark Hard
A. Fit imputation and scaling before cross-validation, then tune only the classifier
B. Place preprocessing and classification in one pipeline and cross-validate the pipeline
C. Fit preprocessing on each validation fold and transform its corresponding training fold
D. Cache the fully transformed dataset before randomly creating validation folds

42 A binary dataset contains 0.2% positive examples. A classifier that always predicts the negative class obtains 99.8% accuracy. Which evaluation strategy is most appropriate when missing a positive is costly?

Techniques of machine learning Hard
A. Optimize recall subject to an acceptable precision using stratified validation data
B. Optimize accuracy using randomly sampled folds with unchanged class proportions
C. Optimize after encoding positive and negative labels as one and zero
D. Optimize specificity after undersampling both classes to equal validation counts

43 A legacy application uses pyspark.mllib models on RDDs, while a new workflow requires pipelines, parameter grids, and DataFrame-based feature transformers. Which migration decision is technically sound?

Introduction to Spark MLlib Hard
A. Keep pyspark.mllib because RDD models automatically participate in ML pipelines
B. Convert DataFrames to local NumPy arrays because parameter grids require local training
C. Mix RDD transformers directly into a pipeline because both APIs share stage contracts
D. Use pyspark.ml because its DataFrame API supports estimators, transformers, and pipelines

44 Consider pipeline = Pipeline(stages=[assembler, scaler, lr]), where scaler and lr are estimators. After calling model = pipeline.fit(train), which statement is correct?

Key concepts of Spark ML Hard
A. model retains unfitted estimators and refits them whenever transform is called
B. model replaces every stage with an evaluator that computes validation metrics
C. model contains fitted transformer stages and can transform new DataFrames
D. model can transform only train because fitted stages store its complete rows

45 A parameter grid compares logistic regression models using CrossValidator, and the final model must be refitted on all training data using the best parameters. What does Spark's CrossValidator.fit(train) return?

Spark ML algorithms using PySpark Hard
A. A parameter map that must be passed to a separate estimator fitting operation
B. A list of fold-specific models whose predictions must be manually averaged
C. A CrossValidatorModel containing a best model refitted on the full input dataset
D. A transformer fitted only on the validation fold with the highest metric

46 Two predictors are nearly perfectly collinear, and an unregularized Spark linear regression model produces unstable coefficients across samples. Which parameter change most directly stabilizes the solution while generally retaining both predictors?

Linear regression Hard
A. Set regParam above zero and set elasticNetParam to zero
B. Set maxIter to one and keep both regularization parameters zero
C. Set regParam to zero and set elasticNetParam to one
D. Set fitIntercept to false and set elasticNetParam to one

47 For labels and predictions , what are the RMSE and values, respectively?

Linear regression Hard
A. and
B. and
C. and
D. and

48 A fitted binary logistic regression model outputs calibrated positive-class probabilities. The false-negative cost is four times the false-positive cost, and both classes have equal prior importance. Under minimum expected cost, at what probability should the model predict the positive class?

Logistic regression Hard
A.
B.
C.
D.

49 A multinomial logistic regression model uses three classes and a feature vector of size five, with intercepts enabled. Ignoring Spark's internal reference-category representation choices, how many class-specific coefficient and intercept values conceptually define the softmax scores before accounting for non-identifiability?

Logistic regression Hard
A.
B.
C.
D.

50 A categorical feature has 40 distinct categories, but a Spark decision tree is configured with maxBins=32. What is the most appropriate correction?

Decision tree Hard
A. Reduce minInstancesPerNode to zero so categories can share split bins
B. Increase maxBins to at least 40 so the categorical feature can be represented
C. Increase maxMemoryInMB because category count is unrelated to maxBins
D. Increase maxDepth to at least 40 so each category receives a tree level

51 A decision tree has nearly zero training error but poor validation performance. Increasing maxDepth worsens the gap. Which change most directly regularizes the tree structure?

Decision tree Hard
A. Decrease maxDepth and increase minInstancesPerNode
B. Decrease maxBins and decrease minInfoGain
C. Increase maxBins and decrease minInfoGain
D. Increase maxDepth and decrease minInstancesPerNode

52 A K-means dataset contains annual income measured in dollars and age measured in years. Income has variance several orders of magnitude larger than age. What is the likely effect of fitting without scaling?

K-means Hard
A. Income will dominate Euclidean distances and therefore dominate cluster assignments
B. Both features will contribute equally because K-means normalizes columns internally
C. Age will dominate Euclidean distances because it contains numerically smaller values
D. Neither feature will affect assignments because K-means uses only angular similarity

53 A K-means run reports a lower training cost for than for , but a substantially lower validation silhouette score for . Which conclusion is best supported?

K-means Hard
A. The solution is necessarily better because training cost always determines generalization
B. The extra centroids reduce within-cluster error but produce less separated validation clusters
C. The solution is invalid because training cost must increase when decreases
D. The silhouette comparison is meaningless because it cannot compare different values of

54 A team attempts to train Spark LinearSVC directly on labels . Which modification is required?

SVM Hard
A. Construct one-vs-rest binary classifiers because LinearSVC is binary
B. Increase aggregationDepth because multiclass support depends on tree aggregation
C. Set family to multinomial because LinearSVC supports softmax classification
D. Convert labels to strings because LinearSVC infers multiclass boundaries

55 For a linear SVM, a training point satisfies . What is its hinge loss , and where is the point relative to the margin?

SVM Hard
A. , correctly classified but inside the margin
B. , misclassified and outside the margin
C. , correctly classified and on the margin
D. , correctly classified and outside the margin

56 A Bernoulli Naive Bayes model is trained on document features that contain term frequencies such as , , and . Which preprocessing best matches the model's likelihood assumptions?

Naive Bayes Hard
A. Normalize each row to unit length while preserving every original frequency ratio
B. Standardize each feature to zero mean while preserving negative standardized values
C. Binarize each feature to indicate whether the corresponding term is present
D. Center each feature by subtracting its corpus-wide average term frequency

57 A three-class classifier performs well on a dominant class but poorly on two rare classes. Which metric is most sensitive to the rare-class failures while assigning each class equal importance?

Evaluation and performance matrix Hard
A. Micro-averaged precision
B. Weighted-averaged recall
C. Overall accuracy
D. Macro-averaged recall

58 A classifier produces , , , and . If the threshold is changed so that 20 false positives become true negatives while 10 true positives become false negatives, what are the new precision and recall?

Confusion matrix Hard
A. and
B. and
C. and
D. and

59 A regression model evaluated on unseen data has . Which interpretation is correct?

R2 Hard
A. Its predictions have a Pearson correlation of with the observed labels
B. Its RMSE is exactly 35% larger than the standard deviation of the observed labels
C. Its squared-error sum is 35% larger than the mean-prediction baseline's squared-error sum
D. It explains 35% of the target variance after reversing the sign of its predictions

60 Two regression models are evaluated on the same four cases. Model A has absolute errors , while Model B has absolute errors . Which comparison is correct?

RMSE Hard
A. Model A has lower MAE but higher RMSE than Model B
B. Model A has higher MAE but lower RMSE than Model B
C. Model A has equal MAE and equal RMSE to Model B
D. Model A has lower MAE and lower RMSE than Model B
Previous MCQ
Unit 5