Back to Subject

Unit 6: Spark ML Programming and PySpark - Practice Quiz

INT315 — Cluster Computing 60 Questions
0 Correct 0 Wrong 60 Left
0/60

1 What is machine learning primarily used for?

Introduction to machine learning and PySpark Easy
A. Creating database tables
B. Learning patterns from data
C. Designing computer hardware
D. Managing network cables

2 What is PySpark?

Introduction to machine learning and PySpark Easy
A. A web server for Python applications
B. A Python API for Apache Spark
C. A Java database management system
D. A Python tool used only for drawing charts and generating detailed graphical reports

3 Which machine learning technique uses labeled training data?

Techniques of machine learning Easy
A. Random processing
B. Supervised learning
C. Data compression
D. Unsupervised learning

4 Which technique finds groups or patterns in data without known labels?

Techniques of machine learning Easy
A. Rule-based sorting
B. Supervised learning
C. Unsupervised learning
D. Database indexing

5 What is the main purpose of Spark MLlib?

Introduction to Spark MLlib Easy
A. Designing web page layouts
B. Managing only relational database transactions across several independent servers
C. Creating operating system drivers
D. Providing scalable machine learning tools

6 In Spark ML, what does an Estimator produce when its fit() method is called?

Key concepts of Spark ML Easy
A. A Python package
B. A raw DataFrame
C. A database schema
D. A fitted Model

7 What is a Spark ML Pipeline used for?

Key concepts of Spark ML Easy
A. Combining multiple ML stages
B. Storing files in local folders
C. Replacing all DataFrame columns with a single manually calculated numeric value
D. Starting a web application

8 Which PySpark package commonly contains classification and regression algorithms?

Spark ML algorithms using PySpark Easy
A. pyspark.html
B. pyspark.net
C. pyspark.sql
D. pyspark.ml

9 What type of value does linear regression usually predict?

Linear regression Easy
A. A sequence of database commands
B. A continuous numeric value
C. A cluster identifier
D. A class probability only

10 Logistic regression is commonly used for which task?

Logistic regression Easy
A. Clustering
B. Classification
C. File compression
D. Database normalization

11 How does a decision tree generally make a prediction?

Decision tree Easy
A. By following feature-based branches
B. By calculating only the average value
C. By checking every possible model and permanently storing all results in memory
D. By assigning points to random clusters

12 What does the value represent in K-means clustering?

K-means Easy
A. The number of features
B. The number of clusters
C. The number of test records
D. The number of labels

13 What is the main goal of a Support Vector Machine classifier?

SVM Easy
A. Finding a separating hyperplane
B. Finding cluster centroids
C. Constructing a complete decision tree with one separate branch for every training record
D. Estimating a numeric average

14 Naive Bayes is based on which mathematical idea?

Naive Bayes Easy
A. Bayes' theorem
B. Linear matrix inversion
C. Pythagoras' theorem
D. Euclidean clustering

15 Why is a test dataset used when evaluating a machine learning model?

Evaluation and performance matrix Easy
A. To increase the number of model features
B. To measure performance on unseen data
C. To rename the columns in the dataset
D. To replace every incorrect prediction with the known correct label before scoring

16 In a confusion matrix, what is a true positive?

Confusion matrix Easy
A. A negative case predicted as positive
B. A negative case predicted as negative
C. A positive case predicted as positive
D. A positive case predicted as negative

17 What does the metric indicate in regression?

R2 Easy
A. The proportion of variance explained
B. The total number of training rows
C. The number of prediction classes
D. The exact percentage of records that must always be removed from the dataset

18 What does RMSE stand for?

RMSE Easy
A. Regression Mean Sampling Equation
B. Root Mean Squared Error
C. Random Model Scoring Estimate
D. Relative Matrix Scaling Error

19 What does MAE measure?

MAE Easy
A. The maximum error in the dataset
B. The average absolute prediction error
C. The mean of all target values after incorrect predictions have been removed
D. The average number of model features

20 What does a correlation heat map usually display?

Correlation heat map Easy
A. Locations of files in a cluster
B. Relationships between numeric variables
C. The execution order of Python statements
D. Detailed source code for every machine learning model used in an application

21 A PySpark pipeline must train a model on a dataset that is too large for a single machine. Which capability of Spark is most relevant to this task?

Introduction to machine learning and PySpark Medium
A. Replacement of training data with metadata
B. Automatic removal of all noisy records
C. Distributed processing across cluster nodes
D. Conversion of every feature into text

22 A company has customer records without labels and wants to group customers according to purchasing behavior. Which machine learning technique is most appropriate?

Techniques of machine learning Medium
A. Supervised classification
B. Supervised regression
C. Reinforcement learning
D. Unsupervised clustering

23 Which statement best describes the purpose of Spark MLlib?

Introduction to Spark MLlib Medium
A. It manages network addresses for clusters
B. It replaces the Spark execution engine
C. It stores only unstructured files
D. It provides distributed machine learning algorithms

24 In Spark ML, what is the main purpose of a Pipeline?

Key concepts of Spark ML Medium
A. To chain transformers and estimators
B. To convert labels into file names
C. To increase the number of cluster nodes
D. To remove the need for model evaluation

25 A Spark ML model expects all predictor values in one column of vector type. Which component is commonly used to create this column from several feature columns?

Key concepts of Spark ML Medium
A. CrossValidator
B. StringIndexer
C. RegressionEvaluator
D. VectorAssembler

26 A PySpark workflow uses StringIndexer on a categorical column before training a classifier. What problem does this step primarily address?

Spark ML algorithms using PySpark Medium
A. It groups observations into clusters
B. It converts categorical values into numeric indices
C. It removes highly correlated features
D. It calculates the model's prediction error

27 A linear regression model predicts house prices using area and number of rooms. If the coefficient for area is positive, what does it indicate when other features remain fixed?

Linear regression Medium
A. Price tends to decrease with area
B. Area has no effect on price
C. Area must be the target variable
D. Price tends to increase with area

28 A regression model has an RMSE of when predicting monthly sales. What is the most appropriate interpretation?

Linear regression Medium
A. Typical prediction error is about sales units
B. The model explains of sales variance
C. The model has an accuracy of
D. Exactly predictions are incorrect

29 A logistic regression model returns a probability of for the positive class. Using a threshold of , what class will normally be predicted?

Logistic regression Medium
A. No class until clustering
B. The negative class
C. The positive class
D. Both classes equally

30 A binary classifier produces many false negatives in a medical screening task. Which change may improve detection of positive cases?

Logistic regression Medium
A. Replace probabilities with cluster IDs
B. Increase the classification threshold
C. Remove the positive training labels
D. Lower the classification threshold

31 A decision tree classifies loan applications by repeatedly splitting records according to feature values. What is the main benefit of selecting a split with high information gain?

Decision tree Medium
A. It removes every numerical feature
B. It guarantees zero test error
C. It forces all leaves to have equal size
D. It creates purer child nodes

32 In K-means clustering, a data point is reassigned during an iteration when:

K-means Medium
A. Another centroid becomes closer
B. The evaluation metric reaches exactly one
C. Its label becomes a text value
D. The number of features becomes zero

33 A K-means model is trained with . What does the value represent?

K-means Medium
A. The number of requested clusters
B. The number of evaluation metrics
C. The number of input features
D. The number of training iterations

34 What objective is central to a linear Support Vector Machine classifier?

SVM Medium
A. Finding a boundary with a large margin
B. Counting the frequency of every word
C. Creating centroids for every class
D. Estimating a continuous target mean

35 The Naive Bayes classifier is called naive because it assumes that features are:

Naive Bayes Medium
A. Conditionally independent given the class
B. Independent of the predicted class
C. Always identical for every record
D. Numerically distributed around zero

36 A model achieves accuracy on a dataset where of records belong to one class. Why might accuracy be misleading?

Evaluation and performance matrix Medium
A. Accuracy cannot be calculated for classifiers
B. Accuracy uses only continuous target values
C. A majority-class predictor can achieve similar accuracy
D. Accuracy always equals the RMSE value

37 A fraud detection model labels a legitimate transaction as fraudulent. Which confusion matrix category describes this result?

Confusion matrix Medium
A. True negative
B. False positive
C. True positive
D. False negative

38 A regression model has . Which interpretation is most appropriate?

R2 Medium
A. The model explains about of target variance
B. The model makes errors of exactly units
C. The model correctly classifies of records
D. The model predicts of targets as positive

39 Two regression models have MAE values of and on the same test set. Which model has better average absolute performance?

MAE Medium
A. The model with MAE
B. The model with MAE
C. Both models have equal performance
D. The model with the larger target mean

40 A correlation heat map shows a correlation of between two predictor variables. What concern should be investigated before fitting a linear model?

Correlation heat map Medium
A. Possible multicollinearity between predictors
B. Automatic conversion to categorical labels
C. Guaranteed absence of a linear relationship
D. Immediate proof that the model is unbiased

41 A PySpark classification workflow imputes missing values, standardizes features, and tunes a logistic regression model with CrossValidator. Which design best prevents information leakage while estimating generalization performance?

Introduction to machine learning and PySpark Hard
A. Cache the fully transformed dataset before randomly creating validation folds
B. Place preprocessing and classification in one pipeline and cross-validate the pipeline
C. Fit preprocessing on each validation fold and transform its corresponding training fold
D. Fit imputation and scaling before cross-validation, then tune only the classifier

42 A binary dataset contains 0.2% positive examples. A classifier that always predicts the negative class obtains 99.8% accuracy. Which evaluation strategy is most appropriate when missing a positive is costly?

Techniques of machine learning Hard
A. Optimize after encoding positive and negative labels as one and zero
B. Optimize recall subject to an acceptable precision using stratified validation data
C. Optimize accuracy using randomly sampled folds with unchanged class proportions
D. Optimize specificity after undersampling both classes to equal validation counts

43 A legacy application uses pyspark.mllib models on RDDs, while a new workflow requires pipelines, parameter grids, and DataFrame-based feature transformers. Which migration decision is technically sound?

Introduction to Spark MLlib Hard
A. Convert DataFrames to local NumPy arrays because parameter grids require local training
B. Mix RDD transformers directly into a pipeline because both APIs share stage contracts
C. Keep pyspark.mllib because RDD models automatically participate in ML pipelines
D. Use pyspark.ml because its DataFrame API supports estimators, transformers, and pipelines

44 Consider pipeline = Pipeline(stages=[assembler, scaler, lr]), where scaler and lr are estimators. After calling model = pipeline.fit(train), which statement is correct?

Key concepts of Spark ML Hard
A. model can transform only train because fitted stages store its complete rows
B. model contains fitted transformer stages and can transform new DataFrames
C. model retains unfitted estimators and refits them whenever transform is called
D. model replaces every stage with an evaluator that computes validation metrics

45 A parameter grid compares logistic regression models using CrossValidator, and the final model must be refitted on all training data using the best parameters. What does Spark's CrossValidator.fit(train) return?

Spark ML algorithms using PySpark Hard
A. A transformer fitted only on the validation fold with the highest metric
B. A CrossValidatorModel containing a best model refitted on the full input dataset
C. A parameter map that must be passed to a separate estimator fitting operation
D. A list of fold-specific models whose predictions must be manually averaged

46 Two predictors are nearly perfectly collinear, and an unregularized Spark linear regression model produces unstable coefficients across samples. Which parameter change most directly stabilizes the solution while generally retaining both predictors?

Linear regression Hard
A. Set maxIter to one and keep both regularization parameters zero
B. Set regParam above zero and set elasticNetParam to zero
C. Set regParam to zero and set elasticNetParam to one
D. Set fitIntercept to false and set elasticNetParam to one

47 For labels and predictions , what are the RMSE and values, respectively?

Linear regression Hard
A. and
B. and
C. and
D. and

48 A fitted binary logistic regression model outputs calibrated positive-class probabilities. The false-negative cost is four times the false-positive cost, and both classes have equal prior importance. Under minimum expected cost, at what probability should the model predict the positive class?

Logistic regression Hard
A.
B.
C.
D.

49 A multinomial logistic regression model uses three classes and a feature vector of size five, with intercepts enabled. Ignoring Spark's internal reference-category representation choices, how many class-specific coefficient and intercept values conceptually define the softmax scores before accounting for non-identifiability?

Logistic regression Hard
A.
B.
C.
D.

50 A categorical feature has 40 distinct categories, but a Spark decision tree is configured with maxBins=32. What is the most appropriate correction?

Decision tree Hard
A. Increase maxMemoryInMB because category count is unrelated to maxBins
B. Increase maxDepth to at least 40 so each category receives a tree level
C. Reduce minInstancesPerNode to zero so categories can share split bins
D. Increase maxBins to at least 40 so the categorical feature can be represented

51 A decision tree has nearly zero training error but poor validation performance. Increasing maxDepth worsens the gap. Which change most directly regularizes the tree structure?

Decision tree Hard
A. Increase maxBins and decrease minInfoGain
B. Decrease maxDepth and increase minInstancesPerNode
C. Increase maxDepth and decrease minInstancesPerNode
D. Decrease maxBins and decrease minInfoGain

52 A K-means dataset contains annual income measured in dollars and age measured in years. Income has variance several orders of magnitude larger than age. What is the likely effect of fitting without scaling?

K-means Hard
A. Neither feature will affect assignments because K-means uses only angular similarity
B. Both features will contribute equally because K-means normalizes columns internally
C. Income will dominate Euclidean distances and therefore dominate cluster assignments
D. Age will dominate Euclidean distances because it contains numerically smaller values

53 A K-means run reports a lower training cost for than for , but a substantially lower validation silhouette score for . Which conclusion is best supported?

K-means Hard
A. The silhouette comparison is meaningless because it cannot compare different values of
B. The solution is necessarily better because training cost always determines generalization
C. The extra centroids reduce within-cluster error but produce less separated validation clusters
D. The solution is invalid because training cost must increase when decreases

54 A team attempts to train Spark LinearSVC directly on labels . Which modification is required?

SVM Hard
A. Construct one-vs-rest binary classifiers because LinearSVC is binary
B. Increase aggregationDepth because multiclass support depends on tree aggregation
C. Set family to multinomial because LinearSVC supports softmax classification
D. Convert labels to strings because LinearSVC infers multiclass boundaries

55 For a linear SVM, a training point satisfies . What is its hinge loss , and where is the point relative to the margin?

SVM Hard
A. , correctly classified but inside the margin
B. , misclassified and outside the margin
C. , correctly classified and outside the margin
D. , correctly classified and on the margin

56 A Bernoulli Naive Bayes model is trained on document features that contain term frequencies such as , , and . Which preprocessing best matches the model's likelihood assumptions?

Naive Bayes Hard
A. Normalize each row to unit length while preserving every original frequency ratio
B. Binarize each feature to indicate whether the corresponding term is present
C. Standardize each feature to zero mean while preserving negative standardized values
D. Center each feature by subtracting its corpus-wide average term frequency

57 A three-class classifier performs well on a dominant class but poorly on two rare classes. Which metric is most sensitive to the rare-class failures while assigning each class equal importance?

Evaluation and performance matrix Hard
A. Macro-averaged recall
B. Micro-averaged precision
C. Overall accuracy
D. Weighted-averaged recall

58 A classifier produces , , , and . If the threshold is changed so that 20 false positives become true negatives while 10 true positives become false negatives, what are the new precision and recall?

Confusion matrix Hard
A. and
B. and
C. and
D. and

59 A regression model evaluated on unseen data has . Which interpretation is correct?

R2 Hard
A. It explains 35% of the target variance after reversing the sign of its predictions
B. Its squared-error sum is 35% larger than the mean-prediction baseline's squared-error sum
C. Its predictions have a Pearson correlation of with the observed labels
D. Its RMSE is exactly 35% larger than the standard deviation of the observed labels

60 Two regression models are evaluated on the same four cases. Model A has absolute errors , while Model B has absolute errors . Which comparison is correct?

RMSE Hard
A. Model A has lower MAE and lower RMSE than Model B
B. Model A has higher MAE but lower RMSE than Model B
C. Model A has equal MAE and equal RMSE to Model B
D. Model A has lower MAE but higher RMSE than Model B
Previous MCQ
Unit 5