Unit 1 - Practice Quiz

The core idea of Machine Learning, as defined by Arthur Samuel, is to give computers the ability to learn without being explicitly programmed. It focuses on developing algorithms that can identify patterns and make decisions from data.

Supervised Learning is characterized by the use of labeled data. The algorithm learns to map input features to a known output label, which acts as a 'supervisor' during training.

Clustering is an unsupervised learning task that involves grouping a set of objects based on similarity, without any predefined labels. Customer segmentation is a classic example.

Artificial Intelligence is the broad concept of creating machines that can think or act intelligently. Machine Learning is a specific subset of AI that focuses on systems learning from data.

Parametric models, like Linear Regression, assume a specific functional form for the relationship they are trying to learn (e.g., a line). The model is fully defined by a fixed number of parameters, regardless of the amount of training data.

A discriminative model learns the conditional probability . Its primary goal is to learn how to distinguish (discriminate) between different classes by finding a decision boundary.

Before any model can be trained, data must be gathered, cleaned, and prepared into a suitable format. This is the foundational first step of any ML project.

The test set is held out during training and validation. It is used only once at the very end to estimate how well the model will generalize to new, real-world data.

Accuracy is a fundamental classification metric calculated as (Number of Correct Predictions) / (Total Number of Predictions).

Overfitting occurs when a model learns the training data too well, including its noise and random fluctuations, and therefore fails to generalize to new, unseen data.

Underfitting happens when a model is not complex enough to learn the relationship between the features and the target variable, leading to poor performance everywhere.

High bias means the model makes strong (and often incorrect) assumptions about the data, leading to underfitting. Low variance means the model's predictions are consistent but may be consistently wrong.

Spam filtering is a classic classification problem where machine learning models are trained to distinguish between spam and non-spam (ham) emails based on their content and metadata.

NumPy's core object is the ndarray, a powerful and efficient N-dimensional array that supports a wide range of mathematical operations.

The numpy.ones(shape) function returns a new array of a given shape and type, filled with ones.

np.arange(n) creates an array with values from 0 up to (but not including) n. Therefore, np.arange(5) creates [0, 1, 2, 3, 4].

When two NumPy arrays of the same shape are added, the operation is performed element-wise. So, the result is [1+4, 2+5, 3+6], which is [5, 7, 9].

This is an example of broadcasting. The scalar 10 is divided element-wise into the array, resulting in [10/10, 20/10, 30/10] which is [1., 2., 3.] (as a float array).

Broadcasting describes the set of rules by which NumPy performs element-wise operations on arrays of different shapes, effectively 'stretching' the smaller array to match the shape of the larger one without making actual copies of the data.

np.random.randint(low, high, size) is used to generate an array of a given size with random integers from the 'low' (inclusive) to 'high' (exclusive) range.

High variance indicates overfitting, where the model has learned the training data too well, including its noise. Gathering more training data is a classic technique to reduce variance, as it helps the model learn a more generalizable pattern. Increasing complexity or decreasing regularization would likely worsen the overfitting (increase variance). Using fewer features could simplify the model, which also reduces variance, but gathering more data is a more direct and generally effective approach.

The significant gap between training and validation accuracy indicates overfitting. The model is too complex. Pruning the tree by limiting its maximum depth (max_depth) or requiring more samples to form a leaf (min_samples_leaf) are common and effective methods to simplify the model, reduce its variance, and improve its ability to generalize.

Non-parametric models (like k-Nearest Neighbors or Decision Trees) do not make strong assumptions about the underlying data distribution. Their flexibility allows them to fit a wider range of functional forms, making them suitable for complex data, especially when a large amount of it is available to avoid overfitting.

The core purpose of a generative model is to learn the joint probability distribution or the data distribution . By learning this distribution, it can generate new data samples that resemble the training data. Discriminative models, in contrast, learn the decision boundary between classes () and are used for classification, not generation.

Since there are no pre-defined labels for the customer segments, the goal is to discover inherent structures or groups within the data. This task is known as clustering, which is a primary example of unsupervised learning.

This scenario perfectly describes reinforcement learning, which involves an agent learning to make a sequence of decisions in an environment to maximize a cumulative reward. The learning happens through trial-and-error interactions rather than labeled examples.

This option correctly hierarchies the fields. AI is the broad concept of creating intelligent machines. ML is a primary approach to achieving AI. Data Science is a wider, interdisciplinary field that uses scientific methods, processes, and systems (including ML and AI) to extract knowledge and insights from data.

Feature engineering is the art and science of creating informative features from raw data. A well-designed feature can make a simple model outperform a complex one. It's crucial because most algorithms learn from the features they are given, and their quality directly determines the performance ceiling of the model.

The test set must remain completely unseen by the model during the entire training process, including preprocessing. If you scale the data before splitting, the scaling parameters (like mean/std or min/max) are calculated using the entire dataset. This means the training process is influenced by the test data, a phenomenon called data leakage. The correct procedure is to fit the scaler on the training data only and then use it to transform both the training and test data.

Recall is calculated as , where TP is True Positives and FN is False Negatives. Maximizing recall means minimizing the number of False Negatives. In this use case, it's crucial to identify as many of the actual fraudulent transactions as possible, even if it means incorrectly flagging some legitimate ones.

Root Mean Squared Error (RMSE) is the square root of the Mean Squared Error (MSE). So, . The key advantage of RMSE over MSE is that its units are the same as the target variable (e.g., dollars for house prices). It represents the typical magnitude or standard deviation of the prediction errors.

This is the classic use-case for recommender systems. Collaborative filtering is a popular technique within this field that makes automatic predictions (filtering) about the interests of a user by collecting preferences or taste information from many users (collaborating).

Broadcasting compares shapes element-wise from right to left, after pre-pending 1s to the smaller shape to match dimensions.

1. A's shape: (6, 1, 5)
2. B's shape: (7, 1), pre-pended to (1, 7, 1)
3. Compare dimensions right-to-left:
   • Last dim: 5 and 1 are compatible.
   • Middle dim: 1 and 7 are compatible.
   • First dim: 6 and 1 are compatible.
4. The result shape is the maximum size along each dimension: (max(6,1), max(1,7), max(5,1)), which is (6, 7, 5).

X has shape (100, 10), so its transpose X.T has shape (10, 100). The matrix multiplication X.T @ X involves shapes (10, 100) and (100, 10), resulting in a (10, 10) matrix. This operation is fundamental in many ML algorithms (like calculating the covariance matrix or solving for weights in linear regression), where it captures the relationships between features.

In NumPy, the * operator is overloaded for element-wise operations. For two arrays of the same shape, it will multiply corresponding elements. The @ operator was introduced in Python 3.5 (and supported by NumPy) specifically for matrix multiplication, equivalent to np.matmul().

The stop parameter in np.arange is exclusive. So, np.arange(10, 50, 5) produces [10, 15, ..., 45]. np.arange(10, 46, 5) also produces the same result. np.linspace(10, 45, 8) creates 8 evenly spaced points between 10 and 45 inclusive, which is the desired array. However, np.arange(10, 51, 5) will include the number 50 in its output ([10, 15, ..., 45, 50]), which is not the desired array.

This is an example of NumPy slicing. 1: selects rows from index 1 to the end (the second and third rows). :2 selects columns from the beginning up to (but not including) index 2 (the first and second columns). The intersection of these two selections is the sub-array containing elements [[4, 5], [7, 8]].

To perform an operation along columns, we use axis=0. This collapses the rows and applies the function (np.max) to each column independently. Since there are 20 columns, the resulting array will have 20 values, one for each column's maximum, resulting in a shape of (20,).

This is a fundamental distinction in NumPy's random module. rand provides uniformly distributed random numbers in the half-open interval [0.0, 1.0). randn (the 'n' stands for 'normal') provides samples from the standard normal or Gaussian distribution.

Computers generate pseudo-random numbers starting from an initial value called a seed. By setting the seed to a specific integer using np.random.seed(), you ensure that the sequence of 'random' numbers generated will be identical every time the code is run. This is crucial for debugging and making experiments reproducible.

A model with high variance and low bias is overfitting the training data. This means the model is already too complex. Adding more complex polynomial features would further increase the model's complexity, likely exacerbating the overfitting problem. The other options are standard techniques to combat high variance: adding more data helps the model generalize better, regularization penalizes large weights to reduce complexity, and using a simpler model directly addresses the complexity issue.

In a 'small n, large p' scenario (few samples, many features), discriminative models like Logistic Regression or SVMs often struggle because they try to estimate a complex decision boundary directly from the limited data, making them prone to overfitting. Generative models like Naive Bayes model the joint probability by making strong independence assumptions. This simplifies the problem to estimating class priors and feature likelihoods, which requires far fewer parameters and is more robust with limited data.

Linear Regression is a parametric model. Its complexity is defined by a fixed number of parameters (coefficients and an intercept), which is determined by the number of features, not the number of training examples (). In contrast, k-NN is a non-parametric model. It doesn't learn explicit parameters in the same way. Its 'model' consists of the entire training dataset, which it must store to make predictions. Therefore, its complexity (in terms of storage and often computation) grows as increases.

Broadcasting works by comparing dimensions from right to left. The shapes are A: (4, 1, 3) and B: (5, 1, 1). Let's compare dimensions:

1. Rightmost dimension: A has 3, B has 1. This is compatible (1 will be stretched to 3).
2. Middle dimension: A has 1, B has 1. This is compatible.
3. Leftmost dimension: A has 4, B has 5. These are not equal, and neither is 1. Therefore, they are not compatible. The broadcasting rules fail at this dimension, and NumPy will raise a ValueError.

The described learning curve is a classic sign of high variance (overfitting). The model has learned the training data very well but fails to generalize to the validation data. Using a more complex model would worsen overfitting. Gathering more data could help, but it is often expensive and time-consuming. Increasing regularization strength (e.g., L1 or L2) or decreasing the number of features are direct methods to reduce model complexity and combat overfitting. Increasing regularization is often the most straightforward and effective first step to try.

For imbalanced datasets, AUROC can be misleadingly optimistic. This is because the large number of true negatives (healthy patients) inflates the true negative rate, which is a component of the ROC curve (Specificity = TN / (TN + FP)). The Precision-Recall (PR) curve does not depend on true negatives and focuses on the performance of the positive class. AUPRC is therefore a better indicator of performance on a rare-class detection task. Since Model B has a significantly higher AUPRC, it is likely the more practical and useful model.

Let's analyze the shapes: X is (100, 10), W is (10, 5), so W.T is (5, 10). The matrix multiplication requires the inner dimensions to match. For X @ W.T, the shapes are (100, 10) and (10, 5). The inner dimensions (10 and 10) match, and the resulting shape is (100, 5). The bias vector b has shape (5,). Due to broadcasting, it can be added to the (100, 5) matrix, where it is effectively added to each of the 100 rows. The other options result in shape mismatches or incorrect computations.

The fundamental unit of analysis is the customer, not the transaction. If you split randomly at the transaction level, it's highly likely that some transactions from a single customer will be in your training set while others from the same customer are in your test set. The model could then learn customer-specific patterns (like a customer ID implicitly) rather than generalizable churn behaviors. This is a form of data leakage, and the model's performance on the test set will be unrealistically high because it has already 'seen' the customers in the test set during training.

While this system would likely use Machine Learning components (e.g., for NLP), the overall goal described—understanding, reasoning, and automated action—aligns with the broader definition of Artificial Intelligence. AI is the science of making machines that can think and act like humans, encompassing areas like knowledge representation, reasoning, and planning. ML is a subset of AI focused on learning patterns from data. Data Science is a broader field that uses scientific methods, including ML and AI, to extract insights from data, but the core task of 'reasoning and composing a query' is a classic AI problem.

To prevent data leakage, any information from the validation set (or test set) must not be used to train the model or any preprocessing step. In k-fold cross-validation, for each fold, the scaler must be 'fit' (i.e., calculate mean and standard deviation) only on the k-1 training partitions. This same fitted scaler is then used to 'transform' both the training partitions and the single validation partition. Fitting the scaler on the entire dataset would leak information about the distribution of the validation/test data into the training process, leading to overly optimistic performance estimates.

The function np.random.seed() resets the state of the pseudo-random number generator (PRNG). The first block sets the seed to 42 and generates arr1. The second block resets the seed back to 42 and then generates arr2. Because the PRNG was reset to the same starting state, the sequence of numbers generated will be identical, making arr1 and arr2 equal. The generation of arr3 happens immediately after arr2 without resetting the seed. Therefore, it will draw the next set of numbers from the sequence, making arr3 different from arr2 (and arr1).

A standard Logistic Regression model fits a linear equation to the log-odds, resulting in a linear decision boundary in the feature space. While you can add polynomial features to make it non-linear, the model itself is linear. Gaussian Naive Bayes models each class conditional as a Gaussian. The decision boundary is where . The log of this ratio results in a function that is quadratic in if the covariance matrices for each class are different. If the covariance matrices are assumed to be identical (a common variant is Linear Discriminant Analysis), the quadratic terms cancel out, and the boundary becomes linear. Therefore, GNB can inherently produce a quadratic boundary while standard LR cannot.

Increasing adds a larger penalty for the magnitude of the model coefficients (). This forces the coefficients towards zero, making the model simpler and less sensitive to the specific training data points. This reduction in sensitivity to the training data directly leads to a decrease in variance. However, by constraining the coefficients, the model may no longer be able to capture the true underlying relationship in the data, thus moving its average prediction away from the true value. This introduces a systematic error, which means the bias increases.

Both np.dot(pixels, weights) and pixels @ weights are designed for this operation. For a 3D array (M,N,K) and a 1D array (K,), np.dot and @ perform a sum product over the last axis of the first array and the only axis of the second. pixels is (1920, 1080, 3) and weights is (3,). The operation np.dot(pixels, weights) correctly computes the sum product over the last axis (the channel axis), resulting in an array of shape (1920, 1080). The option np.sum(pixels * weights, axis=2) is also correct and readable, but np.dot is often more optimized for this specific linear algebra operation. The loop is correct but highly inefficient in NumPy. Between the given choices, np.dot is the canonical way to perform this tensor contraction.

Accuracy can be misleading in imbalanced datasets. A model could achieve 99% accuracy by simply predicting the majority class every time. The F1-score is defined as . This is the harmonic mean of precision and recall. A key property of the harmonic mean is that it is dominated by the smaller value. Therefore, to achieve a high F1-score, a model must have both high precision and high recall. It cannot 'game' the metric by, for example, having perfect precision but near-zero recall, which would result in a very low F1-score. This property makes it a much more reliable metric for imbalanced classes.

A key difference between L1 and L2 regularization is their effect on model coefficients. L1 regularization adds a penalty proportional to the absolute value of the coefficients (). Due to the geometry of its constraint region (a diamond shape), it tends to shrink some coefficients to exactly zero, performing implicit feature selection. This results in a sparse model. L2 regularization adds a penalty proportional to the squared value of the coefficients (). Its circular constraint region shrinks all coefficients towards zero but rarely sets them to exactly zero. This results in a dense model with small coefficient values.

This is an example of advanced or 'fancy' indexing. Let's break it down:

• arr is a (4, 3) array.
• np.arange(2)[:, np.newaxis] creates a column vector [[0], [1]] with shape (2, 1). This will be used to select the rows.
• idx is an array [[0, 2], [1, 1]] with shape (2, 2). This will be used to select the columns.
  NumPy broadcasts these two index arrays against each other. The (2, 1) row index array is broadcast to (2, 2) to match the column index array. The result is a selection of elements at coordinates (row_indices[i,j], col_indices[i,j]).
• For the first row of the output (i=0), it uses row index 0 and column indices [0, 2]. This selects arr[0, 0] and arr[0, 2].
• For the second row of the output (i=1), it uses row index 1 and column indices [1, 1]. This selects arr[1, 1] and arr[1, 1].
  The shape of the output is determined by the shape of the broadcasted index arrays, which is (2, 2).

The key phrase is 'without any pre-existing labels'. The goal is to discover inherent structure (the client segments) in the data itself. This is the definition of Unsupervised Learning. More specifically, the task of grouping data points into segments is called clustering. Supervised learning would require pre-labeled data (e.g., a dataset where clients are already labeled as 'high-value', 'at-risk', etc.). Reinforcement learning involves an agent learning through rewards and penalties, which is not applicable here. While analysts will eventually interpret the clusters (giving them meaning), the machine learning task itself is purely unsupervised.

Concept drift means that the underlying relationship between input features and the target variable changes over time. In fraud detection, this happens because fraudsters constantly change their tactics to evade detection. A model trained on data from last year may not be effective today because the patterns of what constitutes 'fraud' have evolved. This necessitates a robust MLOps strategy involving continuous monitoring of model performance and frequent retraining on recent data to adapt to these changing patterns. A static model would quickly become obsolete.

The test set must be held out and used only once for the final, unbiased evaluation of the chosen model. Hyperparameter tuning is part of the model selection process. If you perform GridSearchCV on the entire dataset, the cross-validation splits within the grid search will use parts of your eventual test set for training during the tuning process. This means the chosen hyperparameters are optimized to perform well on data that includes your test set. When you later evaluate the 'final' model on that same test set, its performance will be artificially inflated because the model's configuration has already been influenced by that data. The correct workflow is: 1) Split data into train and test sets. 2) Perform hyperparameter tuning (e.g., GridSearchCV) only on the train set. 3) Evaluate the final, best model on the held-out test set.