Unit 3 - Practice Quiz

Ordinal data is a type of categorical data where the categories have a natural, ordered relationship. Nominal data has no intrinsic order.

The read_csv() function is the standard and most common way to ingest data from a comma-separated values (CSV) file into a Pandas DataFrame.

A histogram groups a single numerical variable into 'bins' and shows the frequency of data points falling into each bin, thereby illustrating the data's distribution.

The line inside the box of a box plot always represents the median, which is the middle value of the dataset, splitting it into two equal halves.

A count plot shows the number of occurrences (counts) for each category in a categorical variable, making it essentially a histogram for categorical data.

A scatter plot places dots on a graph corresponding to the values of two numerical variables, allowing analysts to observe patterns, trends, and correlations between them.

Line plots are excellent for showing trends and patterns in data where the points are ordered, such as time-series data (e.g., stock prices over a year).

A correlation coefficient of -1.0 signifies a perfect negative correlation, meaning as one variable increases, the other systematically decreases.

Multicollinearity is a phenomenon where predictor (independent) variables are intercorrelated, which can cause problems in interpreting regression models.

Negative skew (or left-skew) means the mass of the distribution is concentrated on the right, and the tail extends to the left, towards the lower values.

Kurtosis is a measure of whether the data are heavy-tailed or light-tailed relative to a normal distribution. High kurtosis indicates heavy tails and more outliers.

An outlier, or anomaly, is an observation that deviates significantly from other observations in the dataset. Identifying them is a key part of EDA.

EDA is an initial, investigative phase focused on summarizing, visualizing, and understanding a dataset before formal modeling begins.

Nominal data represents categories without any intrinsic order or ranking. One country name is not 'greater' or 'less' than another.

A random, cloud-like arrangement of points on a scatter plot indicates that there is no clear linear trend, suggesting a correlation close to zero.

The box in a box plot spans from the first quartile (Q1) to the third quartile (Q3). The length of the box is the Interquartile Range, calculated as .

A violin plot shows the summary statistics of a box plot while also showing the full distribution of the data, similar to a kernel density plot, on each side.

Heatmaps use a color scale to represent numerical values, making it very easy to scan a large correlation matrix and spot strong positive or negative relationships at a glance.

Skewness measures the asymmetry of a distribution. A value of 0 indicates perfect symmetry, like that of a normal distribution.

A logical EDA workflow starts by understanding the dataset as a whole, then moves to analyzing individual variables (univariate analysis), and then to relationships between variables (bivariate/multivariate analysis).

A high correlation coefficient (typically > 0.7 or 0.8) between two independent (predictor) variables is a strong indicator of multicollinearity. This issue doesn't reduce the predictive power of the model as a whole, but it makes the individual coefficients for the correlated variables unreliable and hard to interpret.

A violin plot is specifically designed for this purpose. It shows the full distribution of a numeric variable for different categories. The width of the plot represents the density of the data, and it often includes a box plot or marker for the median and interquartile range inside.

Negative skewness (left-skewed) means the left tail of the distribution is longer or fatter than the right tail. In this context, the mass of the distribution is concentrated on the right side (higher scores), with a tail pointing towards the lower scores.

If the median is closer to Q1, it means the data points between Q1 and Q2 are more compressed than the data points between Q2 and Q3. This results in the upper part of the distribution (and the upper whisker) being more spread out, which is characteristic of a positive or right-skewed distribution.

A correlation coefficient close to -1 indicates a strong inverse (negative) linear relationship. This means that as the value of study_hours tends to go up, the value of error_rate tends to go down. Correlation does not imply causation.

The chunksize parameter in pd.read_csv returns a TextFileReader object, which is an iterator. This allows you to loop through the file in chunks of a specified size, performing calculations on each chunk and aggregating the results, thus avoiding loading the entire large file into RAM.

Seasonality refers to predictable, repeating patterns or variations that occur at regular intervals over a fixed period, such as a year. The holiday season traffic spike is a classic example of seasonality in time-series data.

A bimodal distribution often indicates that the dataset is composed of two different underlying groups or populations. In this case, the two peaks could represent two distinct types of housing, each with its own central tendency for price.

The Pearson correlation coefficient is a measure of the linear relationship between two variables. A value near zero indicates the absence of a linear relationship, but it does not rule out a strong non-linear one. The inverted 'U' shape is a clear pattern that Pearson correlation fails to capture.

EDA is a critical, iterative step that occurs after data collection and initial cleaning. Its purpose is to gain a deep understanding of the data's structure, distributions, relationships, and anomalies. The insights from EDA are crucial for making informed decisions in subsequent steps like feature engineering, data preprocessing, and choosing an appropriate model.

A high kurtosis value (>3) indicates a leptokurtic distribution. In finance, this means the distribution has a sharper peak around the mean and heavier/fatter tails. These fat tails signify that extreme events (large gains or large losses) are more likely to occur than a normal distribution would predict, which translates to higher risk.

This is nominal data. The values are labels or names used to identify unique items. There is no intrinsic order, and arithmetic operations like averaging product_IDs would make no sense. It should be treated as a string or category, not a number.

A count plot (a specific type of bar chart) is designed to show the frequency (count) of observations in each category of a categorical variable. This is the most direct and effective way to visualize the distribution of ticket priorities.

Line plots are the standard and most effective visualization for time-series data. They connect data points in chronological order, making it easy to see trends, seasonality, and other time-dependent patterns in the stock price.

VIF quantifies how much the variance of an estimated regression coefficient is increased because of multicollinearity. A VIF of 1 means no correlation. A common rule of thumb is that VIF values greater than 5 or 10 indicate a problematic level of multicollinearity. A VIF of 25 is very high and suggests the variable should be investigated or removed.

The primary advantage of a violin plot over a box plot is its inclusion of a kernel density estimate. This shows the full shape of the distribution. A box plot would hide the fact that one distribution is bimodal, while a violin plot would clearly visualize the two separate peaks, providing a crucial insight.

Points that deviate significantly from the main trend or pattern are potential anomalies or outliers. In this context, they warrant further investigation to determine if they are errors or belong to a distinct sub-population that follows a different salary structure.

This question tests the ability to interpret a standard heatmap visualization. Typically, distinct colors are used for positive and negative correlations. Logically, higher crime rates are associated with lower property values (negative correlation), and better school ratings are associated with higher property values (positive correlation).

Heteroscedasticity is the term for the situation where the variability (variance) of one variable is unequal across the range of values of a second variable that predicts it. A funnel shape in a scatter plot is the classic visual indicator of this phenomenon, which is an important assumption to check for in linear regression.

This question tests the specific definition used for identifying potential outliers in a Tukey box plot. The 'whiskers' of the plot extend to the furthest data point within from the box (Q1 or Q3). Any data point beyond these whiskers is marked as an outlier.

Linear Regression coefficients are highly sensitive to multicollinearity, as the model struggles to attribute effect to individual correlated predictors, leading to large standard errors and unstable coefficients. Random Forests, being tree-based, are much more robust. If two features are highly correlated, the model might randomly pick one over the other at a given split, but since it builds many trees on different subsets of data and features, this instability is averaged out, and the overall predictive performance remains strong. While feature importance might be split between the correlated features, the model's predictive power is largely unaffected.

A leptokurtic distribution has 'fatter tails' than a normal distribution. This means that extreme outcomes (very large gains or very large losses) are more likely than a normal distribution would predict. A model assuming normality will miscalculate risk by underestimating the frequency and magnitude of these tail events, which is a critical failure in financial risk management.

A short, wide (platykurtic-like) violin plot (Dept A) indicates that a large number of data points are spread out and not clustered around the median; the variance is high among the bulk of employees. A tall, narrow (leptokurtic-like) violin plot (Dept B) indicates that most employees' salaries are tightly clustered around the median, implying consistency. The long tails, however, show that there are significant outliers (very high and/or very low earners). Therefore, salaries are more predictable for the average employee in Dept B, but the range of possibilities is more extreme.

An almost perfect correlation is a major red flag for data leakage, where information from the future (the outcome) has leaked into the feature set. A 'last call complaint' likely occurs at the time the customer is churning, not before. Using this feature would create a model that is unrealistically accurate but useless in practice because the information isn't available at the time of prediction. The critical next step is to confirm this temporal issue, which is a core part of a robust EDA process, before any modeling decisions are made.

This is the most memory-efficient method. The usecols parameter ensures that only the three necessary columns are read from the disk into memory for each chunk. The chunksize parameter reads the file in smaller, manageable pieces, preventing the entire 10 GB file from being loaded at once. By iterating through these chunks and performing partial aggregations, you can compute the final result without ever exceeding the available RAM. The other options are inefficient or infeasible given the memory constraints.

Different transformations apply different levels of 'strength' to correct skewness. The log transformation is quite strong. If it overcorrects a positive skew into a negative one, it implies a less potent transformation is needed. The square root transform is milder than the log transform. A Box-Cox transformation is even better as it algorithmically finds the optimal lambda parameter to achieve the best normalization, making it adaptive to the specific degree of skew in the data.

The first quartile (Q1) is the value below which 25% of the data falls. The median (Q2) is the value below which 50% of the data falls. If Q1 and the median are the same value (let's say V), it means that at least 50% of the data is less than or equal to V, and at least 25% of the data is less than or equal to V. For these to be the same, it requires that all data points from the 25th percentile to the 50th percentile are equal to V. This indicates a significant number of duplicate values in the dataset at that specific point.

While using regularization or PCA are valid modeling strategies, the question asks for a feature selection decision based only on the heatmap. With a correlation of +0.9, feature_X and feature_Y are highly redundant. Including both in a linear model would introduce significant multicollinearity, making the model coefficients unstable and hard to interpret. The most direct and effective feature selection strategy in this scenario is to retain the feature that has the stronger relationship with the target variable (feature_X at +0.7) and discard the other (feature_Y). This simplifies the model while retaining most of the predictive information.

A funnel shape in a residual plot indicates that the variance of the errors is not constant across all levels of the predicted values. This condition is known as heteroscedasticity. While the coefficient estimates in Ordinary Least Squares (OLS) remain unbiased, their standard errors will be biased. This makes hypothesis testing, p-values, and confidence intervals unreliable, potentially leading to incorrect conclusions about the significance of predictor variables.

In anomaly detection, an outlier is typically a single point that deviates from the norm. However, a cluster of points that is anomalous with respect to the rest of the data but internally cohesive represents a novel pattern of behavior. In fraud detection, this is often called a 'wolf pack' and can signify a new, coordinated fraud tactic. Dismissing them as random outliers or errors would be a mistake. The correct action is to flag them for deep investigation as a potential new class of behavior that the model must learn to detect.

This scale is classic ordinal data: there is a clear order, but the intervals between the ranks are not guaranteed to be equal. The psychological 'jump' from 'Very Unsatisfied' to 'Unsatisfied' might be very different from the jump from 'Neutral' to 'Satisfied'. By calculating the mean, the data scientist is implicitly assuming these intervals are equal, treating the data as if it were on an interval scale. This can lead to misleading summary statistics. While common practice, it's a theoretical violation, and robust analysis should also consider medians and modes which are appropriate for ordinal data.

The classic symptom of multicollinearity is a high overall model fit (high R-squared) combined with insignificant individual predictors (high p-values). This happens because the correlated predictors explain the same variance in the target, and the model cannot disentangle their individual effects, inflating their standard errors. A correlation matrix can hint at this, but the definitive diagnostic tool is the Variance Inflation Factor (VIF). A VIF value greater than 5 or 10 for a predictor is a strong indication of problematic multicollinearity.

A bimodal distribution is a strong indicator that the underlying data is a mixture of two different distributions. In a business context, this almost always points to the existence of distinct customer segments. Ignoring this and treating all customers as a single group can lead to poor model performance, as the relationships between variables might differ significantly between the two groups. The most valuable EDA insight is to recognize this segmentation and plan subsequent analysis and modeling around it.

This is a classic 'correlation vs. causation' problem. While the causal link is intuitive and likely true in this case, the correlation coefficient itself does not prove it. It only quantifies the degree to which two variables move together. To establish causation, one would need to conduct a controlled experiment or use more advanced causal inference methods. A confounding variable, like the season, is driving both: in winter (a single seasonal state), temperatures are low and oil consumption is high. The heatmap is a tool for discovering associations, which are candidates for causal investigation, not proof of it.

Kurtosis is a direct measure of the 'tailedness' of a distribution. A kurtosis of 8 is significantly higher than that of a normal distribution (3), indicating a leptokurtic distribution. This means the distribution has much more mass in its tails, which translates to a higher frequency of extreme values (outliers). While the skewness of Feature B (3) indicates asymmetry, the high kurtosis of Feature A is a more direct and severe indicator of the presence of extreme outliers, which is the primary challenge for many anomaly detection algorithms.

Specifying the data type at read-time with the dtype parameter is the most efficient and robust solution. It tells Pandas to not even attempt type inference for that column and to simply read every value as a string from the start. This prevents the DtypeWarning, avoids memory inefficiencies associated with mixed-type columns, and is more direct than post-processing. Using .astype(str) after loading works but is less efficient because Pandas has already gone through the trouble of parsing and storing a mixed-type object column first. low_memory=False might resolve the warning but doesn't guarantee the desired final type.

This question focuses on the impact of imbalance on EDA itself, not just on modeling. When you create visualizations like a box plot of feature_X grouped by the target class, the box plot for the minority class will be constructed from very few data points, making it highly unstable and potentially non-representative. It might look like noise or an outlier. This can lead to incorrect conclusions about how features differ between classes. A good EDA process for imbalanced data involves being aware of this and potentially using stratified sampling for visualizations or using visualizations that handle density better. While the other options are relevant to the overall project, this is the most direct risk within the EDA phase.

EDA in an MLOps context is an ongoing process of validation and monitoring. It is most critical to perform a deep-dive EDA when there's a specific trigger suggesting the underlying data distribution has changed. A schema change (e.g., a column is added, a data type changes) is a hard trigger. A drop in model performance (concept drift) is a soft trigger that requires EDA to diagnose the 'why'. Re-running EDA on every cycle is inefficient if the data is stable. Tying it to a fixed performance drop threshold is arbitrary. EDA should be a diagnostic tool used proactively when changes are expected or reactively when they are observed.

A sudden, permanent shift in the statistical properties (like the mean or variance) of a time series is known as a structural break. Unlike an outlier, which is a temporary blip, a structural break signifies a fundamental change in the underlying data generating process. It's crucial to identify this during EDA because it means that a single time-series model (like ARIMA) trained on data from before the break will not be valid for forecasting after the break. While a data collection issue is a possibility, the term 'structural break' is the correct statistical name for this observed pattern and implies an external cause must be investigated.

Detecting patterns in EDA involves synthesizing information from multiple plots. Seeing these two opposing trends together is highly insightful. A simple interpretation of decreasing users is negative. But when combined with increasing engagement from the remaining users, it paints a more complex picture. The most likely scenario is a change in the user population mix: less engaged, casual users are churning, while the more dedicated users are staying and becoming even more engaged. This is a critical insight that could heavily influence marketing and product strategy (e.g., focus on retaining power users vs. acquiring new casual users).