Unit 2: Pandas for Data Handling - Practice Quiz

A Pandas Series is a fundamental data structure representing a single column of data. It's like a one-dimensional array but with an associated index.

A DataFrame is the primary Pandas data structure. It's a 2D table of data with rows and columns, similar to a spreadsheet or a SQL table.

The pd.DataFrame() constructor is the standard way to create a DataFrame. When passed a dictionary, it uses the keys as column names and the values as column data.

pd.read_csv() is the primary function in Pandas for reading comma-separated values (CSV) files and creating a DataFrame from them.

The .to_csv() method is called on a DataFrame object to write its contents into a CSV file. By default, it includes the index; you can omit it with df.to_csv('data.csv', index=False).

The .head() method is used to get the first n rows of a DataFrame. If n is not specified, it defaults to 5.

The .shape attribute returns a tuple containing the number of rows and the number of columns in the DataFrame, in that order.

The .info() method is very useful for getting a quick overview of a DataFrame's structure, data types, and null values.

The standard and most common way to select a single column in Pandas is by using square bracket notation with the column name as a string, like df['ColumnName'].

This technique is called boolean indexing. The inner expression df['Age'] > 25 creates a boolean Series (True/False), which is then used to select the rows from df where the condition is True.

.iloc[] stands for 'integer-location' based indexing. It is used to access rows and columns by their integer position (from 0 to length-1).

Matplotlib is the foundational plotting library in the Python scientific ecosystem. It is conventionally imported with the alias plt using import matplotlib.pyplot as plt.

Line plots are ideal for showing how a variable changes over time, such as stock prices, temperature readings, or website traffic.

A bar plot is excellent for comparing quantities across discrete categories, such as countries, products, or groups.

A histogram groups numbers into ranges (bins) and the height of the bar shows how many data points fall into that range, revealing the underlying distribution of the data.

A scatter plot places a dot for each data point at a position corresponding to its values for two different numerical variables, revealing correlations or patterns between them.

Pie charts are designed to show how a single entity (the 'pie') is divided into parts. Each 'slice' represents a percentage of the total.

The plt.xlabel('Your Label Text') function sets the label for the horizontal (x) axis of the current plot.

The plt.title() function is the standard way to set the main title for a plot in Matplotlib.

The plt.subplots() function is a convenient way to create a figure and one or more subplots (axes) at once. For example, fig, ax = plt.subplots(2, 2) creates a 2x2 grid of subplots.

Pandas automatically aligns data based on index labels before performing arithmetic operations. For index 'a', s2 has no value, so the result is NaN. For index 'b', the values are 20 and 100, so the sum is 120. For index 'c', s1 has no value, so the result is NaN. The final Series will be a NaN, b 120.0, c NaN.

When applying multiple boolean conditions for filtering in Pandas, you must use the bitwise operators & (for AND) and | (for OR). Additionally, each condition must be enclosed in parentheses due to Python's operator precedence rules. The and keyword will not work as it attempts to find the truth value of the entire Series, not element-wise.

df.info() is used to get a technical summary of the DataFrame, focusing on index dtype, column dtypes, non-null counts, and memory usage. df.describe() is used to get statistical insights, providing counts, mean, standard deviation, min, max, and quartiles for the numerical columns by default.

The sep (or delimiter) parameter is used to specify the column separator, which is ';' in this case. The header=None parameter tells Pandas that there is no header row in the file, so it should treat the first line as data and assign default integer column names (0, 1, 2, ...).

When plt.subplots() is called with more than one row and one column, it returns a NumPy array of Axes objects. The array is indexed like a standard 2D array, using [row, column] with 0-based indexing. The top-right plot is at row 0, column 1, so ax[0, 1] is the correct way to access it.

A histogram is used to visualize the underlying distribution of a single continuous variable. It works by dividing the entire range of values into a series of intervals—called bins—and then counting how many values fall into each interval. The bins parameter directly controls how many of these intervals are created.

When you use .loc or .iloc to select rows based on a condition and specify a single column name, Pandas returns the result as a Series. The index of the Series will be the same as the original DataFrame's index for the selected rows. To get a DataFrame instead, you would need to pass a list containing a single column name, like df.loc[df['Score'] > 90, ['Name']].

The column 'C' is created based on the values of 'A' at that moment ([2, 4, 6]). The line df.iloc[0, 0] = 100 modifies the value in column 'A' at row 0 after column 'C' has already been calculated. This modification is not retroactive and does not trigger a recalculation of 'C'. Therefore, the value of df.loc[0, 'C'] remains its original calculated value, which is 2.

The object-oriented approach in Matplotlib first creates a figure and an axes object (e.g., fig, ax = plt.subplots()). Customizations are then applied by calling methods on the ax object. The correct methods are set_title(), set_xlabel(), and set_ylabel(). The second option uses the state-based pyplot interface, not the object-oriented one. The other options use incorrect method names or chaining.

In a scatter plot, the c parameter can be used to add a third dimension of data represented by color. By passing a column name (or a Series/array), you are telling the plot to vary the color of each point based on the value from that column for the corresponding row. The cmap parameter specifies which colormap to use for this mapping.

The correct method to save a DataFrame to a CSV file is to_csv(). To prevent the DataFrame's index from being written into the file as a column, you must set the index parameter to False.

This is a core concept in Pandas. df.loc is used for selection by labels (index names, column names). df.iloc is used for selection by integer position (0-based indexing, like a standard Python list or NumPy array). A key consequence mentioned in another option is that slices with loc are inclusive of the endpoint, while slices with iloc are exclusive, but the primary distinction is label-based vs. position-based.

Pie charts are best for showing parts of a whole with a small number of slices (typically less than 6-7). They become very difficult to read and interpret when there are many categories. The human eye is not good at comparing angles, so it's hard to judge the relative sizes of slices, especially when their values are close. A bar chart is almost always a better alternative in this scenario.

Plotting df directly would create a bar for every single row, which is incorrect. The goal is to show an aggregated value (total revenue) per category ('Region'). The standard 'group-by-aggregate' pattern is required first: df.groupby('Region')['Revenue'].sum(). The resulting Series can then be directly plotted using .plot(kind='bar').

value_counts() is a Series method specifically designed for this purpose. It returns a new Series where the index is the unique values from the original Series and the values are their counts, sorted in descending order by default. While groupby().count() can achieve a similar result, value_counts() is more direct and idiomatic.

Line plots excel at showing trends or changes in a variable over a continuous and ordered interval, most commonly time. Tracking population over decades is a perfect use case. Comparing discrete categories (sales figures) is better for a bar chart, distribution (ages) for a histogram, and proportions (transportation) for a pie or bar chart.

To plot multiple lines on the same axes, you can simply call plt.plot() multiple times. Matplotlib will automatically assign different colors to each line. To add a legend, you call plt.legend() after all the lines have been plotted, providing a list of labels in the same order as the plot calls. A more robust way is to use the label argument in each plot call: plt.plot(df['x'], df['y1'], label='Series 1') and then simply call plt.legend().

The sharex and sharey parameters are used to link the axes of subplots. When sharex=True is used for a grid of plots, the x-axis limits and scale will be synchronized across the shared plots. A practical benefit is that Matplotlib will automatically turn off the x-tick labels for all but the bottom-most plots, creating a cleaner and more readable figure.

This describes the fundamental object-oriented structure of Matplotlib. The Figure is the outermost container for all plot elements. You can think of it as the page or window. Inside the Figure, you place one or more Axes objects. An Axes is the actual plotting area where data is displayed with x- and y-axes, titles, ticks, and labels.

A standard bar plot displays a single value for each category. A stacked bar plot, on the other hand, represents multiple data series for each category. The bars are segmented, with each segment representing a sub-category, and the total height of the bar represents the sum of its parts. This makes it useful for comparing both the total and the composition of each category.

This is a complex slicing operation on a MultiIndex. pd.IndexSlice is the recommended, idiomatic, and often most performant tool for this. Option A, df.loc[(['bar', 'foo'], 'two'), :], attempts to find combinations ('bar', 'two') and ('foo', 'two'), which is correct, but using pd.IndexSlice is the more robust and explicit way to construct complex slicers. Option C (.query()) is powerful but can be less performant for pure index-based selections. Option D is functionally correct but far more verbose and less efficient than using the optimized index slicing with .loc and pd.IndexSlice.

This question addresses the critical and often confusing concept of views vs. copies in Pandas. Chained indexing (df[...]...) can result in either a view or a copy, depending on NumPy's memory layout of the underlying array. Because it's not guaranteed, Pandas raises a SettingWithCopyWarning to alert the user of this unpredictable behavior. The correct way to guarantee a modification of the original DataFrame is to use a single .loc operation: df.loc[df['A'] > 0, 'B'] = 999.

The key constraint is that the file is larger than RAM. The only viable option is to process the file in pieces. The chunksize parameter in pd.read_csv is designed for this exact scenario. It returns an iterator that yields DataFrames of the specified size. By iterating through these chunks, one can perform calculations (like extracting the year and summing values/counts) and aggregate the results incrementally without ever loading the entire file into memory. The other options would all attempt to load a significant portion (if not all) of the data into memory at once, leading to a MemoryError.

This question tests the understanding of how Matplotlib's GridSpec and subplot creation works, and how to manipulate the returned axes objects. ax1 = fig.add_subplot(gs[:, 0]) creates the large subplot on the left, so ax1 is the correct object to modify. ax2 = fig.add_subplot(gs[0, 1]) creates the subplot in the first row, second column (top-right). Therefore, ax2 is the correct object for the title. Option C is incorrect because GridSpec objects don't have plotting methods like set_title. Option D relies on the order of axes in fig.axes, which is brittle and not guaranteed, making it poor practice compared to using the explicit ax variables.

Log transformation is a powerful tool for handling right-skewed data. By compressing the range of large values and expanding the range of small values, it can make the transformed data's distribution more symmetric and often closer to a normal distribution. This makes it much easier to inspect the data's structure, as features that were clumped together in the original histogram become more spread out and visible. Option B is incorrect; transforming the data is different from changing the axis scale. Option C is a potential side effect but not the primary purpose. Option D describes the effect of using the density=True parameter, not a log transformation.

This is a complex inspection task that perfectly matches the functionality of pivot_table. While groupby().unstack() (Option A) can reshape the data, it doesn't have a built-in margins parameter. pd.crosstab (Option B) is very similar, but it's primarily for computing frequency tables of factors. pivot_table is more general and is the canonical way to re-shape data with aggregation, filling missing values (fill_value), and adding marginal totals (margins=True) all in one powerful command.

ax.axhline is the correct method to draw a horizontal line spanning the entire width of the plot on a given axes object. The key part is the annotation. Option C, ax.text(x=ax.get_xlim()[0], y=mean_val * 1.05, ...) correctly places the text using data coordinates. The x-coordinate is set to the beginning of the plot's x-axis, and the y-coordinate is slightly above the mean line. Option A uses transform=ax.transAxes, which changes the coordinate system for x to be relative to the axes width (0.5 is the middle), which is a valid but different approach, but the question implies data coordinates. Option B's ax.hlines requires xmin and xmax arguments. Option D uses the pyplot interface which is less ideal when working with a specific ax object.

This question probes the deeper differences between two filtering methods. query() uses the numexpr library in the background, which can optimize computations and memory usage for very large arrays, potentially making it faster than standard boolean indexing which can create large intermediate boolean arrays. A key feature of query() is its ability to use variables from the environment (e.g., threshold = 5; df.query('A > @threshold')), which makes for cleaner code than string formatting. However, .loc is more versatile and is the required tool for complex label-based indexing, especially with MultiIndexes.

Pandas/Matplotlib plotting functions will draw a continuous line between consecutive non-NaN points in a Series. To explicitly show gaps, you must make them exist in the data as NaN values. The standard way to do this is to create a full date range for the period of interest and .reindex() the original DataFrame with it. This will insert rows for the missing dates, and the 'value' for these rows will be NaN. When plotted, Pandas will create a break in the line at these NaN points. Option A changes the plot type entirely. Option C mentions a non-existent parameter connect_missing.

This is a complex multivariate visualization. The c (color) parameter in Matplotlib/Pandas plotting expects numeric values to map to a colormap. Directly passing a string column like 'fuel_type' (as in Option A and C) will raise an error. You must first convert the categorical 'fuel_type' column into integer codes. pd.factorize(df['fuel_type'])[0] is a direct way to do this. astype('category').cat.codes (used in Option B's plt.scatter context) is another valid way. However, Option D correctly uses the pandas .plot API, which is often more convenient. The size parameter s can directly take a Series like df['horsepower']. Option C is almost correct, but passing the string 'fuel_type' to c is the mistake.

To create a grouped bar plot, you need data where the index corresponds to the groups on the x-axis (Departments), and the columns correspond to the different colored bars within each group (Gender). df.pivot_table(..., index='Department', columns='Gender', ...) produces exactly this structure. Plotting this result with kind='bar' will automatically generate the desired grouped bar plot. Option A would create a bar plot with a MultiIndex on the x-axis, which is not a grouped plot. Option B would create a stacked bar plot, not a grouped one. Option D (crosstab) computes counts/frequencies by default, not the mean salary.

The goal is to create a new Series where large shares are preserved and small shares are summed up into a new 'Others' category. Option C does this correctly: it first filters to keep the large shares (s[s >= 0.01]), then it calculates the sum of the small shares (s[s < 0.01].sum()) and assigns this sum to a new index label 'Others'. This creates the perfect data structure for plotting. Option A (.where) would replace the values of small shares with the string 'Others', which is incorrect for plotting. Option B modifies the series in place in a way that doesn't correctly create an 'Others' category. Option D is conceptually convoluted and incorrect syntax.

This is a key distinction for advanced data manipulation. groupby().apply() is very flexible; the function passed to it can return almost anything, and pandas will try to stitch the results together. This is useful for complex aggregations. groupby().transform(), however, has a strict requirement: the function must return a result (a Series or array) that is the same size as the group it's operating on. The final result is a Series with the same index as the original DataFrame. This makes .transform() uniquely suited for tasks like creating a new column based on a group-wise calculation, such as de-meaning data within groups (e.g., df['value_demeaned'] = df.groupby('category')['value'].transform(lambda x: x - x.mean())).

The to_csv method has a specific parameter, float_format, for controlling the string representation of floating-point numbers during the writing process. It accepts a C-style format string. '%.4f' is the correct format string for a float with 4 decimal places. While Option C (df.round(4).to_csv(...)) achieves the same result in the output file, it does so by creating a new, modified DataFrame in memory first, which can be less efficient. The question asks for a solution directly within the to_csv command, making float_format the most precise answer. Option A's precision parameter does not exist for to_csv.

This task requires dynamic formatting of tick labels based on their numeric value. The most powerful and correct way to do this in Matplotlib is with the ticker module. A FuncFormatter takes a user-defined function that accepts a tick value and its position and returns a formatted string. This allows for complex logic, like def to_quarter(x, pos): year = int(x); q = int((x - year) * 4) + 1; return f'{year}-Q{q}'. This formatter is then applied using ax.xaxis.set_major_formatter(formatter). Option A is brittle because it hardcodes labels and will not work if the plot limits or data change. Option C uses a format string syntax (%q) that does not exist in Matplotlib's standard formatters.

This is a common data cleaning task. The errors parameter in pd.to_numeric controls its behavior when it encounters a value it cannot parse. The default is errors='raise', which stops execution. errors='ignore' will leave the non-numeric values as they are, resulting in a mixed-type object column. errors='coerce' is the correct choice here; it will replace any value that cannot be converted to a number with pd.NA or np.nan, allowing the rest of the column to be converted to a numeric dtype. This is essential for preparing data for numerical analysis.

Matplotlib's sharex and sharey parameters are designed to create cleaner, less redundant plots. When axes are shared, Matplotlib automatically hides the interior tick labels to avoid clutter. For a (1, 2) layout (one row, two columns), the Y-axis of the right plot (ax2) is interior, so its labels are hidden. The X-axis of both plots are on the bottom row, but to avoid redundancy when sharex is used across columns, Matplotlib's default behavior can be to hide labels on all but the last plot in a row depending on the layout. In this specific (1,2) case, the more noticeable effect is hiding the y-tick labels on the interior plot (ax2). A more general and precise statement is that interior decorations are turned off. For a (2,1) plot, the x-tick labels of the top plot would be hidden. The provided option C describes the behavior for a (2,2) grid for interior plots, but is the closest description of the automatic label hiding feature among the choices for a general case. Let's re-evaluate for (1,2): ax1 is on the left, ax2 is on the right. They share a y-axis. ax2's y-tick labels will be hidden. They share an x-axis. Since they are side-by-side, both are on the 'bottom' of the figure, so both sets of x-tick labels will likely be shown. The options are poorly phrased. Let's correct the question and answer logic. For plt.subplots(2, 1, sharex=True) the x-axis labels of the top plot (ax1) are hidden. For plt.subplots(1, 2, sharey=True), the y-axis labels of the right plot (ax2) are hidden. The question combines sharex and sharey. For a (1,2) plot, sharex doesn't hide anything by default, but sharey=True will hide the y-tick labels for ax2. Let's find the best option. Option C states X-axis from ax1 and Y-axis from ax2 are removed. This is the behavior for an interior plot in a 2x2 grid. For a 1x2 grid, only the y-axis on ax2 is removed. However, this is the kind of behavior that happens. Let's re-examine. For (1,2), sharey=True, y-labels on ax2 are gone. For (1,2), sharex=True, x-labels are often kept on both. The options might be flawed. Let's assume the most common general case. The principle is: remove interior labels. For ax2 in (1,2), the y-axis is interior. For ax1 in (2,1), the x-axis is interior. Option C: X-axis tick labels will be removed from ax1. This would happen if there was a plot below it. Y-axis tick labels will be removed from ax2. This happens because there is a plot to its left. This option describes the general principle of hiding interior labels correctly, even if the ax1 part isn't strictly true for a 1x2 layout. It's the best description of the purpose and effect of sharing axes among the choices.

This question tests a critical edge case in index-based selection. .iloc is strictly integer-position based and is unaffected by the index labels or their uniqueness; df.iloc[0] will always return the first row as a Series. .loc, however, is label-based. If a label is unique in the index, df.loc['unique_label'] returns a Series. But if the label is not unique, .loc will return a DataFrame containing all rows that share that duplicate label. It does not raise an error. This is a fundamental difference in behavior between the two accessors when dealing with non-unique indexes.

Pandas' core feature is index alignment during arithmetic operations. When adding two Series, pandas takes the union of their indexes. If an index label exists in one Series but not the other, the result for that label will be NaN by default (e.g., 5 + NaN = NaN). The .add() method provides a fill_value parameter. s1.add(s2, fill_value=0) tells pandas to substitute 0 for any missing value in s1 or s2 before performing the addition. This allows for meaningful calculations where a missing label implies a value of zero, instead of propagating NaNs.

This question addresses a core concept of using Matplotlib effectively. The pyplot (state-machine) interface maintains a state—it keeps track of the currently active figure and axes, and all plt.* commands apply to them. This is convenient for simple, quick plots. However, for complex figures with multiple subplots, it becomes confusing and error-prone to manage which axes is 'current'. The object-oriented (OO) interface, which starts with fig, ax = plt.subplots(), returns explicit Figure and Axes objects. All subsequent operations are methods of these objects (e.g., ax.set_title(), fig.suptitle()). This gives the programmer complete and unambiguous control over every element of the plot, which is essential for building complex, reusable, and maintainable visualizations.