Pandas is an open-source Python library built on top of NumPy that provides high-performance, easy-to-use data structures and data analysis tools.

Main features:

• Fast and efficient DataFrame object with default and customized indexing.
• Tools for loading data into in-memory data objects from different file formats (CSV, Excel, JSON, SQL databases).
• Data alignment and integrated handling of missing data (NaN).
• Reshaping and pivoting of datasets.
• Label-based slicing, indexing, and subsetting of large datasets.
• Group by functionality for split-apply-combine operations.
• Merging and joining of datasets.
• Time-series functionality.

Why preferred:

• It simplifies complex data manipulation with minimal code.
• Handles heterogeneous data types easily.
• Integrates well with other libraries like Matplotlib and NumPy.
• Provides intuitive tabular data representation similar to spreadsheets.

A Pandas Series is a one-dimensional labeled array capable of holding data of any type (integer, string, float, Python objects, etc.). The axis labels are collectively called the index.

Key points:

• It is like a single column of data.
• Each element has an associated index label.
• Default index starts from 0.

Creating a Series:

import pandas as pd

From a list

data = [10, 20, 30, 40]
s = pd.Series(data)
print(s)

With custom index

s2 = pd.Series([10, 20, 30], index=['a', 'b', 'c'])
print(s2)

From a dictionary

d = {'x': 100, 'y': 200}
s3 = pd.Series(d)
print(s3)

Output of s2:

a 10
b 20
c 30
dtype: int64

Here the labels a, b, c act as the index, allowing label-based access like s2['b'].

A DataFrame is a two-dimensional, size-mutable, heterogeneous tabular data structure with labeled axes (rows and columns). It is the most commonly used Pandas object.

Structure:

• Rows are identified by a row index.
• Columns are identified by column labels.
• Can be thought of as a dictionary of Series objects sharing the same index.

Ways to create a DataFrame:

• From a dictionary of lists:
  python
  import pandas as pd
  data = {'Name': ['Amit', 'Neha'], 'Age': [25, 30]}
  df = pd.DataFrame(data)

• From a list of dictionaries:
  python
  data = [{'a': 1, 'b': 2}, {'a': 5, 'b': 10}]
  df = pd.DataFrame(data)

• From a list of lists:
  python
  data = [[1, 'A'], [2, 'B']]
  df = pd.DataFrame(data, columns=['ID', 'Grade'])

• From a NumPy array.

• From a CSV file using pd.read_csv().

Example output:

Name Age
0 Amit 25
1 Neha 30

Explanation:

• A Series stores a single sequence of values with labels, like one column of data.
• A DataFrame is essentially a collection of Series that share the same index, forming a table.
• Accessing a single column from a DataFrame (e.g., df['Age']) returns a Series.

Example:
python
df = pd.DataFrame({'A': [1, 2], 'B': [3, 4]})
col = df['A'] # This is a Series

CSV (Comma-Separated Values) files are one of the most common formats for storing tabular data. Pandas provides simple functions to handle them.

Reading a CSV file — read_csv():
python
import pandas as pd
df = pd.read_csv('data.csv')
print(df.head())

Useful parameters of read_csv():

• sep – specify a delimiter (default ,).
• header – row number to use as column names.
• index_col – column to set as index.
• names – provide custom column names.
• nrows – number of rows to read.

Example:
python
df = pd.read_csv('data.csv', index_col=0, nrows=100)

Writing to a CSV file — to_csv():
python
df.to_csv('output.csv', index=False)

Useful parameters of to_csv():

• index=False – do not write the row index.
• columns – specify columns to write.
• sep – specify delimiter.
• header – whether to write column names.

This makes it easy to import, process, and export data seamlessly.

Pandas supports numerous operations on DataFrames for data manipulation and analysis.

1. Selecting columns:
python
df['Name'] # single column
df[['Name', 'Age']] # multiple columns

2. Selecting rows (indexing):
python
df.loc[0] # by label
df.iloc[2] # by position

3. Filtering rows (conditional selection):
python
df[df['Age'] > 25]

4. Adding a new column:
python
df['Bonus'] = df['Salary'] * 0.1

5. Deleting a column:
python
df.drop('Bonus', axis=1, inplace=True)

6. Aggregation and statistics:
python
df['Age'].mean()
df.describe()

7. Sorting:
python
df.sort_values(by='Age', ascending=False)

8. Grouping:
python
df.groupby('Department')['Salary'].mean()

These operations enable powerful data wrangling with concise syntax.

These are commonly used functions for inspecting and understanding a DataFrame.

1. head(n)

• Returns the first n rows (default 5).
• Used to quickly preview the data.
  python
  df.head(3)

2. tail(n)

• Returns the last n rows (default 5).
• Useful to check the end of a dataset.
  python
  df.tail()

3. info()

• Displays a concise summary of the DataFrame including:
  • Number of rows and columns
  • Column names and data types
  • Non-null counts
  • Memory usage
    python
    df.info()

4. describe()

• Generates descriptive statistics for numeric columns:
  • count, mean, std, min, max
  • 25%, 50%, 75% percentiles
    python
    df.describe()

Together these give a quick overview of the size, structure, and statistical properties of the dataset.

Matplotlib is a widely used Python library for creating static, animated, and interactive visualisations. Its pyplot module provides a MATLAB-like interface.

Importance:

• Converts raw data into visual insights (charts and graphs).
• Helps identify trends, patterns, and outliers.
• Supports many plot types: line, bar, histogram, pie, scatter, etc.
• Highly customizable (titles, labels, colors, legends).
• Integrates seamlessly with Pandas and NumPy.

Basic steps to create a plot:

1. Import the library:
   python
   import matplotlib.pyplot as plt

2. Prepare the data:
   python
   x = [1, 2, 3, 4]
   y = [10, 20, 25, 30]

3. Create the plot:
   python
   plt.plot(x, y)

4. Add labels and title:
   python
   plt.xlabel('X-axis')
   plt.ylabel('Y-axis')
   plt.title('Sample Plot')

5. Display the plot:
   python
   plt.show()

A line plot displays data points connected by straight lines and is commonly used to show trends over time or continuous data.

Basic line plot:
python
import matplotlib.pyplot as plt
x = [1, 2, 3, 4, 5]
y = [2, 4, 6, 8, 10]
plt.plot(x, y)
plt.show()

Common customizations:

• Color: plt.plot(x, y, color='red')
• Line style: linestyle='--' (dashed), ':' (dotted)
• Line width: linewidth=2
• Markers: marker='o' to highlight data points
• Label & legend:
  python
  plt.plot(x, y, label='Sales')
  plt.legend()

Complete example:
python
plt.plot(x, y, color='green', linestyle='--', marker='o', linewidth=2, label='Growth')
plt.title('Sales Growth')
plt.xlabel('Month')
plt.ylabel('Sales')
plt.legend()
plt.grid(True)
plt.show()

Line plots are ideal for visualizing time-series and continuous relationships.

Subplots allow multiple plots to be displayed within a single figure, arranged in a grid of rows and columns. This is useful for comparing multiple datasets side by side.

Using plt.subplot():

Syntax: plt.subplot(nrows, ncols, index)
python
import matplotlib.pyplot as plt

plt.subplot(1, 2, 1) # 1 row, 2 cols, first plot
plt.plot([1, 2, 3], [4, 5, 6])
plt.title('Plot 1')

plt.subplot(1, 2, 2) # second plot
plt.plot([1, 2, 3], [6, 5, 4])
plt.title('Plot 2')

plt.show()

Using plt.subplots() (recommended):

Returns a figure and an array of axes.
python
fig, ax = plt.subplots(2, 2) # 2x2 grid

ax[0, 0].plot([1, 2, 3])
ax[0, 1].bar(['A', 'B'], [3, 7])
ax[1, 0].hist([1, 2, 2, 3, 3, 3])
ax[1, 1].pie([30, 70])

plt.tight_layout()
plt.show()

Key points:

• tight_layout() prevents overlapping of plots.
• Each subplot can be a different chart type.
• Improves comparison and presentation of data.

A histogram is a graphical representation of the distribution of numerical data. It groups data into bins (intervals) and shows the frequency of data points in each bin using bars.

Key characteristics:

• Bars are adjacent (no gaps) because data is continuous.
• X-axis represents intervals/bins.
• Y-axis represents frequency (count).

Creating a histogram — plt.hist():
python
import matplotlib.pyplot as plt
data = [12, 15, 15, 18, 20, 22, 22, 22, 25, 30]
plt.hist(data, bins=5, color='skyblue', edgecolor='black')
plt.xlabel('Value')
plt.ylabel('Frequency')
plt.title('Histogram Example')
plt.show()

Important parameters:

• bins – number of intervals.
• color – fill color of bars.
• edgecolor – border color of bars.
• range – lower and upper range of bins.

Uses:

• Understanding the frequency distribution of data.
• Detecting skewness, spread, and outliers.
• Common in statistics and exploratory data analysis.

Explanation:

• A histogram represents the distribution of a single continuous variable by dividing it into bins. Bars touch each other because the data is continuous.
• A bar chart compares distinct categories. Since categories are independent, bars are drawn with gaps between them.

Example:

• Histogram → distribution of student marks (0-10, 10-20, ...).
• Bar chart → number of students in each department (CSE, ECE, ME).

A bar chart represents categorical data with rectangular bars whose lengths are proportional to the values they represent.

Vertical bar chart — plt.bar():
python
import matplotlib.pyplot as plt
categories = ['A', 'B', 'C', 'D']
values = [25, 40, 30, 55]
plt.bar(categories, values, color='orange')
plt.xlabel('Category')
plt.ylabel('Value')
plt.title('Vertical Bar Chart')
plt.show()

Horizontal bar chart — plt.barh():
python
plt.barh(categories, values, color='green')
plt.xlabel('Value')
plt.ylabel('Category')
plt.title('Horizontal Bar Chart')
plt.show()

Difference:

• Vertical bar chart (bar): bars rise upward from the x-axis; good for comparing values across categories.
• Horizontal bar chart (barh): bars extend sideways from the y-axis; useful when category names are long or there are many categories.

Customizations: color, width, edgecolor, labels, and legends can be applied to both types.

A pie chart is a circular statistical chart divided into slices (sectors), where each slice represents a proportion of the whole. The size of each slice is proportional to the quantity it represents.

Creating a pie chart — plt.pie():
python
import matplotlib.pyplot as plt
sizes = [30, 25, 20, 25]
labels = ['Python', 'Java', 'C++', 'Others']
plt.pie(sizes, labels=labels, autopct='%1.1f%%', startangle=90)
plt.title('Language Popularity')
plt.show()

Important parameters:

• labels – names of the slices.
• autopct – displays percentage on each slice (e.g., '%1.1f%%').
• startangle – rotation angle of the first slice.
• explode – pulls a slice out for emphasis.
• colors – custom slice colors.

Advantages:

• Easy to understand proportions at a glance.
• Visually appealing for a small number of categories.

Limitations:

• Not suitable when there are too many categories.
• Difficult to compare slices of similar size.
• Cannot show trends over time.

Missing data (represented as NaN) is common in real-world datasets. Pandas provides several methods to detect and handle it.

1. Detecting missing values:
python
df.isnull() # True where value is missing
df.isnull().sum() # count of missing per column
df.notnull() # opposite of isnull

2. Dropping missing values — dropna():
python
df.dropna() # drop rows with any NaN
df.dropna(axis=1) # drop columns with NaN
df.dropna(how='all') # drop rows where all values are NaN

3. Filling missing values — fillna():
python
df.fillna(0) # fill with constant
df['Age'].fillna(df['Age'].mean()) # fill with mean
df.fillna(method='ffill') # forward fill
df.fillna(method='bfill') # backward fill

4. Replacing values — replace():
python
df.replace(to_replace=np.nan, value=0)

Best practice:

• Use dropna() when missing data is small and can be discarded.
• Use fillna() with mean/median/mode to preserve data when appropriate.

Pandas provides two primary indexers for selecting data from a DataFrame: loc[] and iloc[].

loc[] — Label-based indexing:

• Selects data using row/column labels (names).
• The end label in a slice is included.
  python
  df.loc[2] # row with label 2
  df.loc[0:3, 'Name'] # rows 0 to 3 (inclusive), column 'Name'
  df.loc[df['Age'] > 25] # boolean selection

iloc[] — Integer position-based indexing:

• Selects data using integer positions (0-based).
• The end position in a slice is excluded.
  python
  df.iloc[0] # first row
  df.iloc[0:3] # rows at positions 0,1,2
  df.iloc[1, 2] # value at row 1, column 2

Distinction:

Choosing the correct indexer avoids confusion, especially when the index is not a simple integer range.

Adding descriptive elements makes a plot informative and readable.

1. Title — plt.title():
python
plt.title('Monthly Sales Report')

2. Axis labels — plt.xlabel(), plt.ylabel():
python
plt.xlabel('Month')
plt.ylabel('Sales in units')

3. Legend — plt.legend():

• Identifies multiple data series. Requires label in the plot call.
  python
  plt.plot(x, y, label='2023')
  plt.plot(x, z, label='2024')
  plt.legend()

4. Grid — plt.grid():
python
plt.grid(True)

Complete example:
python
plt.plot(x, y, label='Sales')
plt.title('Sales Trend')
plt.xlabel('Month')
plt.ylabel('Sales')
plt.legend()
plt.grid(True)
plt.show()

Importance:

• Title conveys the purpose of the chart.
• Labels clarify what each axis represents.
• Legend distinguishes multiple datasets.
• Grid improves readability of values.

Together these turn a plain chart into a clear, professional visualisation.

DataFrame attributes provide metadata and structural information about the data.

1. shape

• Returns a tuple (rows, columns).
  python
  df.shape # e.g., (100, 5)

2. size

• Returns the total number of elements (rows × columns).
  python
  df.size # e.g., 500

3. columns

• Returns the column labels as an Index object.
  python
  df.columns # Index(['Name', 'Age', 'City'])

4. index

• Returns the row labels/index of the DataFrame.
  python
  df.index # RangeIndex(start=0, stop=100, step=1)

5. dtypes

• Returns the data type of each column.
  python
  df.dtypes # Name: object, Age: int64

6. values

• Returns the data as a NumPy array.
  python
  df.values

These attributes are essential for understanding the structure of a dataset before performing operations.

Pandas integrates tightly with Matplotlib, allowing plots to be generated directly from DataFrame and Series objects using the .plot() method (which internally uses Matplotlib).

Reading and plotting a CSV dataset:
python
import pandas as pd
import matplotlib.pyplot as plt

df = pd.read_csv('sales.csv')

1. Line plot from DataFrame:
python
df.plot(x='Month', y='Sales', kind='line')
plt.show()

2. Bar chart:
python
df.plot(x='Product', y='Revenue', kind='bar')
plt.show()

3. Histogram:
python
df['Age'].plot(kind='hist', bins=10)
plt.show()

4. Pie chart:
python
df['Sales'].plot(kind='pie', labels=df['Region'], autopct='%1.1f%%')
plt.show()

Common kind values: 'line', 'bar', 'barh', 'hist', 'pie', 'scatter', 'box'.

Advantages:

• Less code — plotting directly from data.
• Automatic handling of axis labels from column names.
• Combines data processing and visualisation seamlessly.

This integration makes exploratory data analysis (EDA) fast and intuitive.

Choosing the right chart type is crucial for effective data communication.

Detailed usage:

• Line plot: Use when you want to show how a quantity changes over time or across a continuous variable (e.g., stock prices, temperature).

• Bar chart: Use to compare discrete categories (e.g., sales by region). Bars have gaps.

• Histogram: Use to understand the distribution/spread of a single numeric variable (e.g., exam scores). Bars are adjacent.

• Pie chart: Use to show percentage composition of a whole with few categories (e.g., market share). Avoid when categories are many or similar in size.

Conclusion: The choice depends on whether you want to show trends, comparisons, distributions, or proportions.