Unit 14: Machine Learning Packages in Python

I. Orientation

Python’s machine-learning ecosystem combines packages for loading data, visualizing patterns, preprocessing variables, training models, and evaluating predictions. A typical workflow moves from raw data to a validated model, while keeping training and testing operations separate to prevent misleading results.

• Governing workflow: Import data → inspect and clean → visualize → select features and target → split data → preprocess → train → predict → evaluate.
• Core packages:
  • pandas: Imports and manipulates tabular data through DataFrame objects.
  • Matplotlib: Provides foundational plotting functions and detailed figure control.
  • Seaborn: Creates statistical graphics using a high-level interface built on Matplotlib.
  • scikit-learn: Supplies preprocessing tools, machine-learning algorithms, and evaluation metrics.
• Observation convention: A dataset usually stores one observation per row and one variable per column.
• Feature convention: The feature matrix (X) contains input variables, while the target vector (y) contains the value or class to predict.
• Reproducibility: Fixed values such as random_state=42 make randomized data splits and models repeatable.
• Main assumption: Training data should be sufficiently representative of the unseen data on which the model will operate.

II. Data Import — Bringing External Data into Python

Data import converts information from files, databases, or web-compatible formats into an in-memory structure that Python packages can analyze.

A. Data import

Data is commonly imported into a pandas DataFrame, which preserves row-and-column organization and supports inspection, cleaning, and feature selection.

• CSV files: pd.read_csv() reads comma-separated values; useful arguments include sep, header, names, usecols, dtype, and parse_dates.
• Excel workbooks: pd.read_excel("file.xlsx", sheet_name="Sales") selects a named worksheet; reading .xlsx files generally requires a compatible Excel engine.
• JSON data: pd.read_json() imports structured JSON, while pd.json_normalize() flattens nested records.
• Database tables: pd.read_sql(query, connection) executes a query through an established database connection and returns tabular results.
• Missing values: Entries such as empty fields, "NA", or "?" can be standardized during import with na_values=["NA", "?"].
• Immediate inspection:
  • df.head() displays the first five rows by default.
  • df.shape returns (number_of_rows, number_of_columns).
  • df.info() reports column names, non-null counts, and data types.
  • df.describe() summarizes numerical columns using statistics such as mean and quartiles.

import pandas as pd

df = pd.read_csv(
    "customers.csv",
    usecols=["age", "income", "purchased"],
    dtype={"age": "int64", "income": "float64"},
    na_values=["NA", "?"]
)

print(df.head())
print(df.isna().sum())

• Concrete interpretation: If df.shape is (500, 3), the imported table contains 500 observations and 3 variables.
• Path handling: A relative path such as "data/customers.csv" is resolved from the program’s current working directory; an absolute path identifies the complete location.

B. Applications and limitations

Reliable import preserves the meaning of the source data, but imported values must still be validated before analysis.

• Type validation: A numeric column imported as object may contain currency signs or malformed text; pd.to_numeric(column, errors="coerce") converts invalid entries to missing values.
• Date validation: pd.to_datetime(df["date"], errors="coerce") converts parseable values and marks invalid dates as NaT.
• Memory management: Large files can be processed incrementally with pd.read_csv("large.csv", chunksize=10000).
• Encoding limitation: Text files may require an explicit encoding such as encoding="utf-8"; the wrong encoding can cause errors or damaged characters.
• Data-quality limitation: Successful import does not guarantee correct labels, units, categories, or measurements.
• Security convention: Database credentials and private URLs should be stored in environment variables rather than written directly in source code.

III. Matplotlib — Foundational Python Visualization

Matplotlib represents a visualization through a Figure, which contains one or more Axes; plotting methods add graphical elements to those axes.

A. Visualization with Matplotlib

Matplotlib transforms numerical values into visual marks so distributions, comparisons, trends, and anomalies can be inspected before modeling.

• Standard import: import matplotlib.pyplot as plt exposes the state-based plotting interface under the conventional name plt.
• Object-oriented interface: fig, ax = plt.subplots() explicitly creates the figure and axes, making complex plots easier to control.
• Essential annotations:
  • ax.set_title() states the plot’s purpose.
  • ax.set_xlabel() and ax.set_ylabel() identify variables and units.
  • ax.legend() explains colors or line styles.
  • ax.grid(True) adds reference lines for reading values.
• Figure size: plt.subplots(figsize=(7, 4)) specifies width and height in inches.
• Output operations: plt.show() displays a figure, while fig.savefig("plot.png", dpi=300, bbox_inches="tight") writes a high-resolution image.
• Accurate scales: Axis limits should not be manipulated in ways that exaggerate small differences; labels should include units such as "Temperature (°C)".

import matplotlib.pyplot as plt

fig, ax = plt.subplots(figsize=(6, 4))
ax.bar(["A", "B", "C"], [18, 25, 21], color="steelblue")
ax.set(title="Sales by Region", xlabel="Region", ylabel="Sales (thousands)")
ax.grid(axis="y", alpha=0.3)
plt.show()

• Concrete reading: The bar for region B reaches 25, so it exceeds region A’s value of 18 by 7 thousand units.

B. Simple line and scatter plots

Line plots emphasize ordered change, whereas scatter plots emphasize relationships between paired numerical observations.

1. Simple line plots:

   • Purpose: A line plot connects ordered points and is especially suitable for time series or measurements taken across increasing values.
   • Method: ax.plot(x, y) plots coordinates ((x_i, y_i)), where (x_i) is the position of observation (i) and (y_i) is its measured value.
   • Styling: marker="o", linestyle="--", color="navy", and label="Series A" distinguish observations and series.
   • Interpretation: Connecting points implies meaningful ordering; it should not imply continuity between unrelated categories.

2. Simple scatter plots:

   • Purpose: A scatter plot uses unconnected points to reveal direction, strength, clusters, nonlinear patterns, and possible outliers.
   • Method: ax.scatter(x, y) places one marker at each paired observation.
   • Additional variables: Arguments c and s can encode marker color and size, but legends or color bars must explain those encodings.
   • Interpretation: An upward point pattern suggests positive association, but association alone does not establish causation.

fig, axes = plt.subplots(1, 2, figsize=(10, 4))

days = [1, 2, 3, 4, 5]
sales = [12, 15, 14, 19, 23]
axes[0].plot(days, sales, marker="o")
axes[0].set(title="Daily Sales", xlabel="Day", ylabel="Units")

hours = [1, 2, 3, 4, 5]
scores = [52, 58, 67, 73, 85]
axes[1].scatter(hours, scores, color="crimson")
axes[1].set(title="Study Time and Score",
            xlabel="Study time (hours)", ylabel="Score")

plt.tight_layout()
plt.show()

• Explicit contrast: The first graph connects sales in chronological order; the second leaves score observations unconnected because the aim is to examine association.

IV. Seaborn — Statistical Visualization

Seaborn integrates closely with pandas and provides theme-aware functions that can automatically map DataFrame columns to visual properties.

A. Seaborn

Seaborn simplifies statistical plotting while retaining access to Matplotlib’s figure, axes, labels, and export controls.

• Standard import: import seaborn as sns uses the package’s conventional alias.
• Data mapping: In sns.scatterplot(data=df, x="income", y="spending"), column names are mapped directly to axes.
• Semantic variables: hue="segment", style="segment", and size="orders" represent additional variables through color, marker form, and marker area.
• Common functions:
  • sns.histplot() displays distributions through bins.
  • sns.boxplot() summarizes median, quartiles, spread, and potential outliers.
  • sns.countplot() shows category frequencies.
  • sns.regplot() combines a scatter plot with a fitted regression line.
• Themes: sns.set_theme(style="whitegrid") applies consistent backgrounds, grids, fonts, and color conventions.

import seaborn as sns
import matplotlib.pyplot as plt

sns.set_theme(style="whitegrid")
sns.boxplot(data=df, x="purchased", y="income")
plt.xlabel("Purchase class")
plt.ylabel("Income")
plt.title("Income by Purchase Class")
plt.show()

• Concrete interpretation: The line inside each box is the median; the box extends from the first quartile to the third quartile, forming the interquartile range.
• Limitation: Attractive defaults do not replace correct plot selection, readable labels, representative data, or careful interpretation.

B. Heatmap

A heatmap displays values in a rectangular matrix by mapping numerical magnitude to color.

• Structure: Rows and columns identify variable pairs or categories, while each cell color represents its associated value.
• Correlation use: df.corr(numeric_only=True) computes pairwise correlation coefficients for numerical columns.
• Correlation range: Pearson’s coefficient (r) ranges from (-1) to (1):
  • (r) near (1) indicates strong positive linear association.
  • (r) near (-1) indicates strong negative linear association.
  • (r) near (0) indicates weak linear association, not necessarily no relationship.
• Annotations: annot=True prints values inside cells, and fmt=".2f" displays two decimal places.
• Color selection: A diverging palette centered at zero distinguishes negative and positive correlations.

numeric = df[["age", "income", "purchased"]]
correlations = numeric.corr()

sns.heatmap(
    correlations,
    annot=True,
    fmt=".2f",
    cmap="coolwarm",
    center=0,
    vmin=-1,
    vmax=1
)
plt.title("Feature Correlation Heatmap")
plt.show()

• Modeling application: Two predictors with correlation close to (1) may carry largely overlapping information.
• Limitation: Correlation can be affected by outliers and does not prove that one variable causes another.

V. Scikit-learn — Machine-Learning Tools and Conventions

Scikit-learn provides a consistent estimator interface for supervised learning, unsupervised learning, preprocessing, model selection, and evaluation.

A. Introducing Scikit-learn package

Scikit-learn models generally learn from data with fit() and apply the learned relationship through predict() or transform().

• Data representation: X is commonly a two-dimensional feature matrix of shape (n_samples, n_features), while y contains one target value per sample.
• Estimator interface:
  • model.fit(X_train, y_train) estimates parameters from training data.
  • model.predict(X_test) generates target predictions.
  • transformer.transform(X) applies a learned conversion.
  • fit_transform(X_train) learns and applies a transformation to training data.
• Train-test split: train_test_split(..., test_size=0.2) reserves 20% of observations for evaluation.
• Classification: Algorithms such as LogisticRegression predict categories; metrics include accuracy, precision, recall, and F1 score.
• Regression: Algorithms such as LinearRegression predict continuous values; metrics include mean absolute error and mean squared error.
• Preprocessing: StandardScaler transforms each feature using
  [
  z=\frac{x-\mu}{\sigma},
  ]
  where (x) is the original value, (\mu) is the training mean, (\sigma) is the training standard deviation, and (z) is the standardized value.

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

X = df[["age", "income"]]
y = df["purchased"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)
predictions = model.predict(X_test)

print(accuracy_score(y_test, predictions))

• Concrete interpretation: With 500 observations and test_size=0.2, approximately 400 observations train the model and 100 evaluate it.

B. Applications and limitations

Scikit-learn supports efficient experimentation, but valid results depend on sound data separation and suitable evaluation.

• Pipelines: Pipeline chains preprocessing and modeling so each step is learned only from training data.
• Data leakage: Scaling or imputing the complete dataset before splitting allows test information to influence training and produces overly optimistic scores.
• Cross-validation: cross_val_score() evaluates a model across multiple partitions, reducing dependence on one split.
• Class imbalance: Accuracy can mislead when one class dominates; precision, recall, F1 score, or ROC-AUC may be more informative.
• Generalization: Strong training performance with weak test performance indicates overfitting.
• Scope limitation: Scikit-learn primarily targets conventional machine learning on in-memory data; specialized deep-learning or distributed-data tasks often require other frameworks.