Back to Subject

Unit 1: Introduction and Data Preparation

INT234 — Predictive Analytics 9 min read

I. Orientation — Learning Patterns from Data

Predictive analytics is the systematic use of historical data, statistical methods, and machine-learning algorithms to estimate unknown or future outcomes. Its governing principle is that patterns observed in representative past data can support predictions about new cases, provided that the underlying process remains sufficiently stable.

  • Core objective: Predict a target such as sales revenue, customer churn, disease risk, equipment failure, or product demand.
  • Basic workflow:
    1. Define the prediction problem and target variable.
    2. Collect relevant historical data.
    3. Clean and transform the data.
    4. Train a predictive model.
    5. Evaluate it on unseen data.
    6. Deploy and monitor its predictions.
  • Observation: A row usually represents one entity or event, such as a customer, transaction, patient, or day.
  • Feature: An input variable used for prediction, such as age, income, temperature, or purchase frequency.
  • Target: The outcome to be predicted, such as a numerical price or a class label.
  • Model: A mathematical function that maps features (X) to a predicted target (\hat{y}):
    TEXT
    ŷ = f(X)
    • (X): Input feature vector or matrix.
    • (f): Learned predictive function.
    • (\hat{y}): Predicted output.
  • Generalization assumption: Training and future data should come from sufficiently similar distributions.
  • Data-quality principle: A sophisticated algorithm cannot reliably compensate for incorrect, biased, irrelevant, or poorly prepared data.
  • Evaluation principle: Performance must be measured on data that was not used to fit the model.

II. Predictive Analytics — From Historical Evidence to Future Outcomes

Predictive analytics converts available data into estimates of future events or unknown values. Unlike descriptive analytics, which explains what has already happened, it focuses on what is likely to happen.

A. Introduction to Predictive Analytics

Predictive analytics combines data, domain knowledge, statistics, and computation to produce actionable forecasts or probability estimates.

  • Analytical categories:
    1. Descriptive analytics: Summarizes past events, such as last quarter’s average sales.
    2. Diagnostic analytics: Investigates causes, such as why sales declined.
    3. Predictive analytics: Estimates future outcomes, such as next month’s sales.
    4. Prescriptive analytics: Recommends actions, such as an optimal inventory level.
  • Prediction forms:
    • Numerical prediction: Estimates a continuous quantity, such as a house price of ₹4,500,000.
    • Class prediction: Assigns a category, such as fraud or legitimate.
    • Probability prediction: Produces a risk score, such as a (0.72) probability of churn.
  • Main components:
    • Historical data: Supplies examples from which patterns are learned.
    • Predictor variables: Describe conditions associated with the outcome.
    • Algorithm: Identifies relationships between predictors and targets.
    • Evaluation metric: Quantifies predictive performance.
  • Statistical foundation: A regression model may represent a target as:
    TEXT
    ŷ = β₀ + β₁x₁ + β₂x₂ + ... + βₚxₚ
    • (\beta_0): Intercept.
    • (\beta_j): Effect estimated for feature (x_j).
    • (p): Number of features.
  • Business value: Predictions support decisions such as credit approval, preventive maintenance, demand planning, and targeted marketing.
  • Correlation limitation: A predictive relationship does not automatically establish causation; an accurate churn predictor may identify risk without explaining what intervention will prevent churn.

B. Applications and Limitations

Predictive systems are useful when predictions are connected to well-defined decisions and measurable outcomes.

  • Finance: Credit-scoring models estimate default probability using repayment history, debt, and income.
  • Healthcare: Risk models estimate readmission or disease probability from clinical measurements.
  • Retail: Demand forecasts use prices, promotions, seasons, and previous sales.
  • Manufacturing: Sensor readings predict failure before equipment stops operating.
  • Limitations:
    • Concept drift: Relationships can change, as when customer behavior shifts after a policy change.
    • Bias: Unrepresentative historical data can disadvantage particular groups.
    • Data leakage: A model may accidentally use information available only after the outcome.
    • Uncertainty: Predictions are estimates and should not be treated as guaranteed facts.

III. Machine Learning — Algorithms That Learn from Examples

Machine learning is a branch of artificial intelligence in which computational systems improve their performance on a task by learning patterns from data rather than relying only on explicitly programmed rules.

A. Machine learning and its types

Machine-learning types are distinguished primarily by the information available during training and the kind of feedback received.

  • Learning formulation: Given experience (E), a task (T), and performance measure (P), learning occurs when performance on (T), measured by (P), improves with (E).
  • Major types:
    1. Supervised learning: Learns from labelled examples containing both inputs (X) and known outputs (y).
    2. Unsupervised learning: Finds structures in unlabelled input data (X).
    3. Semi-supervised learning: Uses a small labelled dataset together with a larger unlabelled dataset.
    4. Reinforcement learning: An agent learns actions through rewards and penalties received from an environment.
  • Typical tasks:
    • Classification: Predicts discrete classes, such as spam or not spam.
    • Regression: Predicts continuous values, such as temperature.
    • Clustering: Groups similar observations without known labels.
    • Dimensionality reduction: Represents many variables through fewer informative dimensions.
  • Training process: Model parameters are adjusted to minimize a loss function:
    TEXT
    θ* = arg minθ L(y, fθ(X))
    • (\theta): Model parameters.
    • (f_\theta): Model defined by those parameters.
    • (L): Loss measuring prediction error.
    • (\theta^*): Parameters producing the lowest training objective.
  • Generalization balance:
    • Underfitting: The model is too simple and performs poorly even on training data.
    • Overfitting: The model memorizes training details and performs poorly on unseen data.

IV. Labelled Learning — Predicting Known Outcomes

Supervised learning uses examples for which the correct output is known. Its purpose is to learn a mapping that produces accurate outputs for previously unseen observations.

A. Supervised learning

Supervised learning estimates a relationship between predictor variables and a labelled target variable.

  • Dataset structure:
    TEXT
    D = {(x₁, y₁), (x₂, y₂), ..., (xₙ, yₙ)}
    • (D): Labelled dataset.
    • (x_i): Feature vector for observation (i).
    • (y_i): Known target for observation (i).
    • (n): Number of observations.
  • Regression: Used when (y) is continuous; common algorithms include linear regression, regression trees, random forests, and support-vector regression.
  • Classification: Used when (y) is categorical; common algorithms include logistic regression, decision trees, support-vector machines, k-nearest neighbours, and neural networks.
  • Data partitions:
    • Training set: Fits model parameters.
    • Validation set: Selects features and hyperparameters.
    • Test set: Estimates final performance on unseen data.
  • Regression metrics:
    TEXT
    MAE = (1/n) Σ|yᵢ - ŷᵢ|
MSE = (1/n) Σ(yᵢ - ŷᵢ)²
    • MAE measures average absolute error.
    • MSE penalizes large errors more heavily through squaring.
  • Classification metrics: Accuracy measures the proportion correctly classified, while precision, recall, and F1-score are more informative when classes are imbalanced.
  • Worked example: To predict house prices, floor area, location, age, and room count form (X), while recorded sale price forms (y). A model trained on earlier sales predicts (\hat{y}) for a newly listed house.
  • Limitation: Reliable labels may be expensive, delayed, inconsistent, or affected by human judgement.

V. Unlabelled Learning — Discovering Hidden Structure

Unsupervised learning analyzes data without a known target variable. Its purpose is to reveal natural groups, compact representations, unusual observations, or relationships within the feature space.

A. Unsupervised learning

Unsupervised learning identifies structure using similarities, distances, densities, or statistical dependence among observations.

  • Clustering: Divides observations into internally similar groups.
    • K-means assigns each observation to the nearest of (K) cluster centroids.
    • Hierarchical clustering forms a tree of progressively merged or divided groups.
    • DBSCAN identifies dense regions and can label sparse observations as noise.
  • K-means objective:
    TEXT
    Minimize Σᵢ Σₖ rᵢₖ ||xᵢ - μₖ||²
    • (r_{ik}=1) when observation (i) belongs to cluster (k), otherwise (0).
    • (\mu_k): Centroid of cluster (k).
    • (||x_i-\mu_k||^2): Squared distance to the centroid.
  • Dimensionality reduction: Principal component analysis transforms correlated features into fewer uncorrelated components that preserve as much variance as possible.
  • Association discovery: Market-basket analysis identifies items frequently occurring together, such as bread and butter.
  • Anomaly detection: Rare observations far from normal patterns may indicate fraud, defects, or network intrusion.
  • Worked example: Customers can be clustered using annual spending and visit frequency. The resulting groups might represent occasional, regular, and high-value customers, although the algorithm does not supply those business labels.
  • Evaluation difficulty: Because correct labels are absent, quality may be assessed using cohesion, separation, stability, visualization, or practical usefulness.
  • Interpretation limitation: A mathematically distinct cluster is not automatically a meaningful real-world segment.

VI. Data Preparation — Creating Model-Ready Information

Data preparation converts raw observations into a consistent, informative, and algorithm-compatible dataset. It often determines model reliability more strongly than small differences between algorithms.

A. Data preprocessing

Data preprocessing detects and corrects quality problems before model training while preventing information from leaking across dataset partitions.

  • Data profiling: Examine data types, ranges, unique values, class frequencies, distributions, and missing-value rates.
  • Missing values:
    • Deletion: Remove rows or columns when missingness is limited and removal does not create serious bias.
    • Imputation: Replace numerical gaps using a median or model-based estimate and categorical gaps using a mode or explicit “Unknown” category.
    • Indicator: Add a binary feature when the fact that a value is missing may itself carry information.
  • Duplicate and inconsistent records: Remove unintended duplicate rows and standardize values such as Delhi, DELHI, and New Delhi according to defined business rules.
  • Outliers: Investigate extreme values using domain limits, box plots, z-scores, or the interquartile range:
    TEXT
    IQR = Q₃ - Q₁
Lower fence = Q₁ - 1.5(IQR)
Upper fence = Q₃ + 1.5(IQR)
    • (Q_1): First quartile.
    • (Q_3): Third quartile.
    • An outlier may be an error or a valid rare case; it should not be removed automatically.
  • Categorical encoding:
    • One-hot encoding creates binary columns for unordered categories.
    • Ordinal encoding assigns ordered values only when a genuine ranking exists.
  • Feature scaling:
    TEXT
    Standardization: z = (x - μ) / σ
Min-max scaling: x' = (x - xmin) / (xmax - xmin)
    • (\mu): Training mean.
    • (\sigma): Training standard deviation.
    • (x{\min}), (x{\max}): Training minimum and maximum.
  • Feature engineering: Derive informative variables, such as customer tenure from joining date or unit price from revenue divided by quantity.
  • Feature selection: Remove irrelevant, redundant, highly noisy, or leakage-prone variables to reduce complexity.
  • Class imbalance: Use class weights, resampling, or synthetic sampling carefully when one class is rare; evaluate with precision, recall, F1-score, or precision-recall curves rather than accuracy alone.
  • Correct processing order: Split the dataset first, fit preprocessing operations only on training data, and then apply the learned transformations to validation and test data.
  • Pipeline principle: Store imputation, encoding, scaling, and modelling in one reproducible pipeline so that training and production observations receive identical transformations.
Next
Unit 2