Unit 2: Introduction to Statistics and Data Analysis

I. Orientation — Learning from Data

Statistics is the science of collecting, organizing, analyzing, and interpreting data so that evidence can guide conclusions under uncertainty. Descriptive statistics summarize observed data, whereas inferential statistics use sample evidence and probability to draw conclusions about a larger population.

Defining framework:

• Population: The complete set of individuals, objects, measurements, or events of interest; for example, all batteries produced by a factory this month.
• Sample: A subset observed from the population; a simple random sample gives every possible sample of a fixed size an equal chance of selection.
• Variable: A characteristic recorded on each observational unit, such as lifetime in hours, blood type, or number of defects.
• Parameter: A fixed, usually unknown population quantity, such as the population mean (\mu) or proportion (p).
• Statistic: A quantity calculated from sample data, such as the sample mean (\bar{x}), used to describe data or estimate a parameter.
• Variation: Differences among observations are expected; statistical methods distinguish systematic patterns from random fluctuation.
• Measurement scales:
  • Nominal: Unordered categories, such as blood groups.
  • Ordinal: Ordered categories, such as low, medium, and high.
  • Interval: Meaningful differences without a true zero, such as Celsius temperature.
  • Ratio: Meaningful differences and ratios, such as mass in kilograms.

II. Statistical Inference — Generalizing Beyond Observations

A. Statistical Inference

Statistical inference uses sample data and probability models to estimate population characteristics or assess claims about them.

• Estimation: A point estimate gives one plausible parameter value, while an interval estimate gives a range with a stated confidence level; for example, (\bar{x}=52.1) may estimate (\mu).
• Hypothesis testing: A null hypothesis (H_0) represents a baseline claim, and an alternative (H_1) represents a competing claim.
• Sampling distribution: Repeated samples produce different statistic values; the probability distribution of those values quantifies sampling uncertainty.
• Standard error: The standard deviation of a statistic’s sampling distribution measures its typical sampling fluctuation. For independent observations with population standard deviation (\sigma):

SE(x̄) = σ / √n

Here, (SE(\bar{x})) is the standard error of the sample mean and (n) is sample size.

• Possible errors: A Type I error rejects a true (H_0); a Type II error fails to reject a false (H_0).
• Conditions: Valid inference depends on appropriate sampling, credible model assumptions, independent observations when required, and accurate measurements.

III. Samples, Populations and Experimental Design — Producing Credible Evidence

A. Samples, Populations and Experimental Design

Reliable statistical conclusions begin with a representative sample or a well-controlled experiment.

• Target and sampled populations: The target population is the group about which conclusions are desired; the sampled population is the group actually accessible through the sampling frame.
• Sampling methods:
  • Simple random sampling: Selection is governed entirely by chance.
  • Stratified sampling: The population is divided into groups, such as age bands, and sampled within each group.
  • Cluster sampling: Naturally occurring groups, such as schools, are selected.
  • Convenience sampling: Easily reached units are used, creating substantial selection-bias risk.
• Experimental units and treatments: Experimental units receive specified conditions; in a fertilizer study, plots are units and fertilizer levels are treatments.
• Core design principles: Random assignment balances uncontrolled factors probabilistically, replication reveals variability, and control groups provide comparisons.
• Blocking: Units with similar characteristics are grouped before random assignment; blocking hospital patients by age can reduce age-related variability.
• Confounding: Two effects are confounded when their separate influences cannot be identified; testing one teaching method only in morning classes confounds method with time.
• Observational limitation: An association found without assigned treatments does not alone establish causation because lurking variables may explain it.

IV. Measures of Location — Identifying the Data Center

A. Measures of Location: The Sample Mean and Median

The sample mean and median describe a dataset’s central location but respond differently to skewness and extreme observations.

1. Sample mean:
   • Definition: The arithmetic average uses every observation.

x̄ = (1/n) Σᵢ₌₁ⁿ xᵢ

Here, (x_i) is observation (i), (n) is the number of observations, and (\bar{x}) is their sample mean.

• Property: The deviations from the mean sum to zero: (\sum(x_i-\bar{x})=0).
• Sensitivity: A single unusually large value can pull (\bar{x}) upward, making it most suitable for approximately symmetric data without severe outliers.

2. Sample median:
   • Definition: After ordering observations, the median is the middle value for odd (n), or the average of the two middle values for even (n).
   • Resistance: Its position, rather than every numerical magnitude, determines it; therefore it is resistant to outliers.
   • Contrast: For (2,3,4,5,30), (\bar{x}=8.8), but the median is (4); their separation signals right-skewness.

V. Measures of Variability — Quantifying Spread

A. Measures of Variability

Measures of variability describe how widely observations differ from one another and from their center.

• Range: The maximum minus the minimum is simple but depends only on two observations: (R=x{\max}-x{\min}).
• Sample variance: Squared deviations from the sample mean produce:

s² = [Σᵢ₌₁ⁿ (xᵢ − x̄)²] / (n − 1)

Here, (s^2) is sample variance; division by (n-1) makes it an unbiased estimator of population variance (\sigma^2) under random sampling.

• Sample standard deviation: (s=\sqrt{s^2}) expresses spread in the original measurement unit; if heights are measured in centimeters, (s) is also in centimeters.
• Interquartile range: (IQR=Q_3-Q_1) measures the spread of the middle 50% and resists extreme values.
• Coefficient of variation: For positive ratio-scale data, (CV=(s/\bar{x})\times100\%) compares relative variability across differing units or means.
• Interpretation: A small (s) indicates clustering around (\bar{x}), not necessarily clustering around a scientifically desirable value.

VI. Discrete and Continuous Data — Distinguishing Numerical Forms

A. Discrete and Continuous Data

Quantitative data are classified by whether possible values are countable or occupy a continuum.

1. Discrete data:

   • Nature: Values are distinct and countable, commonly resulting from counting; examples include 0, 1, 2, … defects per component.
   • Probability: A probability mass function assigns probabilities to exact values, so (P(X=3)) may be positive.
   • Display: Bar charts and dot plots preserve the separation between allowable values.

2. Continuous data:

   • Nature: Values can theoretically occur anywhere within an interval; examples include time, mass, and temperature.
   • Probability: A density describes interval probabilities, while any exact value has probability zero: (P(X=x)=0).
   • Measurement: Recorded precision discretizes observations in practice; a mass reported as 12.4 g represents a rounded continuous measurement.
   • Contrast: For continuous (X), meaningful probabilities concern ranges such as (P(10<X<15)), not isolated points.

VII. Statistical Modeling — Representing Systematic and Random Behavior

A. Statistical Modeling

A statistical model is a simplified mathematical representation connecting observed variation to systematic structure and randomness.

• General form: A response may be decomposed into a modeled component and random error:

Y = f(X; θ) + ε

Here, (Y) is the response, (X) contains explanatory variables, (\theta) contains unknown parameters, (f) is the systematic relationship, and (\varepsilon) is random error.

• Linear example: In (Y=\beta_0+\beta_1X+\varepsilon), (\beta_0) is the intercept and (\beta_1) is the expected response change per one-unit increase in (X).
• Assumptions: A model may require independence, constant error variance, a specified distribution, or a correctly shaped mean relationship.
• Fitting: Parameters are estimated from data; least squares chooses coefficients that minimize the sum of squared residuals.
• Purpose: Models support explanation, prediction, uncertainty quantification, and adjustment for multiple variables.
• Limitation: A fitted association is not automatically causal, and predictions outside the observed data range are extrapolations.

VIII. Scientific Inspection — Examining Data Before Formal Analysis

A. Scientific Inspection

Scientific inspection systematically checks how data were generated, recorded, and contextualized before numerical conclusions are accepted.

• Provenance: Identify who collected the data, from which units, under what protocol, and during what period; these facts define the scope of inference.
• Data quality: Check impossible values, inconsistent units, duplicates, coding errors, and missing observations; a recorded human height of 18 m demands investigation.
• Missingness: Distinguish random loss from systematic absence; income nonresponse concentrated among high earners can bias estimates downward.
• Contextual plausibility: Compare patterns with subject-matter knowledge without deleting unexpected values merely because they are inconvenient.
• Outlier handling: Verify the source first; correct documented entry errors, but retain genuine extreme observations and use suitable robust analyses.
• Reproducibility: Preserve raw data, record cleaning decisions, and separate original measurements from transformed variables.

IX. Graphical Diagnostics — Checking Models Visually

A. Graphical Diagnostics

Graphical diagnostics reveal violations of model assumptions that a fitted equation or single numerical statistic may conceal.

• Residuals: For observation (i), the residual is (e_i=y_i-\hat{y}_i), where (y_i) is observed and (\hat{y}_i) is fitted.
• Residual-versus-fitted plot: Random scatter around zero supports the mean structure; curvature suggests nonlinearity, while a funnel shape suggests nonconstant variance.
• Normal quantile–quantile plot: Points near a straight line support approximate normality; strong tail departures indicate skewness or heavy tails.
• Time-order plot: Runs, cycles, or trends among residuals suggest dependence; this is especially important for sequential production or financial data.
• Influence diagnostics: Leverage identifies unusual predictor values, whereas influence reflects how strongly an observation changes the fitted model.
• Interpretive rule: Diagnostic plots identify possible problems rather than proving their causes; apparent patterns should be checked against design and measurement context.

X. Graphical Methods and Data Description — Revealing Distributional Structure

A. Graphical Methods and Data Description

Graphical methods communicate a dataset’s distribution, relationships, and unusual features more directly than isolated summary statistics.

• Categorical data: Bar charts compare category frequencies or proportions; bars remain separated because categories are distinct.
• Quantitative distributions: Histograms group values into intervals, while dot plots and stem-and-leaf displays retain more individual detail for small datasets.
• Boxplots: The box spans (Q_1) to (Q_3), a line marks the median, and points beyond conventional fences may flag potential outliers:

Lower fence = Q₁ − 1.5(IQR)
Upper fence = Q₃ + 1.5(IQR)

• Bivariate data: Scatterplots show direction, form, strength, clusters, and outliers in relationships between two quantitative variables.
• Time series: A line plot in chronological order exposes trend, seasonality, abrupt changes, and temporal dependence.
• Complete description: Report shape, center, spread, and unusual features; “right-skewed, median 18 minutes, IQR 7 minutes, with two high outliers” is more informative than the mean alone.
• Integrity: Axes, scales, units, labels, and bin widths must be clear; truncated axes or unequal visual areas can exaggerate differences.

XI. General Types of Statistical Studies — Matching Design to Conclusions

A. General Types of Statistical Studies

Statistical studies are classified by how units are selected, whether treatments are imposed, and when measurements occur.

1. Observational studies:

   • Cross-sectional: Variables are measured at one time, such as a one-day household survey.
   • Case-control: Subjects are selected by outcome status and prior exposures are compared; this is useful for rare outcomes but vulnerable to recall bias.
   • Cohort: Exposed and unexposed groups are followed to compare later outcomes; prospective ordering strengthens temporal evidence.
   • Conclusion: These designs estimate associations but remain vulnerable to confounding.

2. Experimental studies:

   • Randomized experiment: Investigators assign treatments by chance, permitting stronger causal conclusions when compliance and measurement are sound.
   • Matched-pairs design: Each unit receives both treatments in random order, or similar units are paired, reducing between-unit variation.
   • Factorial design: Two or more factors vary together, allowing estimation of main effects and interactions.
   • Conclusion: Random assignment supports causal inference, while random sampling determines how broadly results can be generalized.

• Retrospective versus prospective: Retrospective studies use existing records or recalled exposures; prospective studies define measurements before outcomes occur.
• Census versus sample survey: A census attempts to measure every population unit, whereas a survey uses a subset and quantifies sampling error.
• Design priority: No later statistical technique can fully repair severe selection bias, uncontrolled confounding, or systematically inaccurate measurement.