Unit 2: Data analysis

Data analysis in this unit means inferential statistics: drawing conclusions about a population from a finite sample, quantifying the uncertainty of those conclusions, and updating belief in light of evidence. Everything below rests on the sampling distribution — the distribution a statistic takes over repeated samples — and on the idea that observed data are one realisation of a random process.

• Population vs sample: the population is the full set of interest (parameter , , ); the sample is the observed subset (statistic , , ). Inference estimates the former from the latter.
• Random variable and distribution: a mapping from outcomes to numbers, described by a probability distribution (Normal, , , ).
• Central Limit Theorem: for sample size large, is approximately Normal with mean and standard error , regardless of the parent shape.
• Estimation vs testing: estimation reports a value plus interval; testing decides between two competing statements about a parameter.
• Frequentist vs Bayesian: frequentist methods treat parameters as fixed and data as random; Bayesian methods treat parameters as random and update them with data.

II. Statistical Analysis — describing and summarising data

The foundation on which every test is built: reducing raw data to summaries whose sampling behaviour is known.

A. Types and measures

• Data types: categorical (nominal/ordinal) vs numeric (interval/ratio); the type dictates which test is valid.
• Central tendency: mean , median (50th percentile, robust to outliers), mode (most frequent value).
• Dispersion: variance and standard deviation.

s^2 = (1/(n-1)) * Σ (x_i - x̄)^2
s   = sqrt(s^2)

Symbols: = i-th observation, = sample mean, = sample size, = degrees of freedom (Bessel's correction, giving an unbiased variance estimate).

• Standard error: — the standard deviation of the sampling distribution of the mean, not of the raw data.
• Distribution shape: skewness (asymmetry) and kurtosis (tail weight) determine whether Normal-based tests apply.

B. Estimation and confidence

• Point estimate: single best value, e.g. for .
• Confidence interval (95%): ; interpreted as: 95% of such intervals constructed over repeated sampling contain the true parameter.

III. Hypothesis Testing — the decision framework

A structured procedure for judging whether sample evidence is strong enough to reject a default claim.

A. Hypothesis testing

• Null hypothesis : the status-quo statement of no effect, e.g. .
• Alternative : the claim of an effect, e.g. (two-tailed) or (one-tailed).
• Test statistic: a number measuring distance between data and in SE units, e.g. .
• Rejection region: values of the statistic so extreme they occur with probability under .
• Procedure: state → choose → compute statistic → compare to critical value or p-value → decide.

B. Error types and power

• Type I error (): rejecting a true (false positive); conventionally fixed at 0.05.
• Type II error (): failing to reject a false (false negative).
• Power (): probability of detecting a real effect; rises with larger , larger effect size, and smaller variance.
• Trade-off: lowering raises for fixed — the two cannot be minimised simultaneously without more data.

IV. Significance of the p-value

The p-value converts a test statistic into a single probability that governs the reject/retain decision.

A. Significance of p-value

• Definition: the probability, assuming is true, of obtaining a result at least as extreme as the one observed.
• Decision rule: reject if ; retain otherwise.
• What it is not: it is not the probability that is true, and not the probability the result arose by chance overall.
• Worked example: testing , sample gives . Two-tailed . Since , reject .
• Effect size caveat: with large a trivial difference can yield a tiny p-value; statistical significance is not practical significance.

V. Chi-Square Test — categorical association and fit

A test comparing observed counts against expected counts using the distribution.

A. Statement and statistic

χ² = Σ (O_i - E_i)² / E_i

Symbols: = observed frequency in cell , = expected frequency under . The statistic follows a distribution with degrees of freedom.

• Conditions: independent observations, counts (not percentages), and in most cells.

B. Chi-square

1. Goodness-of-fit: tests whether one categorical variable matches a hypothesised distribution; where = number of categories.
2. Test of independence: tests whether two categorical variables are associated in a contingency table; where , = numbers of rows and columns; .

• Worked example: a die rolled 60 times, expected 10 per face. If a face shows 18, its contribution is ; summing all six cells and comparing to the critical decides fairness.

VI. t-Test — comparing means with unknown variance

Used when is unknown and estimated by , so the statistic follows Student's rather than the Normal.

A. Statement and the t-distribution

• Principle: the -distribution is heavier-tailed than the Normal; as it converges to the Normal.
• Assumptions: approximately Normal population (or large ), and for two-sample tests, independence.

B. t-test

1. One-sample / paired: compares a mean to a reference or compares matched pairs.

t = (x̄ - μ₀) / (s / √n),   df = n - 1

2. Independent two-sample: compares means of two groups.

t = (x̄₁ - x̄₂) / sqrt( s_p²(1/n₁ + 1/n₂) )

Symbols: = hypothesised mean, = pooled variance, = group sizes; .

• Worked example: , , , : with ; compare to → do not reject at 5%.
• Limitation: sensitive to strong non-Normality in small samples; use non-parametric alternatives then.

VII. ANOVA — comparing three or more means

Analysis of variance tests whether several group means differ by partitioning total variability into between- and within-group parts.

A. Statement and the F-statistic

• Rationale: running many t-tests inflates Type I error; ANOVA tests all means jointly with one .
• : all group means equal (); : at least one differs.

F = MS_between / MS_within
MS_between = SS_between / (k - 1)
MS_within  = SS_within  / (N - k)

Symbols: = number of groups, = total observations, = sum of squares, = mean square. follows an distribution with degrees of freedom.

B. ANOVA

• Between-group variance: variability of group means around the grand mean — reflects the treatment effect.
• Within-group variance: variability inside groups — reflects random error.
• Interpretation: large means between-group variation dominates, evidence against .
• One-way vs two-way: one-way examines a single factor; two-way examines two factors and their interaction.
• Post-hoc tests: if is significant, tests such as Tukey's HSD identify which pairs differ.
• Assumptions: independence, Normality within groups, and homogeneity of variances.

VIII. Bayesian Probability — updating belief with evidence

An alternative inferential paradigm in which probability expresses degree of belief, revised as data arrive.

A. Bayesian probability

• Bayes' theorem:

P(H|D) = P(D|H) * P(H) / P(D)
P(D) = Σ P(D|Hi) * P(Hi)

Symbols: = prior (belief before data), = likelihood, = posterior (belief after data), = evidence/normalising constant.

• Prior: encodes existing knowledge; can be informative or uninformative (flat).
• Likelihood: how probable the observed data are under each hypothesis.
• Posterior: the updated distribution; it becomes the prior for the next observation, enabling sequential learning.
• Worked example: a disease affects 1% (); a test is 99% sensitive and 95% specific. For a positive result: , so — only 17% despite the positive test, because the prior is low.
• Contrast with frequentist inference: frequentist p-values ask "how surprising is the data if holds?"; Bayesian posteriors ask "how probable is the hypothesis given the data?", allowing direct probability statements about parameters.
• Credible interval: the Bayesian analogue of the confidence interval — a range containing the parameter with a stated posterior probability, e.g. 95%.