Unit 9: Hypothesis Testing

I. Foundations of Statistical Hypotheses

Hypothesis testing is a formal inferential method for using sample evidence to evaluate a claim about a population. It begins with competing statements, measures how compatible the observed data are with a baseline claim, and reaches a decision while controlling the probability of error.

A. Hypothesis definition

A hypothesis is a precise, testable statement about a population parameter, probability distribution, or relationship between variables.

• Statistical hypothesis: A claim expressed in terms of a population quantity such as the mean (\mu), proportion (p), mean difference (\mu_1-\mu_2), or paired mean difference (\mu_d).
• Testability: The claim must imply observable consequences that can be investigated with sample data; for example, “the population mean is 50” is testable as (H_0:\mu=50).
• Null hypothesis ((H_0)): The baseline statement tested directly, normally containing equality:
  • (H_0:\mu=\mu_0)
  • (H_0:p=p_0)
  • (H_0:\mu_1-\mu_2=0)
• Alternative hypothesis ((H_1) or (H_a)): The competing statement supported when evidence against (H_0) is sufficiently strong:
  • Two-tailed: (H_1:\mu\ne\mu_0)
  • Right-tailed: (H_1:\mu>\mu_0)
  • Left-tailed: (H_1:\mu<\mu_0)
• Test statistic: A standardized numerical measure of the distance between the sample result and the value asserted by (H_0).
• Significance level ((\alpha)): The maximum chosen probability of rejecting a true null hypothesis, commonly (0.05) or (0.01).
• Core assumptions:
  • Observations are randomly selected and independent unless the design is explicitly paired.
  • The sampling distribution required by the chosen test is valid.
  • Variables and parameters are defined before examining the result.
• Decision convention: Reject (H_0) when the evidence reaches the specified significance level; otherwise, fail to reject (H_0). Failure to reject does not prove that (H_0) is true.

II. Framework of Hypothesis Testing

Hypothesis testing converts a research claim into statistical hypotheses, selects an appropriate sampling distribution, and evaluates whether the observed result would be unusually extreme if the null hypothesis were true.

A. Hypothesis testing process

The hypothesis testing process follows a planned sequence from formulation of claims to contextual interpretation.

• Step 1: Identify the parameter: Specify whether the claim concerns (\mu), (p), (\mu_1-\mu_2), or (\mu_d).
• Step 2: State the hypotheses: Formulate mutually exclusive (H_0) and (H_1); equality belongs in (H_0).
• Step 3: Set (\alpha): Choose the Type I error probability before analyzing the data, such as (\alpha=0.05).
• Step 4: Select the test: Use the parameter, number of samples, dependence structure, sample size, and availability of population variance to choose a (z)- or (t)-test.
• Step 5: Check assumptions: Examine randomization, independence, normality, sample size, outliers, and equal variance where applicable.
• Step 6: Calculate the statistic:

test statistic = (sample estimate - null value) / standard error

• Step 7: Determine statistical significance:
  1. p-value method: Reject (H_0) if (p\text{-value}\le\alpha).
  2. Critical-value method: Reject (H_0) if the statistic lies in the rejection region.
• Step 8: Conclude in context: State whether evidence supports the alternative claim, including the population and variable concerned.
• Confidence-interval link: In a two-tailed test at level (\alpha), reject (H_0:\theta=\theta_0) when the corresponding (100(1-\alpha)\%) confidence interval excludes (\theta_0).

B. Types of hypotheses

Hypotheses are classified by their role, precision, direction, and number of parameters.

• Null and alternative hypotheses: (H_0) represents the reference condition, while (H_1) represents the difference or effect sought by the research claim.
• Simple hypothesis: Specifies one exact population distribution or parameter value, such as (H_0:\mu=100).
• Composite hypothesis: Allows multiple parameter values, such as (H_1:\mu>100).
• Directional hypothesis: Predicts the direction of an effect:
  1. Right-tailed: (H_1:\mu>\mu_0); large positive statistics oppose (H_0).
  2. Left-tailed: (H_1:\mu<\mu_0); large negative statistics oppose (H_0).
• Non-directional hypothesis: Predicts a difference without specifying direction, (H_1:\mu\ne\mu_0); both tails contain rejection regions.
• Research hypothesis: States the expected substantive relationship in words, such as “the intervention changes mean blood pressure.”
• Statistical hypothesis: Translates the research expectation into parameter notation, such as (H1:\mu{\text{after}}\ne\mu_{\text{before}}).
• Type I error: Rejecting a true (H_0), with probability (\alpha).
• Type II error: Failing to reject a false (H_0), with probability (\beta).
• Power: The probability of detecting a specified real effect, (1-\beta); it generally increases with sample size, effect size, and (\alpha).

III. Tests Based on the Student’s (t)-Distribution

A (t)-test evaluates hypotheses about means when the population standard deviation is unknown. Replacing (\sigma) with the sample standard deviation (s) introduces additional uncertainty represented by the heavier-tailed Student’s (t)-distribution.

A. Hypothesis testing procedure for t-tests

The (t)-testing procedure standardizes a sample mean or mean difference using an estimated standard error and appropriate degrees of freedom.

• Selection rule: Use a one-sample, independent-samples, or paired-samples (t)-test according to the study design.
• One-sample statistic:

t = (x̄ - μ₀) / (s / √n)
df = n - 1

• (x̄) = sample mean
• (\mu_0) = null-hypothesized population mean
• (s) = sample standard deviation
• (n) = sample size
• (df) = degrees of freedom

• Independent-samples Welch statistic:

t = [(x̄₁ - x̄₂) - Δ₀] / √(s₁²/n₁ + s₂²/n₂)

• (x̄_1,x̄_2) = sample means
• (s_1,s_2) = sample standard deviations
• (n_1,n_2) = sample sizes
• (\Delta_0) = hypothesized mean difference, usually zero
  • Welch degrees of freedom: Software normally computes the Welch–Satterthwaite approximation; it does not require equal population variances.
  • Paired statistic:

t = (d̄ - μd,0) / (sd / √n)
df = n - 1

• (d̄) = mean of within-pair differences
• (\mu_{d,0}) = hypothesized mean difference
• (s_d) = standard deviation of differences
  • Assumptions: The dependent variable is quantitative, observations or pairs are independent, and the relevant population is approximately normal for small samples.
  • Decision: Compare the calculated (t) with critical (t)-values or use its p-value; reject (H_0) when the chosen criterion is met.
  • Interpretation: Statistical significance should be accompanied by the estimated difference and a confidence interval because significance alone does not measure practical importance.

IV. Tests Based on the Standard Normal Distribution

A (z)-test uses the standard normal distribution, whose mean is (0) and standard deviation is (1). It is appropriate when the standard error is known or justified by a large-sample approximation.

A. Hypothesis testing procedure for z-tests

The (z)-testing procedure expresses the observed estimate as a number of standard errors from its null-hypothesized value.

• One-mean statistic:

z = (x̄ - μ₀) / (σ / √n)

• (\sigma) = known population standard deviation; other symbols retain their earlier meanings.
  • One-proportion statistic:

z = (p̂ - p₀) / √[p₀(1 - p₀)/n]

• (\hat p) = sample proportion
• (p_0) = null-hypothesized population proportion
  • Two-mean statistic with known variances:

z = [(x̄₁ - x̄₂) - Δ₀] / √(σ₁²/n₁ + σ₂²/n₂)

• (\sigma_1,\sigma_2) = known population standard deviations.
  • Critical values: For (\alpha=0.05), a two-tailed test rejects when (|z|>1.96); a one-tailed test rejects beyond (z=1.645) in the specified direction.
  • Procedure: State (H_0) and (H_1), choose (\alpha), verify normal or large-sample conditions, calculate (z), obtain the p-value, and conclude in context.
  • Distinction from the (t)-test: The (z)-test uses normal critical values; the (t)-test accounts for estimating (\sigma) and converges toward the (z)-test as degrees of freedom increase.

V. Population and Sample Comparison Designs

The form of the hypothesis and standard error depends on whether the study compares one population with a reference, two independent populations, or linked observations.

A. Differences for a single population

A single-population test determines whether one population parameter differs from a specified benchmark.

• Mean claim: Test (H_0:\mu=\mu_0) using a one-sample (t)-test when (\sigma) is unknown or a (z)-test when (\sigma) is known.
• Proportion claim: Test (H_0:p=p_0) with a one-proportion (z)-test when (np_0) and (n(1-p_0)) are sufficiently large for normal approximation.
• Difference measured: The numerator is estimate minus benchmark, such as (x̄-\mu_0); the standard error determines whether that difference is large relative to sampling variability.
• Example: If (x̄=52), (\mu_0=50), (s=8), and (n=64), then:

t = (52 - 50) / (8/√64) = 2.00
df = 63

The two-tailed p-value is approximately (0.050), so the precise unrounded p-value and stated (\alpha) determine the decision.

B. Differences for two populations

A two-population test evaluates whether parameters from two independent groups differ by a specified amount.

• Hypotheses: A common formulation is (H_0:\mu_1-\mu_2=0) against (H_1:\mu_1-\mu_2\ne0).
• Independence: Membership in one group must not determine or duplicate membership in the other; separate treatment and control groups are a typical design.
• Mean comparison: Welch’s independent-samples (t)-test is generally appropriate when variances are unknown because it permits unequal variances and sample sizes.
• Pooled alternative: A pooled (t)-test assumes (\sigma_1^2=\sigma_2^2); this stronger condition should be substantively justified.
• Proportion comparison: For (H_0:p_1=p_2), a two-proportion (z)-test uses a pooled proportion to calculate the null standard error.
• Interpretation: Rejecting (H_0) supports a population difference, but causal interpretation additionally requires an appropriate randomized or otherwise credible research design.

C. Differences for paired samples

A paired-samples test evaluates the average within-pair difference rather than treating two related sets of measurements as independent.

• Pairing structures: Common designs include before-and-after measurements on the same individuals, matched participants, or measurements on linked units.
• Transformation: Calculate one difference for each pair, (di=X{i1}-X_{i2}), and test (H_0:\mu_d=0).
• Unit of analysis: The effective sample size is the number of complete pairs, not the total number of individual measurements.
• Assumptions: Pairs are independent of other pairs, observations are quantitative, and the distribution of differences is approximately normal for small samples.
• Advantage: Pairing controls stable within-person or within-pair variation, often reducing the standard error and increasing power.
• Limitation: Missing one observation can make a pair incomplete, while poor matching may provide little precision benefit.
• Essential distinction: An independent-samples test analyzes variation between separate groups; a paired (t)-test analyzes the single derived sample (d_1,d_2,\ldots,d_n).