Unit 4: Hypothesis Testing and Statistical Inferences

I. Orientation

Hypothesis testing is a formal procedure for using sample evidence to make an inference about a population. It developed from the foundations of probability, sampling theory, and statistical decision-making, especially through the work of R. A. Fisher, Jerzy Neyman, and Egon Pearson during the twentieth century. The central principle is to compare an observed sample result with what would be expected if a stated population assumption were true.

• Population and sample: A population is the complete group under study; a sample is the observed subset, such as 100 patients selected from a hospital population.
• Parameter and statistic: A parameter describes a population, such as population mean (\mu); a statistic describes a sample, such as sample mean (\bar{x}).
• Null hypothesis: The null hypothesis (H_0) usually states no difference, no association, or a specified population value.
• Alternative hypothesis: The alternative hypothesis (H_1) or (H_a) states the effect or difference supported when (H_0) is rejected.
• Significance level: The chosen probability of rejecting a true (H_0) is (\alpha), commonly (0.05) or (0.01).
• Errors: A Type I error rejects a true (H_0); a Type II error fails to reject a false (H_0).
• p-value: The p-value is the probability, assuming (H_0) is true, of obtaining a result at least as extreme as the observed result.
• Decision convention: Reject (H_0) when (p \leq \alpha); otherwise, fail to reject (H_0). “Fail to reject” does not prove that (H_0) is true.

II. Concepts of Hypothesis

A. Definition and Purpose

A hypothesis is a clear, testable statement about a population parameter or relationship between variables. It converts a research expectation into a claim that can be evaluated using sample data.

• Research hypothesis: A directional statement may predict that a new teaching method increases achievement, while a non-directional statement predicts only that the methods differ.
• Statistical form: For a population mean, a null hypothesis may be (H_0:\mu=50), with alternatives (H_1:\mu\neq50), (H_1:\mu>50), or (H_1:\mu<50).
• Testability: Variables must be measurable and the expected relationship must be assessable using an appropriate statistical test.
• Operational clarity: “Stress affects performance” is less testable than “mean examination scores differ between students receiving 10 and 30 minutes of daily relaxation training.”

B. Formulation and Testing Process

Hypothesis testing follows a sequence that links research design, probability, and statistical inference.

• Step 1—State hypotheses: Specify (H_0) and (H_1) before inspecting the result; for example, (H_0:\mu_1=\mu_2).
• Step 2—Select (\alpha): Choose (0.05) when a 5% Type I error rate is acceptable.
• Step 3—Choose the test: The decision depends on scale of measurement, sample size, distribution, and whether population variance is known.
• Step 4—Calculate the statistic: Convert the sample difference into a standardized value such as (t), (z), (F), (\chi^2), or (H).
• Step 5—Make an inference: Compare the statistic with a critical value or interpret its p-value, then report the conclusion in relation to the research question.

III. Parametric Test - Student's t-test

Student’s t-test is a parametric procedure for testing means when the population standard deviation is unknown, particularly with small or moderate samples. It assumes that the observations are quantitative and approximately normally distributed.

A. Parametric Test - Student's t-test

The t-test evaluates whether a sample mean or difference between means is sufficiently large relative to estimated sampling variability.

• One-sample purpose: It tests a sample mean against a hypothesized population mean.

t = (x̄ - μ0) / (s / √n)

• Symbols: (\bar{x}) is the sample mean, (\mu_0) is the hypothesized mean, (s) is sample standard deviation, and (n) is sample size.
  • Independent-samples purpose: It compares means from two unrelated groups, such as treatment and control participants. Under equal variances, the pooled estimate is used:

t = (x̄1 - x̄2) / [sp √(1/n1 + 1/n2)]

• Symbols: (\bar{x}_1,\bar{x}_2) are group means; (n_1,n_2) are group sizes; (s_p) is pooled standard deviation.
  • Paired-samples purpose: It tests the mean of within-person differences, such as blood pressure before and after treatment:

t = d̄ / (sd / √n)

• Symbols: (\bar{d}) is mean difference, (s_d) is standard deviation of differences, and (n) is number of pairs.
  • Degrees of freedom: A one-sample test uses (df=n-1); a paired test also uses (df=n-1). The equal-variance independent test uses (df=n_1+n_2-2).
  • Assumptions: Observations should be independent, the outcome should be approximately interval or ratio scale, and the relevant distribution should be approximately normal. Equal variances are additionally required for the pooled independent test.
  • Interpretation: With (n=10), (\bar{x}=54), (\mu_0=50), and (s=5), (t=(54-50)/(5/\sqrt{10})\approx2.53), which is compared with a t distribution having (9) degrees of freedom.

IV. Parametric Test - Z-test

The Z-test uses the standard normal distribution to test means or proportions when sampling variability is known or sufficiently approximated by large-sample theory.

A. Parametric Test - Z-test

A Z-test standardizes the distance between an observed statistic and its hypothesized value using a known standard error.

• One-mean formula: When population standard deviation (\sigma) is known:

z = (x̄ - μ0) / (σ / √n)

• Symbols: (\bar{x}) is sample mean, (\mu_0) is hypothesized mean, (\sigma) is population standard deviation, and (n) is sample size.
  • Large-sample logic: Even when (\sigma) is not known, a large sample may permit normal approximation, commonly when (n\geq30), provided other conditions are reasonable.
  • One-proportion formula: For a sample proportion (\hat{p}) tested against (p_0):

z = (p̂ - p0) / √[p0(1-p0)/n]

• Symbols: (\hat{p}) is observed proportion, (p_0) is hypothesized proportion, and (n) is sample size.
  • Critical values: For a two-tailed test at (\alpha=0.05), the critical values are approximately (-1.96) and (+1.96); values beyond them lead to rejection of (H_0).
  • Assumptions: Observations should be independent, the measurement should be appropriate for a mean or proportion, and normal approximation conditions should hold. For proportions, both (np_0) and (n(1-p_0)) should generally be at least 5.
  • Distinction from t-test: The Z-test uses a standard normal distribution and known or large-sample standard error; the t-test estimates variability with (s) and accounts for added uncertainty through degrees of freedom.

V. Parametric Test - F-test

The F-test compares variances or evaluates whether several group means differ through analysis of variance. Its statistic is a ratio of two variance estimates.

A. Parametric Test - F-test

The F distribution is nonnegative and depends on two degrees of freedom, making the F-test useful for variance comparisons and ANOVA.

• Variance-comparison formula: For two independent normal populations:

F = s1² / s2²

• Symbols: (s_1^2) and (s_2^2) are sample variances; the larger variance is often placed in the numerator so (F\geq1).
  • ANOVA formula: For (k) groups, the statistic compares between-group variation with within-group variation:

F = MSbetween / MSwithin

• Symbols: (MS{\text{between}}=SS{\text{between}}/(k-1)) and (MS{\text{within}}=SS{\text{within}}/(N-k)); (N) is total sample size.
  • Hypotheses in ANOVA: (H_0:\mu_1=\mu_2=\cdots=\mu_k); (H_1) states that at least one group mean differs.
  • Interpretation: If group means differ greatly while observations within groups are relatively similar, (MS{\text{between}}) exceeds (MS{\text{within}}), producing a large F value.
  • Assumptions: Observations must be independent, residuals should be approximately normal, and group variances should be reasonably homogeneous. Severe outliers can distort both variance terms.
  • Limitation: A significant ANOVA F-test identifies that a difference exists but not which groups differ; post-hoc comparisons are required for that question.

VI. Nonparametric Test - Chi-Square Test

The chi-square test is a nonparametric method for categorical data. It compares observed frequencies with frequencies expected under independence or a specified distribution.

A. Nonparametric Test - Chi-Square Test

The test measures discrepancy between observed and expected counts, rather than differences between numerical means.

• Goodness-of-fit purpose: It tests whether observed category frequencies follow a claimed distribution, such as equal preference for four brands.
• Independence purpose: It tests whether two categorical variables are associated, such as smoking status and disease category in a contingency table.
• Core formula:

χ² = Σ (O - E)² / E

• Symbols: (O) is observed frequency, (E) is expected frequency, and the summation covers all categories or cells.
  • Expected frequency: In an independence table:

Eij = (row total × column total) / grand total

• Symbols: (E_{ij}) is expected count in cell (i,j).
  • Degrees of freedom: Goodness-of-fit commonly uses (df=k-1), where (k) is number of categories. An (r\times c) independence table uses (df=(r-1)(c-1)).
  • Worked example: If a fair four-sided distribution is observed as (30,20,25,25) in 100 trials, expected counts are (25) each and (\chi^2=(5^2/25)+(-5^2/25)+0+0=2), judged using (df=3).
  • Assumptions: Data must be counts of independent cases, categories must be mutually exclusive, and expected frequencies should generally be at least 5. Percentages or means cannot be inserted directly.
  • Interpretation: A significant result indicates departure from the expected pattern or association, but it does not establish causation or identify the strength of association by itself.

VII. Nonparametric Test - Kruskal-Wallis Test

The Kruskal-Wallis test is a rank-based nonparametric alternative to one-way ANOVA for comparing three or more independent groups. It evaluates whether groups come from the same distribution, often interpreted as a difference in central tendency when group shapes are similar.

A. Nonparametric Test - Kruskal-Wallis Test

The method replaces raw scores with ranks across all groups and tests whether rank totals differ more than expected by chance.

• Use case: It is suitable for ordinal outcomes, non-normal quantitative data, or small samples, such as pain ratings from four independent treatment groups.
• Procedure: Combine all observations, assign ranks from 1 upward, calculate each group’s rank sum (R_j), and then compare group rank patterns.
• Test statistic:

H = [12 / (N(N + 1))] Σ(Rj² / nj) - 3(N + 1)

• Symbols: (N) is total observations, (R_j) is rank sum for group (j), and (n_j) is size of group (j).
  • Distribution: For sufficiently adequate samples, (H) is compared with a chi-square distribution having (df=k-1), where (k) is number of groups.
  • Assumptions: Groups must be independent, the outcome must be at least ordinal, observations must be randomly or appropriately sampled, and distributions should have similar shapes if the conclusion is stated specifically about medians.
  • Ties: Equal observations receive average ranks; substantial ties may require a correction to the H statistic.
  • Interpretation: A significant result means at least one group differs in rank distribution. It does not specify the differing pair, so corrected post-hoc pairwise tests are needed.
  • Advantage and limitation: Ranking reduces sensitivity to outliers and non-normality, but it discards magnitude information and may have less power than ANOVA when parametric assumptions are satisfied.