Unit 4: Hypothesis Testing and Statistical Inferences - Subjective Questions

Statistical hypothesis: A statistical hypothesis is a testable statement about a population parameter or probability distribution based on sample data.

• Null hypothesis (): It states that there is no significant difference, relationship, or effect in the population. It is the hypothesis tested directly.
• Alternative hypothesis ( or ): It states that a significant difference, relationship, or effect exists.

Example: Suppose a researcher wants to test whether a new teaching method improves student performance.

• : The new teaching method does not improve performance, .
• : The new teaching method improves performance, .

The statistical test determines whether the sample evidence is strong enough to reject in favor of .

The main steps in hypothesis testing are:

1. State the hypotheses: Formulate the null hypothesis and alternative hypothesis .
2. Select the level of significance: Choose , commonly or .
3. Choose a suitable test statistic: The choice depends on the sample size, data type, population distribution, and whether the population variance is known.
4. Determine the critical region or p-value: Identify the values of the test statistic that lead to rejection of .
5. Calculate the test statistic: Use the sample data in the appropriate formula.
6. Make a decision: Reject if the calculated value falls in the critical region or if the p-value is less than .
7. State the conclusion: Interpret the statistical decision in the context of the research problem.

A statistical decision does not prove that a hypothesis is absolutely true; it indicates whether the sample provides sufficient evidence against .

Type I error: This occurs when the null hypothesis is rejected even though it is true. Its probability is denoted by and is called the level of significance.

Type II error: This occurs when the null hypothesis is not rejected even though it is false. Its probability is denoted by .

Statistical power: The power of a test is the probability of correctly rejecting a false null hypothesis:

Important points include:

• Reducing generally makes it more difficult to reject .
• Increasing the sample size usually reduces and increases power.
• A more variable population makes it harder to detect a true difference.
• The effect size also influences power; larger effects are easier to identify.

Thus, hypothesis testing involves balancing the risks of Type I and Type II errors.

The form of the alternative hypothesis determines whether a test is one-tailed or two-tailed.

• Right-tailed test: Used when the alternative hypothesis states that a parameter is greater than a specified value, such as . The rejection region is in the upper tail.
• Left-tailed test: Used when . The rejection region is in the lower tail.
• Two-tailed test: Used when . The rejection regions are located in both tails.

For a significance level of :

• A one-tailed standard normal test has a critical value of approximately or .
• A two-tailed test divides the significance level equally between the two tails and has critical values of approximately .

The rejection region must be decided before examining the sample results.

Student's -test is a parametric test used to compare means when the population standard deviation is unknown, especially with small samples.

Assumptions:

• The observations are independent.
• The dependent variable is measured on an interval or ratio scale.
• The population is approximately normally distributed.
• For the independent two-sample test, the population variances are usually assumed to be equal unless a version for unequal variances is used.

Applications:

• Testing whether one sample mean differs from a specified population mean.
• Comparing the means of two independent groups.
• Comparing the means of two related or paired observations.

Major types:

1. One-sample -test.
2. Independent-samples -test.
3. Paired-samples -test.

The calculated value is compared with the tabulated value using the appropriate degrees of freedom.

Suppose a sample of size has mean and sample standard deviation . The population mean to be tested is .

The hypotheses may be written as:

Since the population standard deviation is unknown, the standard error of the sample mean is estimated by:

Therefore, the test statistic is:

Under , this statistic follows Student's distribution with:

Decision rule: Reject when the calculated exceeds the critical value for the chosen significance level and degrees of freedom. Alternatively, reject when the p-value is less than .

This test is appropriate when the population is approximately normal and the population variance is unknown.

The independent-samples -test compares the means of two unrelated groups, such as the average scores of students taught by two different methods. For equal population variances, its test statistic is based on the pooled variance:

where is the pooled standard deviation and the degrees of freedom are .

The paired-samples -test compares two measurements from the same subjects or matched subjects, such as pre-test and post-test scores. Let be each pair's difference. Then:

with .

Distinction:

• Independent test: observations in the two groups are unrelated.
• Paired test: observations are naturally matched or repeated.
• The paired test analyzes within-pair differences and can control for variation between subjects.

The Z-test is a parametric hypothesis test based on the standard normal distribution. It is used to test hypotheses about means or proportions when the sampling distribution can be treated as normal.

For a sample mean with known population standard deviation , the test statistic is:

The Z-test is generally preferred when:

• The population standard deviation is known.
• The sample size is large, commonly .
• The population is normally distributed, or the central limit theorem makes the sampling distribution approximately normal.
• A hypothesis concerns a population proportion or difference between proportions.

The Student's -test is preferred when the population standard deviation is unknown and must be estimated from a small sample. As the degrees of freedom increase, the distribution approaches the standard normal distribution.

Let two independent samples have means and , sizes and , and known population standard deviations and . To test:

The expected difference under is . The standard error of the difference between two independent sample means is:

Thus, the Z statistic is:

Under the null hypothesis, follows the standard normal distribution. The calculated value is compared with the appropriate critical value according to the direction of the alternative hypothesis. The null hypothesis is rejected when the statistic lies in the rejection region.

The F-test is a parametric test based on the ratio of two independent estimates of variance. If and are sample variances, the statistic is commonly written as:

Usually, the larger sample variance is placed in the numerator so that .

Main applications:

• Testing whether two population variances are equal.
• Testing the overall significance of a regression model.
• Comparing several population means through analysis of variance, or ANOVA.

The F distribution depends on two degrees of freedom:

• Numerator degrees of freedom: .
• Denominator degrees of freedom: .

Assumptions: The observations should be independent, the populations should be normally distributed, and the variance comparison requires random samples. The F distribution is nonnegative and generally right-skewed, although its shape changes with the degrees of freedom.

To test whether two population variances are equal, formulate:

The procedure is:

1. Obtain independent random samples and calculate and .
2. Place the larger sample variance in the numerator:

3. Determine the degrees of freedom:

4. Select the significance level .
5. Compare the calculated F value with the critical F value from the F table.
6. Reject if the calculated value falls in the rejection region.

The test assumes that both populations are normally distributed. It is sensitive to departures from normality, so alternative robust tests may be considered when this assumption is doubtful.

Parametric tests make assumptions about the population distribution and usually involve parameters such as means and variances. Examples include the Z-test, Student's -test, and F-test.

Nonparametric tests do not require a specific population distribution and often use ranks or frequencies. Examples include the chi-square test and Kruskal-Wallis test.

Comparison:

• Assumptions: Parametric tests usually require normality and may require homogeneity of variance. Nonparametric tests require fewer distributional assumptions.
• Measurement scales: Parametric tests are generally suitable for interval or ratio data. Nonparametric tests can be used for ordinal or categorical data.
• Power: Parametric tests are often more powerful when their assumptions are satisfied.
• Robustness: Nonparametric tests are more suitable for skewed data, outliers, and small samples.
• Interpretation: Parametric tests commonly compare means, whereas nonparametric tests may compare distributions, ranks, or frequencies.

The choice should be based on the research question, data scale, sample characteristics, and validity of assumptions.

The chi-square test of independence determines whether two categorical variables are statistically associated.

The hypotheses are:

• : The two categorical variables are independent.
• : The two categorical variables are associated.

For each cell of a contingency table, the expected frequency is calculated as:

The test statistic is:

where is the observed frequency and is the expected frequency.

For an table, the degrees of freedom are:

Reject when the calculated chi-square value exceeds the critical value or when the p-value is less than . The test requires independent observations and sufficiently large expected frequencies.

Both procedures use the chi-square distribution, but they address different research questions.

Chi-square goodness-of-fit test:

• Examines whether the observed frequencies of one categorical variable agree with specified or theoretical proportions.
• Example: Testing whether a die is fair.
• Hypotheses compare observed frequencies with expected frequencies based on a stated distribution.
• Degrees of freedom are commonly:

where is the number of categories, with adjustments when parameters are estimated.

Chi-square test of independence:

• Examines whether two categorical variables are associated.
• Example: Testing whether educational level is related to voting preference.
• Expected frequencies are calculated from row and column totals.
• Degrees of freedom are:

The goodness-of-fit test uses one variable and a theoretical distribution, whereas the independence test uses two variables and a contingency table.

The main assumptions of the chi-square test are:

• The data are frequency counts, not percentages or continuous measurements.
• Each observation belongs to only one category.
• Observations are independent of one another.
• Categories are mutually exclusive and collectively exhaustive.
• Expected frequencies are sufficiently large. A common guideline is that expected frequencies should generally be at least 5, although exact rules may vary.

Limitations:

• The test can be influenced by sample size; very large samples may produce significance for small practical differences.
• It does not measure the strength or direction of an association by itself.
• It cannot establish causation.
• Results may be unreliable when expected frequencies are very small.
• Grouping continuous data into categories can cause loss of information.

When assumptions are violated, categories may be combined, an exact test may be used, or another suitable method may be selected.

The Kruskal-Wallis test is a nonparametric method used to compare three or more independent groups. It is an alternative to one-way ANOVA when normality or equal-variance assumptions are not satisfied, or when the data are ordinal.

The hypotheses are:

• : The populations have the same distribution, often interpreted as having equal medians when the distributions have similar shapes.
• : At least one population differs from another.

The procedure is:

1. Combine all observations from the groups.
2. Rank the observations from smallest to largest.
3. Calculate the rank sum for each group.
4. Compute the Kruskal-Wallis statistic.
5. Compare the statistic with a chi-square distribution having degrees of freedom, where is the number of groups.

A significant result shows that at least one group differs, but a post hoc analysis is needed to identify which groups differ.

Suppose there are independent groups, with group containing observations. Let be the total number of observations:

After ranking all observations together, let be the sum of ranks for group . The Kruskal-Wallis statistic is:

When tied ranks occur, a tie correction may be applied.

For sufficiently large samples, approximately follows a chi-square distribution with:

Decision procedure:

• State and .
• Rank all observations and calculate .
• Select , such as .
• Reject if exceeds the critical chi-square value or if the p-value is less than .
• If is rejected, conduct an appropriate post hoc multiple-comparison test.

The test assumes independent observations and an ordinal or continuous response variable. For a median interpretation, group distributions should have similar shapes.

One-way ANOVA and the Kruskal-Wallis test both compare three or more independent groups, but they differ in their assumptions and calculations.

One-way ANOVA:

• Parametric test based on group means.
• Uses the F statistic:

• Assumes independent observations, approximately normal errors, and homogeneity of variances.
• Generally has greater power when assumptions are satisfied.

Kruskal-Wallis test:

• Nonparametric test based on ranks rather than raw values.
• Does not require normality and is suitable for ordinal data.
• Is less affected by outliers and skewed distributions.
• May have lower power than ANOVA under ideal parametric conditions.

ANOVA tests equality of means, while Kruskal-Wallis tests whether group distributions differ. When distributions have similar shapes, the latter is often interpreted as a test of equal medians.

A p-value is the probability, assuming that the null hypothesis is true, of obtaining a result at least as extreme as the observed result. It is not the probability that the null hypothesis is true.

Decision using a p-value:

• If , reject .
• If , fail to reject .

A confidence interval gives a range of plausible values for a population parameter at a stated confidence level. For example, a confidence interval represents a procedure that captures the true parameter in of repeated samples.

For a two-sided test at , the null value is rejected when it lies outside the corresponding confidence interval. If the interval contains the null value, the result is generally not statistically significant at the level.

Both tools provide evidence about the population, but practical importance should also be considered.

Statistical significance indicates whether the observed result is unlikely under the null hypothesis at a chosen significance level. It is commonly assessed using a p-value or a confidence interval.

Practical significance considers whether the size of the observed effect is large enough to matter in real-world terms. It may be evaluated using:

• Difference between means or proportions.
• Effect size measures.
• Confidence interval width.
• Costs, benefits, or consequences of the finding.

A very large sample may produce a statistically significant result for a trivial difference. Conversely, a meaningful effect may fail to reach statistical significance in a small or highly variable sample.

Therefore, a complete interpretation should report:

• The estimated effect.
• Its confidence interval.
• The p-value.
• The practical or subject-matter importance of the result.

Statistical significance alone does not establish that a finding is useful or important.