Unit 2: Data analysis - Subjective Questions

Statistical analysis is the process of collecting, organizing, interpreting, and drawing conclusions from data using mathematical and statistical techniques.

Descriptive Statistics:

• Summarizes and describes the main features of a dataset.
• Uses measures such as mean, median, mode, variance, and standard deviation.
• Example: Reporting the average marks of students in a class.

Inferential Statistics:

• Draws conclusions and makes predictions about a population based on a sample.
• Uses techniques like hypothesis testing, confidence intervals, and regression.
• Example: Estimating the average height of all citizens by measuring a sample of people.

Key Difference:

• Descriptive statistics describe what the data shows, while inferential statistics generalize beyond the immediate data to a larger population.

Hypothesis testing is a statistical method used to make decisions or inferences about a population parameter based on sample data.

General Steps:

1. State the hypotheses:

   • Null hypothesis (): assumes no effect or no difference.
   • Alternative hypothesis ( or ): states the presence of an effect or difference.

2. Choose the significance level (): commonly or .

3. Select the appropriate test statistic: e.g., , , , or .

4. Compute the test statistic from the sample data.

5. Determine the p-value or compare with the critical value.

6. Make a decision:

   • If , reject .
   • If , fail to reject .

7. Interpret the result in the context of the problem.

In hypothesis testing, two types of errors can occur when making decisions about .

Type I Error (False Positive):

• Rejecting a true null hypothesis.
• Probability denoted by (significance level).
• Example: Concluding a drug works when it actually does not.

Type II Error (False Negative):

• Failing to reject a false null hypothesis.
• Probability denoted by .
• Example: Concluding a drug does not work when it actually does.

Relationships:

• Significance level: .
• Power of the test: .
• Reducing generally increases (a trade-off).

The p-value is the probability of obtaining a test result at least as extreme as the observed one, assuming the null hypothesis () is true.

Interpretation:

• A small p-value () indicates strong evidence against , so we reject it.
• A large p-value () indicates weak evidence against , so we fail to reject it.
• Formally: .

Significance:

• It quantifies the strength of evidence against the null hypothesis.
• It allows objective, standardized decision-making.

Common Misconceptions:

• The p-value is NOT the probability that is true.
• A small p-value does NOT measure the size or importance of an effect.
• does NOT prove is true; it only means insufficient evidence to reject it.
• Statistical significance is not the same as practical significance.

The chi-square () test is a non-parametric test used for categorical data to assess how well observed frequencies match expected frequencies.

Formula:

where = observed frequency and = expected frequency.

Types of Chi-Square Tests:

1. Goodness-of-Fit Test:

   • Tests whether a sample distribution matches an expected theoretical distribution.
   • Example: Checking if a die is fair.

2. Test of Independence:

   • Tests whether two categorical variables are independent.
   • Uses a contingency table.
   • Expected frequency: .

3. Test of Homogeneity:

   • Tests whether different populations have the same distribution of a categorical variable.

Degrees of Freedom:

• Goodness-of-fit: .
• Independence: .

A large value indicates a significant difference between observed and expected data.

Step 1: State Hypotheses

• : The die is fair (all faces equally likely).
• : The die is not fair.

Step 2: Expected Frequency
For a fair die, for each face.

Step 3: Compute Chi-Square Statistic

Step 4: Degrees of Freedom
.

Step 5: Decision
Critical value . Since , we fail to reject .

Conclusion: There is insufficient evidence to conclude the die is unfair; the die appears to be fair.

A t-test is a parametric statistical test used to determine whether there is a significant difference between the means of groups, especially when the sample size is small and the population standard deviation is unknown.

Types of t-tests:

1. One-Sample t-test:

   • Compares the sample mean to a known population mean.
   • Formula: .
   • Example: Testing if the average weight of a product equals a target value.

2. Independent (Two-Sample) t-test:

   • Compares means of two independent groups.
   • Formula: .
   • Example: Comparing test scores of two different classes.

3. Paired (Dependent) t-test:

   • Compares means from the same group at two different times.
   • Formula: , where is the mean of differences.
   • Example: Measuring blood pressure before and after treatment.

Assumptions: Data is approximately normally distributed, observations are independent, and (for independent t-test) variances are roughly equal.

Step 1: State Hypotheses

• : hours.
• : hours (two-tailed test).

Step 2: Given Data

• Sample mean
• Population mean
• Sample standard deviation
• Sample size

Step 3: Compute Test Statistic

Step 4: Degrees of Freedom
.

Step 5: Decision
Critical value . Since , we reject .

Conclusion: There is sufficient evidence at the level to conclude that the mean lifetime of the bulbs differs from the claimed hours.

ANOVA (Analysis of Variance) is a statistical technique used to compare the means of three or more groups to determine whether at least one group mean differs significantly from the others.

Purpose:

• To test the null hypothesis that all group means are equal: .
• The alternative states that at least one mean is different.
• It analyzes variance between groups relative to variance within groups.

Why ANOVA over multiple t-tests:

• Conducting multiple pairwise t-tests inflates the Type I error rate. For example, with across many comparisons, the chance of a false positive rises well above .
• ANOVA controls the overall (family-wise) error rate with a single test.
• It is more efficient and statistically sound for multi-group comparisons.

Test Statistic:

A large value suggests significant differences among group means. Post-hoc tests (e.g., Tukey's HSD) identify which groups differ.

A one-way ANOVA partitions the total variation in data into components attributable to different sources.

Sum of Squares:

• Total Sum of Squares (SST):
  
• Between-Groups Sum of Squares (SSB):
  
• Within-Groups Sum of Squares (SSW):
  
• Relationship: .

Degrees of Freedom:

• Between:
• Within:
• Total:

(where = number of groups, = total observations)

Mean Squares:

F-Ratio:

ANOVA Table:

If the computed exceeds the critical , we reject .

Bayesian probability interprets probability as a degree of belief in an event, which is updated as new evidence becomes available.

Bayes' Theorem:

where:

• = posterior probability (updated belief in given ).
• = likelihood (probability of given ).
• = prior probability (initial belief in ).
• = evidence (marginal probability of ).

Bayesian vs Frequentist:

Key Idea: The Bayesian approach combines prior beliefs with observed data to form updated (posterior) beliefs, making it powerful for sequential and adaptive decision-making.

Given:

• Prevalence (prior): , so .
• Sensitivity: .
• Specificity: , so false positive rate .

Bayes' Theorem:

Step 1: Numerator

Step 2: Denominator (Total probability of positive)

Step 3: Compute Posterior

Conclusion: Despite the test being accurate, the probability the person actually has the disease is only about 33.2\%. This counterintuitive result occurs because the disease is rare, so false positives from the large healthy population dominate.

Both the z-test and t-test are used to compare means, but they apply under different conditions.

Z-Test:

• Used when the population standard deviation () is known.
• Appropriate for large sample sizes ().
• Based on the standard normal distribution.
• Formula: .

T-Test:

• Used when the population standard deviation is unknown and estimated by the sample standard deviation .
• Appropriate for small sample sizes ().
• Based on the Student's t-distribution, which has heavier tails.
• Formula: .

Comparison Table:

Note: As sample size increases, the t-distribution approaches the normal distribution, and the two tests converge.

A confidence interval (CI) is a range of values, derived from sample data, that is likely to contain the true population parameter with a specified level of confidence.

Construction (for a mean):

or, when is unknown:

where:

• = sample mean
• or = critical value
• = standard error

Interpretation:

• A 95\% confidence interval means that if we repeated the sampling process many times, about of the calculated intervals would contain the true parameter.
• It does NOT mean there is a probability that the true value lies in this specific interval.

Factors Affecting Width:

• Higher confidence level wider interval.
• Larger sample size narrower interval.
• Greater variability wider interval.

Confidence intervals provide more information than a single point estimate by quantifying uncertainty.

The choice between one-tailed and two-tailed tests depends on the directionality of the alternative hypothesis.

One-Tailed Test:

• Tests for a difference in a specific direction (greater than OR less than).
• (right-tailed) or (left-tailed).
• The entire rejection region () is in one tail.
• Example: Testing if a new fertilizer increases crop yield.

Two-Tailed Test:

• Tests for a difference in either direction.
• .
• The rejection region is split into both tails ( each).
• Example: Testing if a machine's output differs from a target value.

Effect on Critical Region and p-value:

• In a one-tailed test, all of is concentrated in one tail, making it easier to detect an effect in that direction (smaller critical value).
• In a two-tailed test, the critical value is larger (more conservative).
• For the same test statistic, the one-tailed p-value is half the two-tailed p-value.

Caution: The direction must be decided before collecting data to avoid bias.

In Bayesian analysis, three key components combine to update our beliefs about a hypothesis or parameter.

1. Prior — :

• Represents our initial belief about a parameter before observing any data.
• Can be based on previous studies, domain expertise, or assumed uninformative.
• Example: Believing a coin is fair, so .

2. Likelihood — :

• The probability of observing the data given a particular value of .
• Reflects how well the parameter explains the observed data.
• Example: Probability of observing heads in flips for a given bias.

3. Posterior — :

• The updated belief about after combining the prior and the data.
• Computed via Bayes' theorem:
  

Updating Process:

• Start with a prior belief.
• Observe data and compute the likelihood.
• Multiply and normalize to obtain the posterior.
• The posterior can serve as the new prior as more data arrives, enabling sequential updating.

ANOVA relies on several key assumptions for its results to be valid.

Assumptions:

1. Independence of observations:

   • Each observation must be independent of the others.
   • Violation (e.g., correlated data) leads to invalid F-tests.

2. Normality:

   • The residuals (or the data within each group) should be approximately normally distributed.
   • Can be checked using Q-Q plots or the Shapiro-Wilk test.

3. Homogeneity of variances (Homoscedasticity):

   • The variances across all groups should be approximately equal.
   • Checked using Levene's test or Bartlett's test.

4. Continuous dependent variable measured on an interval or ratio scale.

If Assumptions Are Violated:

• Non-normality: For large samples, ANOVA is fairly robust (Central Limit Theorem). For small samples, use non-parametric alternatives like the Kruskal-Wallis test.
• Unequal variances: Use Welch's ANOVA instead of standard ANOVA.
• Lack of independence: Requires different models (e.g., repeated-measures ANOVA or mixed models).

Conclusion: Violations can inflate Type I error rates and lead to incorrect conclusions, so assumption checks are essential before interpreting ANOVA results.

Step 1: State Hypotheses

• (all teaching methods equally effective).
• : At least one mean differs.

Step 2: Degrees of Freedom

• Groups , Total observations .

Step 3: Mean Squares

Step 4: F-Ratio

ANOVA Table:

Step 5: Decision
Critical value . Since , we reject .

Conclusion: There is a statistically significant difference among the three teaching methods. Post-hoc tests would identify which methods differ.

Statistical Significance:

• Indicates that an observed effect is unlikely due to random chance.
• Determined by the p-value: if , the result is statistically significant.
• Heavily influenced by sample size — with very large samples, even tiny, trivial effects can become statistically significant.

Practical Significance:

• Refers to whether the effect is large enough to be meaningful in the real world.
• Measured by effect size metrics such as Cohen's , correlation , or odds ratios.
• Considers the real-world context and cost-benefit implications.

Example:

• A study on users finds that a new UI increases click-through rate by with .
• The result is statistically significant but likely practically insignificant — the improvement is too small to matter for business decisions.

Why Consider Both:

• Statistical significance alone can be misleading, especially with large datasets.
• Practical significance ensures decisions are based on meaningful impact, not just mathematical detectability.
• Reporting effect sizes and confidence intervals alongside p-values provides a complete picture.

The chi-square test of independence determines whether two categorical variables are statistically associated or independent.

Hypotheses:

• : The two variables are independent.
• : The two variables are associated (dependent).

Expected Frequency Formula:

Test Statistic:

Degrees of Freedom: .

Example — Gender vs Preference:

Observed contingency table:

Expected values:

• (all cells = 25 by symmetry here).

Compute :

Degrees of Freedom: .

Decision: Critical value . Since , we reject — there is a significant association between gender and product preference.