Unit4 - Subjective Questions

Definition of Hypothesis:
A hypothesis is a tentative statement or proposition about the relationship between two or more variables. It serves as a prediction that researchers test through scientific methods. It acts as a guide for the research study.

Difference between Null and Alternative Hypothesis:

1. Null Hypothesis ():

   • It is a statement of 'no effect' or 'no difference'.
   • It assumes that any observed difference in samples is due to random chance.
   • Example: There is no significant difference between the mean height of men and women. ()

2. Alternative Hypothesis ( or ):

   • It is the statement that opposes the null hypothesis.
   • It suggests that the observed difference is real and significant.
   • Example: The mean height of men is different from the mean height of women. ()

In hypothesis testing, errors can occur when making a decision about the Null Hypothesis ().

Type I Error (False Positive):

• Definition: Occurs when we reject a true Null Hypothesis.
• It means concluding there is an effect when there actually isn't.
• Probability: The probability of committing a Type I error is denoted by Alpha (), which is the level of significance (e.g., 0.05).

Type II Error (False Negative):

• Definition: Occurs when we fail to reject (accept) a false Null Hypothesis.
• It means failing to detect an effect that actually exists.
• Probability: The probability of committing a Type II error is denoted by Beta ().

Relationship:

• There is generally an inverse relationship between Type I and Type II errors; reducing (making the test stricter) typically increases (reducing the power to detect an effect), unless the sample size is increased.

The general procedure for hypothesis testing involves the following systematic steps:

1. Set up the Hypothesis:
   • Formulate the Null Hypothesis () and Alternative Hypothesis ().
2. Select the Level of Significance ():
   • Choose a critical probability threshold, typically 5% (0.05) or 1% (0.01).
3. Choose the Appropriate Statistical Test:
   • Decide between Parametric (t-test, z-test, F-test) or Nonparametric (Chi-square, Kruskal-Wallis) based on data distribution and sample size.
4. Compute the Test Statistic:
   • Calculate the value using the sample data and the specific formula for the chosen test.
5. Determine the Critical Region:
   • Find the critical value from statistical tables based on degrees of freedom and .
6. Make a Decision:
   • Compare the calculated value with the table value.
   • If Calculated Value > Table Value, reject .
   • If Calculated Value < Table Value, accept (fail to reject).

Two-tailed Test:

• Definition: A test where the critical region is distributed in both tails of the probability distribution.
• Usage: Used when the hypothesis predicts a difference but does not specify the direction.
• Hypothesis: vs .
• Example: Testing if a new fertilizer changes crop yield (it could increase or decrease it).

One-tailed Test:

• Definition: A test where the critical region is located in only one tail (either left or right) of the distribution.
• Usage: Used when the hypothesis predicts a specific direction of the effect (greater than or less than).
• Hypothesis: vs (Right-tailed) or (Left-tailed).
• Example: Testing if a new fertilizer increases crop yield.

Parametric Tests:
Parametric tests are statistical tests that make assumptions about the parameters of the population distribution from which the sample is drawn. They generally assume that the data follows a specific distribution, usually the Normal Distribution.

Key Assumptions:

1. Normality: The population from which the sample is drawn must be normally distributed.
2. Homogeneity of Variance: The variances of the populations being compared should be approximately equal (Homoscedasticity).
3. Interval or Ratio Scale: The data should be quantitative and measured on an interval or ratio scale.
4. Independence: The observations must be independent of each other.

Examples: z-test, t-test, F-test.

Z-test:
A Z-test is a parametric statistical test used to determine whether two population means are different when the variances are known and the sample size is large.

Formula for One Sample Z-test:

Where:

• = Sample mean
• = Population mean
• = Population standard deviation
• = Sample size

Conditions for Application:

1. Large Sample Size: The sample size should be large, typically .
2. Known Variance: The population standard deviation () or variance must be known.
3. Normality: If is large, the Central Limit Theorem allows the test to be used even if the population is not perfectly normal. If is small, the population must be normal (though a t-test is usually preferred for small ).

Comparison:

Decision Criteria:

1. Use Z-test when: Sample size is large () OR Sample is small but Population Standard Deviation () is known.
2. Use t-test when: Sample size is small () AND Population Standard Deviation () is unknown (Sample SD is used instead).

F-test:
The F-test is a parametric test based on the F-distribution. It is primarily used to compare the variances of two independent samples to test the hypothesis that the populations have the same variance.

Formula:

Where is the larger sample variance and is the smaller sample variance.

Applications:

1. Testing Equality of Variances: Determining if two populations vary at the same level (Homogeneity of Variance).
2. ANOVA (Analysis of Variance): The F-test is the core statistic used in ANOVA to compare means of more than two groups by analyzing the ratio of between-group variance to within-group variance.
3. Regression Analysis: To test the overall significance of a regression model.

Chi-Square () Test:
It is a non-parametric test used to determine if there is a significant association between two categorical variables or if an observed frequency distribution differs from a theoretical distribution.

Formula:

Where:

• = Observed frequency
• = Expected frequency

Conditions for Validity:

1. Random Sampling: Data must be drawn randomly from the population.
2. Independence: Observations should be independent.
3. Large Sample: Total frequency () should be reasonably large (usually ).
4. Frequency Constraints: No expected cell frequency should be less than 5. If it is < 5, adjacent classes must be pooled (Yates' correction may apply for tables).
5. Raw Counts: Data must be in absolute frequencies (counts), not percentages or ratios.

Difference between Parametric and Nonparametric Tests:

1. Assumptions:

   • Parametric: Assumes data follows a specific distribution (usually Normal) and requires homogeneity of variance.
   • Nonparametric: Distribution-free; makes no assumptions about the underlying population distribution.

2. Data Type:

   • Parametric: Requires Interval or Ratio scale data (Quantitative).
   • Nonparametric: Can handle Nominal or Ordinal data (Qualitative/Ranked).

3. Central Tendency:

   • Parametric: Uses the Mean.
   • Nonparametric: Uses the Median.

4. Power:

   • Parametric: Generally more powerful if assumptions are met.
   • Nonparametric: Less powerful than parametric tests if data is actually normal, but more robust if assumptions are violated.

5. Examples:

   • Parametric: t-test, z-test, ANOVA.
   • Nonparametric: Chi-square, Mann-Whitney U, Kruskal-Wallis.

Kruskal-Wallis Test (H-test):
The Kruskal-Wallis test is a rank-based non-parametric test that can be used to determine if there are statistically significant differences between two or more groups of an independent variable on a continuous or ordinal dependent variable.

Usage:
It is considered the non-parametric alternative to the One-Way ANOVA.

When to use:

1. Non-Normality: When the data does not meet the assumption of normality required for ANOVA.
2. Ordinal Data: When the dependent variable is measured on an ordinal scale (ranks) rather than an interval/ratio scale.
3. Small Samples: When sample sizes are small and the distribution is unknown or skewed.
4. Comparison: It compares the medians (ranks) of three or more independent groups.

Chi-Square Goodness of Fit Test:
This application of the Chi-Square test determines how well a set of observed data fits a specific theoretical distribution (such as Normal, Binomial, or Poisson).

Objective:
To test if the discrepancy between Observed Frequencies () and Expected Frequencies () is due to sampling fluctuation or if it is significant.

Steps:

1. Hypothesis:
   • : The data follows the specified distribution (Fit is good).
   • : The data does not follow the specified distribution (Fit is not good).
2. Calculate Expected Frequencies (): Based on the theoretical probability distribution.
3. Calculate :
4. Degrees of Freedom: (where is the number of categories).
5. Conclusion: If , reject , concluding the data does not fit the distribution.

Paired t-test:
A statistical procedure used to determine whether the mean difference between two sets of observations is zero. It is used when the samples are dependent or related.

Scenarios:

• Before and After studies (e.g., weight before and after a diet).
• Matched pairs (e.g., twins, husband-wife).

Difference from Independent t-test:

1. Sample Relationship:
   • Paired: The two samples are related (same subjects measured twice or matched pairs).
   • Independent: The two samples are completely unrelated groups (e.g., Males vs Females).
2. Formula Focus:
   • Paired: Focuses on the difference () between pairs (Mean of differences).
   • Independent: Focuses on the difference between the means of two groups.
3. Degrees of Freedom:
   • Paired: (where is number of pairs).
   • Independent: .

Degrees of Freedom ():
Degrees of freedom refer to the number of independent values or quantities which can be assigned to a statistical distribution. It represents the number of values in the final calculation of a statistic that are free to vary.

Conceptual Explanation:
If you have to select 5 numbers that sum to 50, you have the "freedom" to choose the first 4 numbers randomly. However, the 5th number is fixed by the constraint (Sum = 50). Thus, .

Importance:

• It is crucial for determining the critical values from statistical tables (t-table, -table, F-table).
• As increases, the t-distribution approaches the standard normal (Z) distribution.
• Different tests have different formulas for (e.g., for t-test, for Chi-square independence).

Chi-Square Test of Independence:
Used to test if two categorical variables are independent of each other.

Contingency Table:
Data is arranged in a table with rows and columns.

Calculating Expected Frequency ():
For a cell in row and column , the expected frequency under the assumption of independence is:

Test Statistic:

Degrees of Freedom:

If the calculated exceeds the critical value, we reject the Null Hypothesis (that variables are independent) and conclude there is an association.

Power of a Test ():
The power of a statistical test is the probability that the test correctly rejects a false Null Hypothesis (). It is the ability of the test to detect an effect if that effect actually exists.

Formula:

(Where is the probability of Type II error).

Significance:

1. Reliability: A test with high power is more reliable in detecting significant differences.
2. Sample Size Planning: Researchers perform power analysis to determine the minimum sample size required to detect an effect with a certain degree of confidence.
3. Avoiding False Negatives: Low power increases the risk of Type II errors, meaning a researcher might miss a significant discovery.

Comparison of Statistical Tests:

1. Student's t-test:

   • Purpose: To compare the means of two groups (sample vs population, or two samples).
   • Data Requirement: Interval/Ratio data; Normal distribution; Small sample size (); Unknown population SD.
   • Type: Parametric.

2. F-test:

   • Purpose: To compare variances of two populations or to compare means of more than two groups (ANOVA).
   • Data Requirement: Interval/Ratio data; Normal distribution; Homogeneity of variance.
   • Type: Parametric.

3. Chi-Square () Test:

   • Purpose: To test association between categorical variables or goodness of fit.
   • Data Requirement: Nominal/Categorical data (frequencies/counts); Non-normal data acceptable.
   • Type: Non-parametric.

Level of Significance ():
It is the probability of rejecting the Null Hypothesis when it is actually true (Type I Error). It defines the critical region boundary.

Choosing 1% vs 5%:

1. 5% Level ():

   • Indicates a 5% risk of concluding a difference exists when it doesn't.
   • Outcome: It is easier to reject the Null Hypothesis. Used for general research.

2. 1% Level ():

   • Indicates a stricter standard (99% confidence).
   • Outcome: It is harder to reject the Null Hypothesis. Used in sensitive fields (e.g., medicine) where false positives are dangerous.

Trade-off: Lowering (from 5% to 1%) reduces Type I error but increases the risk of Type II error (missing a real effect).

Assumptions of ANOVA:

1. Normality: Each group sample is drawn from a normally distributed population.
2. Homogeneity of Variance: The variances of the groups are approximately equal.
3. Independence: Observations are independent of each other.

Preference over Multiple t-tests:
If we have 3 groups (A, B, C) and use t-tests, we would need to compare A-B, B-C, and A-C separately.

• Error Inflation: Performing multiple t-tests increases the Family-wise Error Rate (Type I error accumulates). For 3 tests at , the probability of at least one false positive is approx (14%).
• ANOVA Efficiency: ANOVA compares all means simultaneously in a single test, maintaining the Type I error rate at exactly (0.05).

Standard Error (SE):
The Standard Error is the standard deviation of the sampling distribution of a statistic (usually the mean). It measures how much the sample mean is expected to vary from the true population mean.

Formula:

Role in t-test and z-test:
In formulas like:

• The Numerator () represents the observed difference (signal).
• The Denominator (SE) represents the expected variation due to chance (noise).
• Role: It acts as a "measuring stick." The test statistic calculates how many "standard errors" away the sample mean is from the population mean. A larger ratio implies the difference is significant and not just due to random sampling noise.