Unit5 - Subjective Questions

Definition:
Correlation is a statistical technique used to determine the degree to which two variables are related or associated. It measures the strength and direction of the linear relationship between two quantitative variables.

Types of Correlation:

1. Positive Correlation:
   • When both variables move in the same direction. If one variable increases, the other also increases, and vice versa.
   • Example: Height and Weight, Income and Expenditure.
2. Negative Correlation:
   • When the variables move in opposite directions. If one variable increases, the other decreases.
   • Example: Price and Demand, Volume and Pressure of a gas.
3. Zero (No) Correlation:
   • When the change in one variable has no relation to the change in the other.
   • Example: Shoe size and Intelligence Quotient (IQ).

Assumptions of Pearson’s Correlation Coefficient ():

1. Level of Measurement: The variables must be measured on an interval or ratio scale (continuous data).
2. Linearity: There should be a linear relationship between the two variables. This means that if you plot the data on a scatter diagram, the points should roughly form a straight line.
3. Normality: Both variables should be normally distributed (bivariate normality).
4. Homoscedasticity: The variability in scores for one variable should be roughly similar at all values of the other variable.
5. Absence of Outliers: Extreme outliers can significantly skew the value of the correlation coefficient and should be removed or analyzed separately.
6. Related Pairs: Each observation in the dataset must consist of a pair of values derived from the same subject.

The most common formula for Pearson's correlation coefficient () is:

Where:

• : Pearson correlation coefficient.
• : Total number of pairs of scores.
• : Sum of the products of paired scores.
• : Sum of scores.
• : Sum of scores.
• : Sum of squared scores.
• : Sum of squared scores.

Alternatively, using deviations from the mean:

Properties of Correlation Coefficient ():

1. Range: The value of always lies between -1 and +1 ().
2. Unit Free: It is a pure number and is independent of the units of measurement (e.g., kg, cm).
3. Symmetry: The correlation between and is the same as the correlation between and ().
4. Independence of Origin and Scale: The value of does not change if the origin (adding/subtracting a constant) or scale (multiplying/dividing by a constant) of the data is changed.
5. Significance:
   • : Perfect positive correlation.
   • : Perfect negative correlation.
   • : No linear correlation.

Definition:
The Coefficient of Determination, denoted as (r-squared), is the square of the Pearson correlation coefficient. It represents the proportion of the variance in the dependent variable that is predictable from the independent variable.

Interpretation:

• It is usually expressed as a percentage.
• For example, if the correlation coefficient , then the coefficient of determination is or .
• Meaning: This implies that 64% of the variation in the dependent variable can be explained by the variation in the independent variable. The remaining 36% is due to other factors (unexplained variance or error).
• It provides a measure of how well the regression model fits the observed data.

Definition:
Spearman’s Rank Correlation Coefficient (denoted by Greek letter or ) is a non-parametric measure of rank correlation. It assesses how well the relationship between two variables can be described using a monotonic function.

When to use it:

1. Ordinal Data: When the data is in the form of ranks/categories (e.g., First, Second, Third) rather than exact values.
2. Non-Normal Distribution: When the data does not satisfy the assumption of normality required for Pearson’s correlation.
3. Non-Linear Monotonic Relationships: When the relationship is consistently increasing or decreasing but not necessarily at a constant rate (curved but not reversing direction).

Formula (No Ties):

Where is the difference between ranks and is the number of pairs.

When two or more items have the same value, they are assigned the average of the ranks they would have occupied. This situation is called 'Tied Ranks'. Since the standard formula assumes distinct ranks, a correction factor () must be applied.

Correction Factor:
For each group of tied ranks, we add to the term, where is the number of times an item is repeated.

Modified Formula:

• : Sum of squared differences of ranks.
• : The number of items in each tied group.
• : Total number of paired observations.

Definition:
A Scatter Diagram (or Scatter Plot) is a graphical representation of the relationship between two quantitative variables. One variable is plotted on the horizontal axis () and the other on the vertical axis ().

Utility in Correlation:

1. Visual Confirmation: It provides an immediate visual clue about the existence of an association.
2. Direction:
   • Rising from left to right: Positive correlation.
   • Falling from left to right: Negative correlation.
3. Strength:
   • Points close to a straight line: Strong correlation.
   • Points scattered widely: Weak correlation.
4. Form: It reveals whether the relationship is linear (straight line) or curvilinear.
5. Outliers: It helps identify extreme values that might skew statistical calculations.

Regression Analysis:
Regression is a statistical method used to estimate the strength and character of the relationship between one dependent variable (usually denoted by ) and one or more independent variables (denoted by ). It allows the researcher to predict the value of the dependent variable based on known values of the independent variable.

Difference between Correlation and Regression:

1. Objective:
   • Correlation: Measures the degree/strength of association.
   • Regression: Predicts the value of one variable based on another.
2. Nature of Variables:
   • Correlation: and are symmetric; there is no distinction between cause and effect.
   • Regression: Defines a clear dependency; is the independent (predictor) variable, and is the dependent (outcome) variable.
3. Causality:
   • Correlation does not imply causation.
   • Regression suggests a functional relationship (though true causality requires experimental design).

Simple Linear Regression:
This form of regression analyzes the relationship between two variables by fitting a linear equation to the observed data. The goal is to minimize the distance between the data points and the regression line.

The Equation:
The mathematical model is represented as:

Where:

• : The dependent variable (variable to be predicted).
• : The independent variable (predictor).
• (Alpha): The Y-intercept (the value of when ).
• (Beta): The Slope of the line (regression coefficient). It indicates the change in for a one-unit change in .
• (Epsilon): The random error term (residual), representing the difference between the observed and predicted values.

Concept:
The Method of Least Squares is a mathematical procedure used to find the best-fitting curve or line for a set of data points. "Best-fitting" is defined as the line that minimizes the sum of the squares of the vertical offsets (residuals) from the points to the line.

Explanation:
If is the predicted value on the line and is the actual observed value:

1. The residual (error) is .
2. We square these errors to avoid positive and negative values cancelling each other out: .
3. The method minimizes the sum: .

This method ensures that the regression line represents the central trend of the data as accurately as possible.

To determine the values of the intercept () and the slope () for the regression line using the method of least squares, we solve the following two simultaneous equations, known as the Normal Equations:

1. First Equation (Sum of Y):
   

2. Second Equation (Sum of XY):
   

Solving these equations:
From these, we calculate the slope () and intercept () as:

Concept:
Regression coefficients quantify the relationship between variables. In simple linear regression, there are two coefficients:

1. : Regression coefficient of on (Change in per unit change in ).
2. : Regression coefficient of on (Change in per unit change in ).

Properties:

1. Geometric Mean: The coefficient of correlation () is the geometric mean of the two regression coefficients.
   
2. Signs: Both regression coefficients must have the same algebraic sign (+ or -). If one is positive, the other must be positive, and will be positive.
3. Magnitude: If one regression coefficient is greater than unity (>1), the other must be less than unity (<1), because their product () cannot exceed 1.
4. Arithmetic Mean: The arithmetic mean of the regression coefficients is greater than or equal to the correlation coefficient: .

In correlation analysis, variables are treated symmetrically. However, in regression, we generate two distinct lines because the objective differs based on which variable is dependent:

1. Regression Line of Y on X ():

   • Here, is the dependent variable and is independent.
   • This line is derived by minimizing the sum of squares of vertical distances (errors in ).
   • Used to predict given .

2. Regression Line of X on Y ():

   • Here, is the dependent variable and is independent.
   • This line is derived by minimizing the sum of squares of horizontal distances (errors in ).
   • Used to predict given .

Unless there is perfect correlation (), these two lines will not coincide. They intersect at the means of the variables .

Definition:
The Standard Error of Estimate ( or ) is a measure of the accuracy of predictions made with a regression line. It represents the standard deviation of the residuals (the differences between observed values and predicted values).

Significance:

• It measures the "average" distance that the observed values fall from the regression line.
• Low : Indicates that the data points are close to the regression line, meaning the prediction is accurate.
• High : Indicates high variability around the line, meaning predictions are less reliable.

Formula:

To ensure the validity of linear regression results, the following assumptions must be met (often remembered by the acronym LINE):

1. Linearity: The relationship between the independent variable(s) and the dependent variable is linear.
2. Independence of Errors: The residuals (errors) are independent of each other (no autocorrelation). This is crucial in time-series data.
3. Normality of Error Distribution: The residuals should be normally distributed with a mean of zero.
4. Homoscedasticity (Constant Variance): The variance of the residuals is constant across all levels of the independent variables. If the variance changes (cone shape in residual plot), it is called heteroscedasticity.
5. No Multicollinearity: (For multiple regression) The independent variables should not be too highly correlated with each other.

Definition:
Spurious correlation (or nonsense correlation) refers to a situation where two variables appear to be statistically correlated but are not causally related. The correlation is often due to coincidence or the presence of a third, unseen factor (confounding variable) that influences both.

Example:

• Scenario: A study finds a high positive correlation between Ice Cream Sales and Drowning Incidents.
• Spuriousness: Does eating ice cream cause drowning? No.
• Explanation: The third variable is Temperature (Summer). In summer, more people buy ice cream, and more people go swimming (increasing the risk of drowning). The two variables are correlated mathematically but have no direct causal link.

To calculate the regression coefficient of on (), we use the relationship between the correlation coefficient () and the standard deviations of and .

Formula:

Given:

• (Standard Deviation of Y) = 12
• (Standard Deviation of X) = 5

Calculation:

Answer:
The regression coefficient is 1.92.

Correlation and Regression are vital tools in business research for decision-making and forecasting.

1. Prediction and Forecasting: Regression allows businesses to predict future trends. For example, predicting Sales based on Advertising Spend.
2. Identifying Key Drivers: Correlation helps identify which factors most strongly affect business outcomes (e.g., relating Employee Satisfaction to Productivity).
3. Risk Management: Beta coefficients in finance (a form of regression) measure the volatility of a stock compared to the market, aiding in portfolio management.
4. Quality Control: analyzing the relationship between production inputs (temperature, speed) and product quality.
5. Marketing Strategy: Understanding the demographics (age, income) that correlate with high purchase intent helps in targeting ads effectively.