Unit 4: Correlation

I. Orientation

Correlation is a statistical method for describing the direction and strength of association between variables. It originated in the development of quantitative biometrics in the late nineteenth century, especially through the work of Francis Galton and Karl Pearson. Correlation is descriptive: it indicates how variables vary together, but it does not by itself establish that one variable causes another.

• Governing principle: Correlation examines whether changes in one variable are systematically related to changes in another.
• Direction: A positive relationship means that higher scores on one variable tend to accompany higher scores on the other; a negative relationship means that higher scores tend to accompany lower scores.
• Strength: The strength of a relationship is represented by a coefficient, usually ranging from (-1) to (+1).
• Zero correlation: A coefficient of zero indicates no linear association, although a nonlinear association may still exist.
• Causation: Correlation cannot prove causation because of third variables, reverse direction, sampling features, or coincidence.
• Measurement: Variables should generally be measured independently for each participant, and observations should be paired correctly.
• Interpretation: Statistical significance and practical importance are different; a weak association may be significant in a large sample, while a strong sample association may be uncertain in a small sample.

II. Types of Correlation

A. Concept and classification

Types of correlation are distinguished by direction, strength, form, and the number or measurement level of variables involved.

B. types of correlation

Types of correlation provide different ways of describing associations observed in psychological data.

• Positive correlation: Both variables tend to increase or decrease together. For example, study time and examination scores may show a positive association when students who study longer generally score higher.
• Negative correlation: The variables move in opposite directions. For example, stress scores and sleep duration may be negatively related if greater stress is associated with fewer hours of sleep.
• Zero correlation: The variables show no consistent linear tendency. A correlation close to (0.00) does not necessarily mean that the variables are completely unrelated.
• Perfect correlation: A coefficient of (+1.00) or (-1.00) indicates that all observations fall exactly on a straight line. Such values are rare in psychological research.
• Linear correlation: The relationship can be adequately represented by a straight line. Pearson’s coefficient is designed primarily for this form.
• Nonlinear or curvilinear correlation: The variables are related, but the pattern bends. For example, moderate arousal may support performance while very low or very high arousal reduces it; this pattern may have a curved rather than straight-line form.
• Simple correlation: The association is examined between two variables, such as anxiety and test performance.
• Partial correlation: The association between two variables is measured after statistically controlling a third variable. For example, the relationship between screen time and sleep can be examined while controlling age.
• Multiple correlation: One criterion variable is related jointly to two or more predictor variables, such as academic performance predicted from intelligence and motivation.
• Correlation and causality: A correlation between depression and social isolation does not establish whether depression causes isolation, isolation causes depression, or another factor contributes to both.
• Outliers: One unusually high or low observation can substantially change a correlation coefficient. A scatterplot should therefore accompany numerical analysis whenever possible.

III. Pearson’s Correlation Coefficient — Linear Association Between Two Variables

A. Definition and conditions

Pearson’s correlation coefficient, usually written as (r), is a standardized measure of the direction and strength of the linear relationship between two quantitative variables.

B. Pearson's correlation coefficient correlation

Pearson’s coefficient compares the paired deviations of two variables from their respective means.

• Core formula:

r = Σ[(X − X̄)(Y − Ȳ)] / √[Σ(X − X̄)² × Σ(Y − Ȳ)²]

• (X): Score of a participant on the first variable.
• (Y): Score of the same participant on the second variable.
• (X̄): Mean of all (X) scores.
• (Ȳ): Mean of all (Y) scores.
• (\Sigma): Sum across all paired observations.
• (r): Pearson correlation coefficient.

• Interpretation of the sign: A positive (r), such as (+0.72), indicates that high values of (X) tend to accompany high values of (Y). A negative (r), such as (-0.72), indicates that high values of (X) tend to accompany low values of (Y).
• Interpretation of magnitude: The absolute value (|r|) indicates strength. A value near (1.00) indicates a strong linear association, whereas a value near (0.00) indicates a weak linear association.
• Coefficient of determination: Squaring the coefficient gives (r^2), the proportion of variance in one variable statistically associated with the other in a linear model.

r² = proportion of shared linear variance

• Example: If (r = .60), then (r^2 = .36), so 36% of variance is shared linearly. The remaining 64% is not explained by this simple linear association.
  • Assumptions: Pearson’s method is most appropriate when scores are quantitative, observations are independent, the relationship is approximately linear, and extreme outliers are absent.
  • Distributional consideration: For significance testing, bivariate normality is commonly assumed, particularly in small samples. With large samples, moderate departures may be less serious, but severe skew or outliers can still distort (r).
  • Worked example: Suppose higher social-support scores consistently accompany lower loneliness scores, producing (r=-.70). The negative sign describes direction, while (.70) describes a fairly strong linear association; it does not mean that social support necessarily causes reduced loneliness.
  • Significance testing: A sample correlation can be tested against the null hypothesis that the population correlation is zero.

t = r√(n − 2) / √(1 − r²)

• (t): Test statistic.
• (n): Number of paired observations.
• (r): Sample Pearson correlation.
• Degrees of freedom: (n-2).
  • Limitations: Pearson’s (r) can miss curved relationships, can be strongly affected by outliers, and can be misleading when the range of scores is restricted. A statistically significant result should therefore be interpreted with a scatterplot, sample description, and theoretical context.

IV. Multiple Correlation — Joint Prediction from Several Variables

A. Definition and purpose

Multiple correlation measures the association between one criterion variable and the best linear combination of two or more predictor variables. It is commonly used when psychological outcomes are influenced by several related factors.

B. multiple correlation

Multiple correlation evaluates the collective predictive relationship rather than only one pairwise association.

• Notation: The multiple correlation coefficient is written as (R), where the subscript identifies the criterion and predictors. For example, (R_{Y.12}) represents the relationship between criterion (Y) and predictors (X_1) and (X_2).
• Criterion variable: (Y) is the outcome to be predicted, such as depression score or academic achievement.
• Predictor variables: (X_1, X_2,\ldots,X_k) are explanatory variables, such as self-esteem, social support, and coping style.
• Prediction equation:

Ŷ = a + b₁X₁ + b₂X₂ + ... + bₖXₖ

• (\hat{Y}): Predicted value of the criterion.
• (a): Intercept, or predicted value when all predictors equal zero.
• (b_1,\ldots,b_k): Unstandardized regression coefficients.
• (X_1,\ldots,X_k): Predictor scores.
• (k): Number of predictors.
  • Range: (R) ranges from (0) to (1), not from (-1) to (+1), because it describes the strength of the overall prediction; direction is represented by the individual regression coefficients.
  • Coefficient of multiple determination: (R^2) gives the proportion of variance in (Y) explained jointly by all predictors.

R² = explained variance in Y / total variance in Y

• Example: If (R=.70), then (R^2=.49), indicating that the predictor set accounts for 49% of the variance in the criterion in the sample.
  • Unique and shared prediction: A predictor may correlate strongly with (Y) but add little unique information if it overlaps heavily with other predictors. This overlap is examined through semi-partial correlations, regression coefficients, or changes in (R^2).
  • Multicollinearity: Very high correlations among predictors make individual coefficients unstable. For example, two nearly identical anxiety scales may jointly predict well but make it difficult to determine which scale contributes uniquely.
  • Adjusted (R^2): Ordinary (R^2) tends to increase whenever predictors are added, even weak ones. Adjusted (R^2) compensates for sample size and number of predictors and is more suitable for comparing models with different numbers of predictors.
  • Interpretive caution: A high (R) indicates accurate linear prediction in the studied data, not proof that every predictor causes the outcome. Cross-validation or replication is important because an overly complex model may fit one sample by chance.

V. Spearman’s Rank-Order Correlation — Association Based on Ranks

A. Definition and purpose

Spearman’s rank-order correlation, written as (\rho) or (r_s), measures the direction and strength of a monotonic relationship using ranked rather than raw scores.

B. Spearman's rank-order correlation

Spearman’s coefficient is useful when data are ordinal, non-normally distributed, affected by outliers, or related monotonically but not linearly.

• Ranking procedure: Each participant’s scores are converted into ranks separately for the two variables. The smallest score may receive rank 1, and the largest receives rank (n), where (n) is the number of observations.
• Formula without tied ranks:

rₛ = 1 − [6Σd² / n(n² − 1)]

• (r_s): Spearman rank-order correlation.
• (d): Difference between the two ranks for one participant.
• (\Sigma d^2): Sum of squared rank differences.
• (n): Number of paired observations.
  • Meaning of the coefficient: Values range from (-1) to (+1). A positive value indicates that high ranks on one variable tend to accompany high ranks on the other; a negative value indicates opposite rank ordering.
  • Monotonic relationship: The relationship consistently moves in one direction, although it need not change at a constant rate. For example, stress may increase as workload increases, even if the increase becomes smaller at very high workload levels.
  • Tied ranks: When participants have equal scores, tied values receive an average rank, such as ranks 2 and 3 becoming rank 2.5. In this situation, software generally computes the correlation from the ranked values rather than relying mechanically on the no-ties shortcut formula.
  • Worked example: For four participants, suppose the ranks for Variable A are (1,2,3,4) and the ranks for Variable B are (1,3,2,4). The rank differences are (0,-1,1,0), so (\Sigma d^2=2).

rₛ = 1 − [6(2) / 4(4² − 1)]
rₛ = 1 − 12/60
rₛ = .80

• Interpretation: The coefficient (r_s=.80) indicates a strong positive rank-order association.
  • Advantages: Spearman’s method can be used with ordinal ratings, such as ranked therapy preferences, and is less sensitive than Pearson’s method to extreme raw scores because it uses relative order.
  • Limitations: Ranking discards information about exact score differences. A participant scoring 10 and another scoring 20 may receive adjacent ranks even though their raw scores differ substantially. Spearman’s coefficient also does not detect relationships that rise and then fall.
  • Comparison with Pearson: Pearson’s (r) uses raw quantitative distances and targets linear association; Spearman’s (r_s) uses ranks and targets monotonic association. When assumptions for Pearson’s method are satisfied and no influential outliers exist, the two coefficients may be similar, but they need not be identical.