Unit 3: Mathematical Expectations

ECAP790 1 min read

I. Foundations — Probability-Weighted Description

Mathematical expectation summarizes a random quantity by weighting each possible value according to its probability. For continuous random variables, probabilities and expectations are obtained through integration with respect to a probability density or, more generally, a distribution function.

Probability model: A random variable (X) assigns a numerical value to each outcome in a sample space.
Continuous convention: If (X) has probability density function (PDF) (f_X(x)), then (f_X(x)\ge 0) and
TEXT
```
  ∫_{-∞}^{∞} f_X(x) dx = 1
```
where (f_X(x)) is the density of (X).
Probability of an interval:
TEXT
```
  P(a < X ≤ b) = ∫_a^b f_X(x) dx
```
where (a) and (b) are real numbers with (a<b).
Moment convention: The (r)th raw moment is (E[X^r]); the (r)th central moment is (E[(X-\mu)^r]), where (\mu=E[X]).
Existence condition: An expectation is finite when the relevant absolute integral converges; for example, (E[X]) is finite if (E[|X|]<\infty).

II. Mathematical Expectation — Probability-Weighted Averages

Expectation is the long-run average value predicted by a probability model, provided the necessary sum or integral exists.

A. Definition of mathematical expectation

Mathematical expectation is the probability-weighted mean of the possible values of a random variable.

Discrete definition: For a discrete random variable (X) with probability mass function (p_X(x)),
TEXT
```
  E[X] = Σ_x x p_X(x)
```
where the sum extends over all possible values (x) of (X).
Continuous definition: For a continuous random variable with density (f_X(x)),
TEXT
```
  E[X] = ∫_{-∞}^{∞} x f_X(x) dx
```
where the improper integral must be absolutely convergent for a finite expectation.
Interpretation: If independent repetitions produce values (X_1,\ldots,X_n), the sample average tends toward (E[X]) under the law of large numbers.
Not necessarily attainable: The expectation need not be a possible observed value; a fair die has expectation (3.5), although (3.5) cannot appear on one roll.

B. Expected value of a random variable

The expected value identifies the balance point or probability center of a random variable’s distribution.

Notation: (E[X]), (\operatorname{E}(X)), and (\mu_X) commonly denote the mean of (X).
Weighted balance: Values in high-density regions contribute more to the integral than values in low-density regions.
Units: (E[X]) has the same physical unit as (X); if (X) is measured in kilograms, its expectation is also in kilograms.
Example: If (X) is uniformly distributed on ([0,4]), then (f_X(x)=1/4) on that interval:
TEXT
```
  E[X] = ∫_0^4 x(1/4) dx
       = (1/4)[x²/2]_0^4
       = 2
```
Thus the distribution balances at (2).

C. Expected value of a function of a random variable

The expectation of (g(X)) can be calculated directly from the distribution of (X), without first finding the distribution of (g(X)).

Law of the unconscious statistician: For a continuous (X),
TEXT
```
  E[g(X)] = ∫_{-∞}^{∞} g(x)f_X(x) dx
```
where (g) is a measurable function and the integral exists.
Discrete counterpart:
TEXT
```
  E[g(X)] = Σ_x g(x)p_X(x)
```
Moment application: Taking (g(x)=x^r) gives the (r)th raw moment (E[X^r]).
Example: For (X\sim U(0,2)), (f_X(x)=1/2), so
TEXT
```
  E[X²] = ∫_0^2 x²(1/2) dx = 4/3
```
Here (g(x)=x^2), and no separate density for (X^2) is required.

D. Properties of expectations

Expectation is a linear operator, making sums and affine transformations especially simple to analyze.

Constant: For any constant (c),
TEXT
```
  E[c] = c
```
Scaling and translation:
TEXT
```
  E[aX+b] = aE[X]+b
```
where (a) and (b) are constants.
Linearity for sums:
TEXT
```
  E[aX+bY] = aE[X]+bE[Y]
```
where independence of (X) and (Y) is not required.
Products under independence: If (X) and (Y) are independent and their expectations exist, then
TEXT
```
  E[XY] = E[X]E[Y]
```
Order preservation: If (X\le Y) almost surely, then (E[X]\le E[Y]).
Non-negativity: If (X\ge0) almost surely, then (E[X]\ge0).
Indicator expectation: For an event (A), (E[I_A]=P(A)), where (I_A=1) when (A) occurs and (0) otherwise.

III. Continuous Distributions and Descriptive Measures — Shape, Center, and Spread

A continuous distribution is described by cumulative probability and summarized through measures of location, dispersion, asymmetry, and tail behavior.

A. Continuous distribution function

A continuous distribution function records the probability that a continuous random variable does not exceed a specified value.

Cumulative distribution function (CDF):
TEXT
```
  F_X(x) = P(X ≤ x)
```
where (F_X) is the CDF of (X).
Density relationship: For an absolutely continuous distribution,
TEXT
```
  F_X(x) = ∫_{-∞}^x f_X(t) dt
```
where (t) is a dummy integration variable.
Recovery of density: At points of differentiability,
TEXT
```
  f_X(x) = dF_X(x)/dx
```
Core properties:
- (F_X(x)) is non-decreasing.
- (0\le F_X(x)\le1).
- (\lim_{x\to-\infty}FX(x)=0) and (\lim{x\to\infty}F_X(x)=1).
- Every CDF is right-continuous; an absolutely continuous CDF is continuous.
Interval probability:
TEXT
```
  P(a < X ≤ b) = F_X(b)-F_X(a)
```
Point probability: For a continuous random variable, (P(X=x)=0), even though (f_X(x)) may be positive.

B. Measures of Central Tendency

Measures of central tendency identify a representative or central location of a continuous distribution.

Mean:
- Definition: The arithmetic center is (\mu=E[X]).
- Feature: It uses every distribution value but is sensitive to extreme tails.
Median:
- Definition: A median (m) satisfies
  TEXT
```
     P(X ≤ m) ≥ 1/2 and P(X ≥ m) ≥ 1/2
```
- Continuous case: When the CDF is strictly increasing, (F_X(m)=0.5).
- Feature: It is resistant to extreme values and skewed tails.
Mode:
- Definition: A mode (x_m) is a point at which (f_X(x)) reaches a maximum.
- Feature: A distribution may be unimodal, multimodal, or have no unique mode.

Symmetric unimodal case: For distributions such as the normal distribution, mean, median, and mode coincide.

C. Dispersion

Dispersion measures the extent to which values are spread around a center.

Range: The difference between the upper and lower endpoints of bounded support; it depends only on extremes.
Interquartile range:
TEXT
```
  IQR = Q₃-Q₁
```
where (Q_1) and (Q_3) satisfy (F_X(Q_1)=0.25) and (F_X(Q_3)=0.75), when the quantiles are unique.
Mean absolute deviation:
TEXT
```
  E[|X-c|]
```
where (c) is a chosen center, commonly the mean or median.
Standard deviation:
TEXT
```
  σ = √Var(X)
```
where (\sigma) has the same unit as (X).
Coefficient of variation:
TEXT
```
  CV = σ/|μ|
```
where (\mu\ne0); this dimensionless ratio supports comparisons between positive-scale variables measured in different units.

D. Skewness and Kurtosis for continuous probability distribution

Skewness measures asymmetry, while kurtosis measures standardized fourth-moment behavior associated with tails and extreme deviations.

Skewness:
- Definition:
  TEXT
```
     γ₁ = E[(X-μ)³]/σ³
```
  where (\gamma_1) is skewness, (\mu=E[X]), and (\sigma>0) is standard deviation.
- Interpretation: (\gamma_1>0) indicates a longer right tail; (\gamma_1<0) indicates a longer left tail; (\gamma_1=0) indicates zero moment-skewness, not necessarily symmetry.
Kurtosis:
- Definition:
  TEXT
```
     β₂ = E[(X-μ)⁴]/σ⁴
```
  where (\beta_2) is kurtosis.
- Excess kurtosis:
  TEXT
```
     γ₂ = β₂-3
```
  where the normal distribution has (\beta_2=3) and (\gamma_2=0).
- Interpretation: Higher kurtosis generally reflects greater influence from extreme standardized deviations; it should not be interpreted only as peak height.
- Existence: These coefficients require finite third and fourth central moments, respectively.

IV. Second-Order Relationships — Variability and Joint Movement

Second-order moments quantify squared deviation within one variable and linear co-movement between two variables.

A. Variance

Variance is the expected squared distance of a random variable from its mean.

Definition:
TEXT
```
  Var(X) = E[(X-μ)²]
```
where (\mu=E[X]).
Computational form:
TEXT
```
  Var(X) = E[X²]-(E[X])²
```
obtained by expanding ((X-\mu)^2) and applying linearity.

Continuous form:

TEXT

  Var(X) = ∫_{-∞}^{∞}(x-μ)²f_X(x) dx

Units: If (X) is measured in meters, variance is measured in square meters; standard deviation restores the original unit.
Existence: A finite variance requires (E[X^2]<\infty).

B. Properties of variance

Variance is non-negative, translation-invariant, and scales quadratically.

Non-negativity: (\operatorname{Var}(X)\ge0) because squared deviations cannot be negative.
Zero variance: (\operatorname{Var}(X)=0) exactly when (X) is constant almost surely.
Affine transformation:
TEXT
```
  Var(aX+b) = a²Var(X)
```
where translation by (b) has no effect and scaling by (a) multiplies variance by (a^2).
Sum of variables:
TEXT
```
  Var(X+Y) = Var(X)+Var(Y)+2Cov(X,Y)
```
Difference of variables:
TEXT
```
  Var(X-Y) = Var(X)+Var(Y)-2Cov(X,Y)
```
Independent sum: If (X) and (Y) are independent, their covariance is zero, so variances add.

C. Covariance

Covariance measures the direction and magnitude of linear co-movement between two random variables.

Definition:
TEXT
```
  Cov(X,Y) = E[(X-μ_X)(Y-μ_Y)]
```
where (\mu_X=E[X]) and (\mu_Y=E[Y]).
Computational form:
TEXT
```
  Cov(X,Y) = E[XY]-E[X]E[Y]
```
Sign:
- Positive covariance indicates that larger values of (X) tend to accompany larger values of (Y).
- Negative covariance indicates an opposing linear tendency.
- Zero covariance indicates no linear association, but not necessarily independence.
Symmetry and scaling:
TEXT
```
  Cov(X,Y) = Cov(Y,X)
  Cov(aX+b,cY+d) = ac Cov(X,Y)
```
where (a,b,c,d) are constants.
Connection to variance: (\operatorname{Cov}(X,X)=\operatorname{Var}(X)).
Independence: Independence implies zero covariance when the moments exist; zero covariance does not generally imply independence.
Standardization:
TEXT
```
  ρ_XY = Cov(X,Y)/(σ_Xσ_Y)
```
where (\rho_{XY}) is correlation and (\sigma_X,\sigmaY>0); consequently, (-1\le\rho{XY}\le1).