Unit 3: Mathematical Expectations - Subjective Questions

Definition: If is a continuous random variable with probability density function , its mathematical expectation is

The expectation exists as a finite value if

Important points:

• The expectation is the probability-weighted average of all possible values of .
• It represents the long-run average value of over repeated experiments.
• The density must satisfy and .
• If the positive and negative parts of the integral are both infinite, is undefined.

First, verify that the density is normalized:

The expected value is

Therefore,

Hence, the mean or expected value of is

If has density , the expectation of a function is given by the law of the unconscious statistician:

provided the integral exists. It is not necessary to first determine the probability density of .

Here, and on . Thus,

Therefore,

Hence,

Linearity property: For random variables and and constants , , and ,

provided the required expectations exist. Independence of and is not required.

For jointly continuous random variables with joint density ,

Using linearity of integration,

Since the three integrals are , , and , respectively,

Important properties of expectation include:

• Expectation of a constant:
  
• Scaling:
  
• Additivity:
  
  whether or not and are independent.
• Product of independent variables: If and are independent, then
  
• Monotonicity: If almost surely, then
  
• Non-negativity: If almost surely, then .
• Absolute-value inequality:
  

These properties simplify calculations involving sums, transformations, and comparisons of random variables.

• Mean: The arithmetic center or expected value is
  
• Median: A median is a value satisfying
  
  For a continuous distribution with a strictly increasing CDF, .
• Mode: A mode is a value at which the density attains its maximum.

Distinction:

• The mean uses every value and is sensitive to extreme observations.
• The median divides the probability distribution into two equal halves and is resistant to extremes.
• The mode identifies the most densely concentrated value and may be non-unique.
• For a symmetric unimodal distribution, the mean, median, and mode coincide.

If has mean , its variance is defined as

For a continuous random variable,

Expanding the square gives

By linearity of expectation,

Since ,

Therefore,

Let . Its expectation is

where . By the definition of variance,

Substituting gives

Thus,

Hence,

Consequences:

• Adding a constant does not change variance: .
• Multiplying by multiplies the standard deviation by .

The first two raw moments are

and

Using the computational formula,

Therefore,

The standard deviation is

Thus,

The covariance of and measures their joint linear variation. It is defined by

where and .

Expanding the product,

Applying linearity,

Substituting and gives

A positive covariance indicates a tendency to move in the same direction, while a negative covariance indicates a tendency to move in opposite directions.

Let . Using deviations from the respective means,

Therefore,

Expanding and applying expectation,

Thus,

If and are independent, then , so

If and are independent and their expectations exist, then

Consequently,

Thus, independence implies zero covariance.

The converse is generally false: zero covariance means only that there is no linear association. For example, let be symmetric about zero and let . Then

and , so . Nevertheless, is completely determined by , so they are not independent.

Therefore:

• Independence zero covariance.
• Zero covariance independence in general.
• For jointly normal random variables, zero covariance does imply independence.

The cumulative distribution function, or CDF, of a random variable is

If is continuous with density , then

Its main properties are:

• .
• is non-decreasing.
• is right-continuous.
• .
• .
• For ,
  
• If is differentiable, then
  

For a continuous random variable, probabilities at individual points are zero.

The CDF is obtained by integrating the density. For ,

For ,

For ,

Hence,

The required probability is

Thus,

Dispersion describes how widely the values of a distribution are spread around a central value.

Absolute measures retain the units of the variable or their squares:

• Variance: .
• Standard deviation: .
• Mean absolute deviation about : .
• Interquartile range: .

Relative measures are dimensionless and permit comparison between distributions with different units or means. A common example is the coefficient of variation:

provided .

A smaller coefficient of variation indicates greater relative consistency, whereas a larger value indicates greater relative dispersion.

The th raw moment about the origin is

The th central moment about the mean is

Important central moments are:

• .
• .
• measures asymmetry.
• measures tail weight and peakedness.

The moment coefficient of skewness is

The coefficient of kurtosis is

and excess kurtosis is

These standardized measures are dimensionless.

Skewness measures the asymmetry of a probability distribution about its mean. It is commonly measured by

Because the denominator is positive, the sign of is determined by :

• If , then and the distribution is positively skewed, usually with a longer right tail.
• If , then and the distribution is negatively skewed, usually with a longer left tail.
• If , the distribution has zero moment skewness. A symmetric distribution has zero skewness, although zero skewness alone does not always guarantee symmetry.

In many unimodal cases, positive skewness gives mode median mean, with the reverse ordering for negative skewness.

Kurtosis is based on the standardized fourth central moment:

Excess kurtosis is

The categories are:

• Mesokurtic: or . The normal distribution is the standard reference.
• Leptokurtic: or . Such a distribution generally has heavier tails and more extreme observations than the normal reference.
• Platykurtic: or . Such a distribution generally has lighter tails than the normal reference.

Kurtosis is best interpreted as a measure of tail heaviness and susceptibility to extreme values, rather than only as a measure of peak height.

Let . Write

Squaring and taking expectations gives

Since

we obtain

The variance does not depend on , and . Therefore, the expression is minimized uniquely when

The minimum value is

Thus, the mean is the best constant predictor of under squared-error loss.

Use the variance formula

For , and . Therefore,

Hence,

The covariance term is negative here because the coefficients of and have opposite signs. Since the covariance itself is positive, subtracting from reduces the variance. If the covariance were zero, the variance would instead be .