Unit 4: Moments - Subjective Questions

Raw moment: The th raw moment of about the origin is

Central moment: The th central moment of about its mean is

Distinction:

• Raw moments are measured about the origin, whereas central moments are measured about the mean.
• The first raw moment is the mean: .
• The first central moment is always zero: .
• The second central moment is the variance: .
• Higher central moments describe the shape of a distribution. The third relates to skewness, while the fourth relates to kurtosis.

Let . Using

and expanding by the binomial theorem gives:

• First central moment:
  
• Second central moment:
  
• Third central moment:
  
• Fourth central moment:
  

In general,

where .

Chebyshev's inequality: If has mean and finite variance , then for every ,

Equivalently, for ,

Proof: Define the nonnegative random variable . By Markov's inequality,

Since and

we obtain

Thus, at least of the probability lies within standard deviations of the mean.

The mean and standard deviation are

For the first probability,

Here . Chebyshev's inequality gives

Therefore,

For the second probability, , so

These are distribution-free bounds; no assumption about the form of the distribution is required.

For random variables and , the joint raw moment of order is

The corresponding mixed central moment is

Important special cases include:

• and .
• .
• .
• The mixed central moment of order is
  

In terms of raw moments,

Thus, covariance measures the second-order joint variation of and .

If and are independent and the required moments exist, their joint distribution factorizes. Therefore,

For continuous variables, for example,

Taking gives

and hence . Thus, independence implies zero covariance when second moments exist.

The converse is generally false. For example, let be uniformly distributed on and set . Symmetry gives and , so

However, is completely determined by , so and are not independent.

By linearity of expectation,

Thus, the first raw moment is

For the second raw moment,

Taking expectations,

Therefore,

where . If and are independent, then , and

By linearity of expectation,

For the variance,

Expanding the square gives

Equivalently, if is the coefficient vector and is the covariance matrix,

If the variables are independent, all pairwise covariance terms are zero, so

For , this reduces to the variance formula for a sum.

The moment generating function (MGF) of is

for those values of for which the expectation is finite.

Expanding the exponential,

Under conditions that justify interchanging expectation and summation,

Consequently, the th raw moment is obtained by differentiation:

Important points are:

• .
• An MGF is normally required to be finite on an open interval containing .
• If it exists on such an interval, it uniquely determines the distribution.
• The existence of all ordinary moments alone does not necessarily imply that the MGF exists near .

For ,

Separating the constant term gives

The mean can be found directly or by differentiating at zero:

Similarly,

Therefore,

Thus, adding shifts the mean but does not affect variance, while multiplication by multiplies the variance by .

Let

Then

Because are independent, the transformed variables are also independent. Hence,

Independence is essential because it permits the expectation of the product to be written as the product of expectations. Without independence, the joint dependence structure affects , and the product formula need not hold.

If the variables are independent and identically distributed with MGF , then

The MGF of is

Raw moments are obtained from . Differentiation gives:

and

Therefore, the first four raw moments are

The corresponding third central moment is , and the fourth central moment is .

The cumulant generating function (CGF) is the logarithm of the MGF:

Its series expansion is

where

is the th cumulant. In terms of the mean and central moments:

• First cumulant:
  
• Second cumulant:
  
• Third cumulant:
  
• Fourth cumulant:
  

Thus, the first cumulant is the mean, the second is the variance, and the higher cumulants measure distributional characteristics beyond location and dispersion.

Let and be independent. Since

taking logarithms gives

Differentiating times at yields

More generally, cumulants of a sum of independent variables are the sums of the corresponding cumulants.

For ,

Therefore,

and, for ,

The shift affects only the first cumulant, while scaling affects the th cumulant by the factor .

For , the MGF is

Therefore, the cumulant generating function is

Differentiating,

for every integer . Evaluating at gives

Hence, the first four cumulants are

Thus, the Poisson distribution has mean and variance both equal to . Its third central moment is , while its fourth central moment is

The joint MGF of and is

provided it is finite in a neighborhood of . Mixed raw moments are obtained through partial differentiation:

In particular,

and

Therefore,

If the joint MGF factorizes as near the origin, then and are independent.

A random variable may possess finite moments of every positive integer order while its MGF is infinite for every .

Let , where . Then has a lognormal distribution. Its th raw moment is

which is finite for every positive integer .

However, for ,

diverges because grows too rapidly in the upper tail. Hence, the MGF is not finite on any open interval around .

This demonstrates that:

• Finite moments of all orders do not guarantee MGF existence near zero.
• The formal moment series need not converge to an MGF.
• When the MGF does exist in a neighborhood of zero, it uniquely determines the distribution.

Let , , and denote the second, third, and fourth central moments.

The coefficient of skewness is

Its interpretation is:

• : symmetric in terms of the third central moment.
• : positive or right skewness.
• : negative or left skewness.

The coefficient of kurtosis is

and excess kurtosis is

A normal distribution has and . Since

standardized excess kurtosis can also be written as

Expanding the cube,

Therefore,

Taking expectations gives

If , , and are mutually independent, the mixed moments factorize. Thus,

with analogous expressions for the other terms. Substitution produces the third raw moment entirely in terms of the individual raw moments.

Define the standardized sum

Let the cumulants of each be , , , and . Additivity of cumulants gives cumulant for the unstandardized sum.

Centering and scaling then yield:

and

As , the third and fourth cumulants approach zero, while the first two approach and . These are the cumulants of the standard normal distribution, illustrating the moment-cumulant basis of normal convergence under suitable conditions.