Unit1 - Subjective Questions

Definition of a Random Variable

A random variable is a numerical description of the outcome of a statistical experiment. It is a function that maps the outcomes from a sample space to a real number. Random variables are typically denoted by capital letters, such as or .

Types of Random Variables

There are two main types of random variables:

1. Discrete Random Variable:

   • Definition: A discrete random variable is a random variable that can take on a finite or countably infinite number of values. These values are typically integers and can be listed.
   • Characteristics: There are gaps between the possible values it can take. It usually results from counting processes.
   • Example: The number of heads when flipping a coin three times (possible values: 0, 1, 2, 3). The number of cars passing a point on a road in an hour.

2. Continuous Random Variable:

   • Definition: A continuous random variable is a random variable that can take on any value within a given range or interval. It can take on an uncountably infinite number of values.
   • Characteristics: Its possible values are typically measurements, such as length, weight, or time. There are no gaps between the possible values.
   • Example: The height of a randomly selected student (e.g., any value between 1.50m and 1.90m). The time it takes for a bus to arrive at a stop (e.g., any value between 0 and 30 minutes).

Definition of Probability Mass Function (PMF)

For a discrete random variable , its Probability Mass Function (PMF), denoted by or , gives the probability that the random variable takes on a specific value . Formally, it is defined as:

where is one of the possible values that can take.

Essential Properties of PMF

The PMF must satisfy the following properties:

1. Non-negativity: The probability for any specific value must be non-negative. That is, for all possible values of :
   

2. Summation to One: The sum of probabilities for all possible values of the discrete random variable must equal 1. This ensures that the PMF covers all possible outcomes in the sample space.
   
   where the sum is taken over all possible values that can assume.

3. Probabilities for specific values: The PMF directly gives the probability for a single outcome. For any value not in the set of possible values for , .

Definition of Probability Density Function (PDF)

For a continuous random variable , its Probability Density Function (PDF), denoted by , is a function such that the probability that falls within a certain interval is given by the integral of over that interval. It does not directly give probabilities for specific values, but rather describes the relative likelihood for the random variable to take on a given value.

Formally, for an interval :

Essential Properties of PDF

The PDF must satisfy the following properties:

1. Non-negativity: The PDF must be non-negative for all possible values of . That is:
   

2. Total Probability: The total area under the entire PDF curve must be equal to 1. This signifies that the probability of the random variable taking any value within its entire range is 1.
   

Why for a Continuous Random Variable?

For a continuous random variable, the probability of taking on any single specific value is always zero. This is because a continuous random variable can take an uncountably infinite number of values within any given interval. If each specific value had a non-zero probability, the sum (integral) over all possible values would become infinite, violating the second property of the PDF.

Mathematically, this can be seen as:

Instead, probabilities are defined over intervals. The height of the PDF curve at a specific point indicates the probability density at that point, not the probability.

Definition of Cumulative Distribution Function (CDF)

The Cumulative Distribution Function (CDF), denoted by , gives the probability that a random variable takes on a value less than or equal to . Formally, for any real number :

For a Discrete Random Variable :

If is a discrete random variable with PMF , its CDF is given by the sum of probabilities for all values less than or equal to :

The CDF for a discrete random variable is a step function.

For a Continuous Random Variable :

If is a continuous random variable with PDF , its CDF is given by the integral of the PDF from negative infinity up to :

The CDF for a continuous random variable is a continuous function.

General Properties of CDF

The CDF, whether for a discrete or continuous random variable, must satisfy the following properties:

1. Non-decreasing: The CDF is a non-decreasing function. If , then . This means that as increases, the cumulative probability either stays the same or increases.

2. Limits at Extremes:

   • As approaches negative infinity, the CDF approaches 0:
     
     This means there is no probability for values below the minimum possible value of .
   • As approaches positive infinity, the CDF approaches 1:
     
     This means that the total probability of the random variable taking any value within its entire range is 1.

3. Right-Continuity: The CDF is right-continuous. That is, . This property is particularly important for discrete random variables, where the CDF "jumps" at each possible value.

4. Probability of an Interval: For any two real numbers , the probability that falls within the interval can be found using the CDF:
   
   For continuous random variables, due to .

Relationship between CDF and PMF for a Discrete Random Variable

Let be a discrete random variable with possible values (or countably infinite) in increasing order, and let be its Probability Mass Function (PMF). The Cumulative Distribution Function (CDF), , is defined as:

This means that the CDF at any point is the sum of the probabilities of all possible values of that are less than or equal to . The CDF for a discrete random variable is a step function, where jumps occur at the values and the height of each jump is equal to .

Conversely, the PMF can be derived from the CDF:

For a specific value , the probability is the jump in the CDF at . This can be expressed as:

where is the largest possible value of strictly less than . If is the smallest possible value, then would be $0$.

Illustration with a Simple Example

Consider a discrete random variable representing the number of heads in two coin flips. The possible outcomes are $0, 1, 2$. The PMF is:

Now, let's derive the CDF :

• For :
• For :
• For :
• For :

So, the CDF is:

We can also derive the PMF from the CDF:

This confirms the relationship.

Relationship between CDF and PDF for a Continuous Random Variable

Let be a continuous random variable with Probability Density Function (PDF) . The Cumulative Distribution Function (CDF), , is defined as the probability that takes a value less than or equal to . This is given by the integral of the PDF from negative infinity up to :

This fundamental relationship shows how the CDF accumulates the probability density over the range of the random variable.

Inverse Relationship: Deriving PDF from CDF

Conversely, the PDF can be obtained from the CDF by differentiation. According to the Fundamental Theorem of Calculus, if is an integral of , then the derivative of with respect to gives :

This means that the PDF represents the rate of change of the cumulative probability at a given point . The height of the PDF curve at signifies the probability density at that point.

Summary:

• To get CDF from PDF: Integrate the PDF.
• To get PDF from CDF: Differentiate the CDF.

This relationship is crucial for working with continuous random variables, as it allows us to move between the density function (which describes the distribution's shape) and the cumulative function (which gives direct probabilities for intervals).

Definition of the -th Moment about the Origin (Raw Moment)

The -th moment about the origin, often denoted by , is a measure of the shape of a probability distribution. It captures information about the average value of .

For a Discrete Random Variable :

If is a discrete random variable with possible values and PMF , the -th moment about the origin is defined as:

For a Continuous Random Variable :

If is a continuous random variable with PDF , the -th moment about the origin is defined as:

Relationship with the Mean

The mean of a random variable, denoted by or , is directly related to the first moment about the origin. Specifically:

The mean of a random variable is its first moment about the origin ().

• For a discrete random variable:
  

• For a continuous random variable:
  

Thus, the mean represents the expected value of the random variable itself, which is a measure of the central tendency of the distribution.

Definition of the -th Central Moment

The -th central moment, often denoted by , measures the shape of a probability distribution relative to its mean. It is defined as the expected value of the -th power of the deviation of from its mean . Let be the mean of the random variable .

For a Discrete Random Variable :

If is a discrete random variable with possible values and PMF , the -th central moment is defined as:

For a Continuous Random Variable :

If is a continuous random variable with PDF , the -th central moment is defined as:

Significance of the First Central Moment

The first central moment () is given by:

Using the linearity property of expectation, we have:

Since is a constant (the mean of ), . Therefore:

Significance: The first central moment is always zero. This is an important property because it means that the sum (or integral) of the deviations of all values from the mean, weighted by their probabilities, always cancels out to zero. It signifies that the mean is the "balancing point" or the center of gravity of the distribution. This property validates the mean as a measure of central tendency.

Definition of Mean (Expectation)

The mean, or expectation, of a random variable , denoted by or , is a measure of the central tendency of the probability distribution. It represents the long-run average value of the random variable if the experiment were repeated many times.

• For a Discrete Random Variable with PMF :
  

• For a Continuous Random Variable with PDF :
  

Important Properties of Expectation

1. Expectation of a Constant: The expectation of a constant is the constant itself.
   

   • Explanation: If a variable always takes the value , its average value is simply .

2. Linearity of Expectation: For any constants and , and any random variables and :
   

   • Explanation: This is a very powerful property, indicating that expectation is a linear operator. The expectation of a linear combination of random variables is the same linear combination of their individual expectations. This holds true regardless of whether and are independent.

3. Expectation of a Function of a Random Variable: If is a function of a random variable :

   • For Discrete :
   • For Continuous :
   • Explanation: To find the expected value of a transformed random variable , we apply the transformation to each possible value (or range) of and weight it by its corresponding probability (or density). This is crucial for calculating higher moments like or .

Definition of Variance

The variance of a random variable , denoted by or , is a measure of the spread or dispersion of its probability distribution. It quantifies how much the values of the random variable deviate from its mean. A higher variance indicates that the data points are more spread out from the mean, while a lower variance indicates that they are clustered closer to the mean.

Mathematically, variance is defined as the second central moment, i.e., the expected value of the squared deviation of from its mean :

• For a Discrete Random Variable with PMF :
  

• For a Continuous Random Variable with PDF :
  

Derivation of the Alternative Formula for Variance

We start with the definition of variance:

Expand the squared term :

Now, substitute this back into the variance definition:

Using the linearity property of expectation ( and ):

Since $2$ and are constants with respect to the expectation of , we can pull them out of the expectation operator:

We know that and (as is a constant).

Substitute these back into the equation:

Finally, replacing with :

This alternative formula is often more convenient for calculations because it avoids calculating deviations from the mean for each value.

Concept of Skewness

Skewness is a measure of the asymmetry of the probability distribution of a real-valued random variable about its mean. It indicates the degree to which the distribution's tail on one side of the mean is longer or fatter than the tail on the other side.

Calculation of Skewness

Skewness is typically calculated using the third central moment. The most common measure is Pearson's moment coefficient of skewness, :

where:

• is the third central moment.
• is the standard deviation.
• is the cube of the standard deviation, used for standardization to make the measure unitless.

Interpretation of Skewness

1. Positive Skewness (Right-Skewed):

   • Implication: If , the distribution is positively skewed. This means the tail on the right side of the distribution is longer or fatter than the left side.
   • Shape: The bulk of the observations (mode and median) are concentrated towards the left, and there are relatively few observations with large values, pulling the mean to the right of the median.
   • Order of Measures: Mode Median Mean
   • Example: Income distribution, where most people earn lower incomes, but a few high earners pull the average up.

2. Negative Skewness (Left-Skewed):

   • Implication: If , the distribution is negatively skewed. This means the tail on the left side of the distribution is longer or fatter than the right side.
   • Shape: The bulk of the observations are concentrated towards the right, and there are relatively few observations with small values, pulling the mean to the left of the median.
   • Order of Measures: Mean Median Mode
   • Example: Test scores on an easy exam, where most students score high, but a few low scores pull the average down.

3. Zero Skewness (Symmetric):

   • Implication: If , the distribution is perfectly symmetrical. The tails on both sides of the mean are of equal length and fatness.
   • Shape: The distribution is balanced around its mean. For symmetric distributions, the mean, median, and mode are approximately equal.
   • Example: Normal distribution, uniform distribution.

Concept of Kurtosis

Kurtosis is a measure that describes the "tailedness" or "peakedness" of the probability distribution of a real-valued random variable. It indicates how heavily the tails of a distribution differ from the tails of a normal distribution. A distribution with high kurtosis has heavy tails and a sharp, high peak, while a distribution with low kurtosis has light tails and a flatter peak.

Calculation of Kurtosis

Kurtosis is typically calculated using the fourth central moment. The moment coefficient of kurtosis, , is defined as:

where:

• is the fourth central moment.
• is the fourth power of the standard deviation, used for standardization to make the measure unitless.

Interpretation of Kurtosis Values

Kurtosis is often compared to that of a normal distribution, which has a kurtosis of 3. For this reason, Excess Kurtosis is commonly used:

Based on excess kurtosis, distributions are classified into three types:

1. Mesokurtic Distribution (Normal Kurtosis):

   • Excess Kurtosis: Approximately 0 (i.e., )
   • Shape: A distribution that has kurtosis similar to that of a normal distribution. It is neither particularly peaked nor flat, and its tails are of moderate length and thickness.
   • Example: The standard normal distribution.

2. Leptokurtic Distribution (High Kurtosis):

   • Excess Kurtosis: Greater than 0 (i.e., )
   • Shape: A distribution that has a higher peak and heavier (thicker and longer) tails than a normal distribution. This indicates that there is a higher probability of extreme values (outliers) compared to a normal distribution, and also a higher concentration of data around the mean.
   • Example: Student's t-distribution with low degrees of freedom, Laplace distribution.

3. Platykurtic Distribution (Low Kurtosis):

   • Excess Kurtosis: Less than 0 (i.e., )
   • Shape: A distribution that has a lower peak and lighter (thinner and shorter) tails than a normal distribution. This indicates that data points are more spread out from the mean and that there is a lower probability of extreme values compared to a normal distribution.
   • Example: Uniform distribution, Beta distribution with certain parameters.

Step 1: Verify PMF (optional but good practice)

First, let's ensure the probabilities sum to 1:

Sum = . The PMF is valid.

Step 2: Calculate the Mean ()

The mean for a discrete random variable is given by .

Step 3: Calculate

To find the variance using the formula , we first need .

Step 4: Calculate the Variance ()

Now, use the formula :

Final Answer:

• The mean
• The variance

Condition for a Valid PDF

For to be a valid Probability Density Function, two conditions must be met:

1. Non-negativity: for all . Since is defined for , both and are non-negative. Thus, must be non-negative. If , it's trivial, so we assume .
2. Total Probability: The integral of over its entire domain must be equal to 1.
   

Step-by-Step Calculation

We need to integrate from $0$ to $1$ and set the result equal to $1$ to find :

First, expand the term inside the integral:

Now, pull the constant outside the integral:

Perform the integration:

Evaluate the definite integral at the limits:

Find a common denominator for the fractions:

Finally, solve for :

Conclusion: The value of the constant that makes a valid PDF is .

Given Information

• Mean,
• Second raw moment,

Step 1: Calculate Variance

The variance of a random variable can be calculated using the formula that relates raw moments:

Substitute the given values into the formula:

Step 2: Calculate Standard Deviation

The standard deviation is the square root of the variance:

Substitute the calculated variance:

Final Answer:

• The variance of is .
• The standard deviation of is .

Raw Moments (Moments about the Origin)

• Definition: The -th raw moment, denoted by , is the expected value of . It is calculated relative to the origin (zero).

  • For discrete RV:
  • For continuous RV:

• Significance:

  • First Raw Moment (Mean): is the mean, a measure of central tendency. It indicates the average value or the 'center' of the distribution.
  • Higher Raw Moments: Raw moments provide information about the distribution's shape and scale. For example, (the second raw moment) is used in the calculation of variance. They are foundational for deriving central moments.
  • Direct Calculation: They are often easier to calculate directly from the data or the distribution function than central moments.

Central Moments (Moments about the Mean)

• Definition: The -th central moment, denoted by , is the expected value of , where is the mean of . It is calculated relative to the mean of the distribution.

  • For discrete RV:
  • For continuous RV:

• Significance: Central moments are more directly interpretable as measures of the shape of the distribution, independent of its location.

  • First Central Moment: . This signifies that the mean is the balancing point of the distribution.
  • Second Central Moment (Variance): . This is a crucial measure of the spread or dispersion of the data around the mean.
  • Third Central Moment (Skewness): . This is used to calculate skewness, indicating the asymmetry of the distribution.
  • Fourth Central Moment (Kurtosis): . This is used to calculate kurtosis, indicating the peakedness and tail fatness of the distribution.

Relationship and Usage

Central moments can be expressed in terms of raw moments, and vice versa. For example:

In summary, raw moments describe the characteristics of a distribution with respect to the origin, while central moments provide a location-independent description of the distribution's shape, specifically focusing on spread, asymmetry, and peakedness.

The first four central moments (after appropriate standardization) are crucial for describing the shape of a probability distribution because they quantify its fundamental characteristics: central tendency, spread, asymmetry, and peakedness/tailedness.

Let be the mean of the distribution and be its standard deviation.

1. First Central Moment (): Central Tendency (implied by 0)

   • Definition:
   • Quantification: As shown previously, . This doesn't directly describe a 'shape' feature other than confirming that the mean is indeed the center of gravity of the distribution. It implies that the sum of deviations from the mean is zero, indicating the balance point.

2. Second Central Moment (): Spread (Variance)

   • Definition:
   • Quantification: The variance measures the spread or dispersion of the data points around the mean. A larger variance indicates that values are more spread out, leading to a wider, flatter distribution curve. A smaller variance means values are clustered closer to the mean, resulting in a narrower, taller curve.
   • Standardization: The standard deviation is the square root of variance, providing spread in the original units of .

3. Third Central Moment (): Asymmetry (Skewness)

   • Definition:
   • Quantification: The third central moment is used to calculate skewness, which measures the asymmetry of the distribution. The standardized measure is Pearson's moment coefficient of skewness, .
     • If , the distribution is positively (right) skewed: a longer tail to the right, with most data concentrated on the left.
     • If , the distribution is negatively (left) skewed: a longer tail to the left, with most data concentrated on the right.
     • If , the distribution is symmetric.

4. Fourth Central Moment (): Peakedness and Tailedness (Kurtosis)

   • Definition:
   • Quantification: The fourth central moment is used to calculate kurtosis, which measures the peakedness of the distribution and the fatness of its tails compared to a normal distribution. The standardized measure is . Often, excess kurtosis () is used for comparison to the normal distribution.
     • If Excess Kurtosis (Leptokurtic): The distribution has a sharper peak and fatter tails (more outliers).
     • If Excess Kurtosis (Platykurtic): The distribution has a flatter peak and thinner tails (fewer outliers).
     • If Excess Kurtosis (Mesokurtic): The distribution has peakedness and tail characteristics similar to a normal distribution.

Together, these four moments provide a comprehensive summary of the distribution's location (mean), spread (variance/standard deviation), directional asymmetry (skewness), and the presence of extreme values (kurtosis).

Proof

Let . We want to find .

The definition of variance for any random variable is:

First, let's find the mean of , , using the linearity property of expectation ():

Let . So, .

Now, substitute and into the variance definition:

Simplify the expression inside the square bracket:

Substitute this back into the variance formula:

Since is a constant, we can pull it out of the expectation operator:

By definition, is the variance of , i.e., .

Therefore,

Interpretation

This property implies:

• Adding a constant to a random variable does not change its variance, because it merely shifts the entire distribution without changing its spread.
• Multiplying a random variable by a constant scales the variance by . The standard deviation would be scaled by . This is intuitive because squaring the deviations means the scaling factor also gets squared.

Construction of the CDF for a Discrete Random Variable

Let be a discrete random variable with ordered possible values and corresponding probabilities . The Cumulative Distribution Function (CDF), , is defined as .

To construct the CDF, we sum the probabilities of all values of that are less than or equal to a given :

• For : There are no possible values of less than or equal to . So, .

• For : The only possible value of less than or equal to is . So, .

• For : The possible values of less than or equal to are and . So, .

This pattern continues up to the largest value .

• For : .

• For : The sum of all probabilities is 1. So, .

General Shape of the CDF Graph

The graph of a CDF for a discrete random variable is a step function (or staircase function). Here's why:

• It starts at for values below the smallest possible outcome.
• It remains constant at a certain cumulative probability between two consecutive possible values ( and ). For instance, for all in the interval , will hold the value .
• It makes a sudden jump (discontinuity) at each possible value . The size of the jump at is exactly .
• It is right-continuous, meaning the function value at a jump point is at the 'top' of the step.
• It eventually reaches for values greater than or equal to the largest possible outcome.

Values of for which the CDF is Constant

The CDF for a discrete random variable is constant for any interval where and are consecutive possible values of . In these intervals, no new outcomes are included that are less than or equal to , so the cumulative probability does not change. The CDF only changes value (jumps) exactly at the points where can take a specific probability mass.

1. Probability Mass Function (PMF)

For a fair six-sided die, each outcome has an equal probability. The possible outcomes for are .

The PMF is:

2. Cumulative Distribution Function (CDF)

The CDF is a step function:

• for
• for
• for
• for
• for
• for
• for

In summary:

3. Mean ()

4. Variance ()

First, calculate :

Now, use the formula :

Find a common denominator (12):

Final Answer Summary:

• PMF: for
• CDF: Step function as defined above.
• Mean:
• Variance: