Unit 2 - Notes

ECE180

Unit 2: Distribution and Density Functions

1. Fundamentals of Distribution and Density Functions

1.1 Cumulative Distribution Function (CDF)

The Cumulative Distribution Function, denoted as $F_X(x)$ , describes the probability that a random variable $X$ takes on a value less than or equal to a specific number $x$ .

Mathematical Definition:

F_X(x) = P(X \le x)

Properties of CDF:

Boundedness: $0 \le F_X(x) \le 1$ for all $x$ .
Monotonicity: $F_X(x)$ is a non-decreasing function. If $x_1 < x_2$ , then $F_X(x_1) \le F_X(x_2)$ .
Limits:
- $F_X(-\infty) = 0$
- $F_X(+\infty) = 1$
Right-Continuity: $F_X(x)$ is continuous from the right. $\lim_{\epsilon \to 0^+} F_X(x + \epsilon) = F_X(x)$ .
Interval Probability: $P(x_1 < X \le x_2) = F_X(x_2) - F_X(x_1)$ .

1.2 Probability Density Function (PDF)

The Probability Density Function, denoted as $f_X(x)$ , describes the likelihood of a continuous random variable falling within a particular range of values. It is the derivative of the CDF.

Mathematical Definition:

f_X(x) = \frac{d}{dx}F_X(x)

Conversely:

F_X(x) = \int_{-\infty}^{x} f_X(u) du

Properties of PDF:

Non-negativity: $f_X(x) \ge 0$ for all $x$ .
Normalization: The total area under the PDF curve is 1.
$\int_{-\infty}^{+\infty} f_X(x) dx = 1$
Interval Probability:
$P(x_1 < X \le x_2) = \int_{x_1}^{x_2} f_X(x) dx$

2. Standard Discrete Distributions

Although often referred to via probability mass functions (PMF), these are critical distributions.

2.1 Binomial Distribution

Describes the number of successes ( $k$ ) in a fixed number of independent Bernoulli trials ( $n$ ), each with the same probability of success ( $p$ ).

Notation: $X \sim B(n, p)$
PMF ( $P(X=k)$ ):
$P(X=k) = \binom{n}{k} p^k (1-p)^{n-k} \quad \text{for } k = 0, 1, ..., n$
Mean ( $E[X]$ ): $np$
Variance ( $\sigma^2_X$ ): $np(1-p)$

2.2 Poisson Distribution

Models the number of events occurring in a fixed interval of time or space, given they occur with a known constant mean rate and independently of the time since the last event.

Parameter: $\lambda > 0$ (average rate of occurrence).
PMF:
$P(X=k) = \frac{e^{-\lambda} \lambda^k}{k!} \quad \text{for } k = 0, 1, 2, ...$
Mean ( $E[X]$ ): $\lambda$
Variance ( $\sigma^2_X$ ): $\lambda$

3. Standard Continuous Distributions

3.1 Uniform Distribution

The probability is constant over a defined interval $[a, b]$ and zero elsewhere.

Notation: $X \sim U(a, b)$
Density Function (PDF):
$f_X(x) = \begin{cases} \frac{1}{b-a} & a \le x \le b \\ 0 & \text{otherwise} \end{cases}$
Distribution Function (CDF):
$F_X(x) = \begin{cases} 0 & x < a \\ \frac{x-a}{b-a} & a \le x \le b \\ 1 & x > b \end{cases}$
Mean: $\frac{a+b}{2}$
Variance: $\frac{(b-a)^2}{12}$

3.2 Exponential Distribution

Often models the time between events in a Poisson process (e.g., time between failures). It possesses the memoryless property.

Parameter: $\lambda > 0$ (rate parameter).
Density Function (PDF):
$f_X(x) = \begin{cases} \lambda e^{-\lambda x} & x \ge 0 \\ 0 & x < 0 \end{cases}$
Distribution Function (CDF):
$F_X(x) = \begin{cases} 1 - e^{-\lambda x} & x \ge 0 \\ 0 & x < 0 \end{cases}$
Mean: $\frac{1}{\lambda}$
Variance: $\frac{1}{\lambda^2}$

3.3 Gaussian (Normal) Distribution

The most important distribution in statistics due to the Central Limit Theorem. It is bell-shaped and symmetric.

Parameters: Mean $\mu$ and Variance $\sigma^2$ .
Density Function (PDF):
$f_X(x) = \frac{1}{\sqrt{2\pi}\sigma} e^{-\frac{(x-\mu)^2}{2\sigma^2}} \quad \text{for } -\infty < x < \infty$
Properties:
- Symmetric about $x = \mu$ .
- Maximum value is $\frac{1}{\sqrt{2\pi}\sigma}$ at $x = \mu$ .
Standard Normal Distribution ( $Z$ ): When $\mu = 0$ and $\sigma = 1$ .
Mean: $\mu$
Variance: $\sigma^2$

3.4 Rayleigh Distribution

Frequently used in communication theory to model the envelope of a narrowband noise signal or fading channels.

Parameter: $\sigma^2 > 0$ (related to the mode).
Density Function (PDF):
$f_X(x) = \begin{cases} \frac{x}{\sigma^2} e^{-x^2 / 2\sigma^2} & x \ge 0 \\ 0 & x < 0 \end{cases}$
Distribution Function (CDF):
$F_X(x) = 1 - e^{-x^2 / 2\sigma^2} \quad \text{for } x \ge 0$
Mean: $\sigma \sqrt{\frac{\pi}{2}}$
Variance: $\frac{4-\pi}{2}\sigma^2$

4. Conditional Distribution and Density

Conditional probability extends to random variables, defining the behavior of $X$ given that a specific event $B$ has occurred.

4.1 Conditional Distribution Function

The conditional distribution function of a random variable $X$ , given event $B$ with $P(B) > 0$ , is defined as:

F_X(x | B) = P(X \le x | B) = \frac{P(\{X \le x\} \cap B)}{P(B)}

Properties:

$F_X(-\infty | B) = 0$ and $F_X(+\infty | B) = 1$ .
$0 \le F_X(x | B) \le 1$ .
$F_X(x | B)$ is a non-decreasing function of $x$ .

4.2 Conditional Density Function

The conditional density function is the derivative of the conditional CDF:

f_X(x | B) = \frac{d}{dx} F_X(x | B)

Properties:

$f_X(x | B) \ge 0$ .
$\int_{-\infty}^{\infty} f_X(x | B) dx = 1$ .
$P(x_1 < X \le x_2 | B) = \int_{x_1}^{x_2} f_X(x | B) dx$ .

5. Methods of Defining Conditioning Events

The event $B$ generally restricts the sample space of the random variable $X$ .

Case A: Point Conditioning (Discrete)

Used mainly for discrete random variables.
Let event $B = \{X = x_k\}$ .

$P(X=x_j | X=x_k) = 1$ if $j=k$ , and $0$ otherwise.

Case B: Interval Conditioning (Continuous)

Let event $B$ be the event that $X$ falls in the interval $(x_1, x_2]$ .

B = \{x_1 < X \le x_2\}

Probability of B:

P(B) = \int_{x_1}^{x_2} f_X(x) dx = F_X(x_2) - F_X(x_1)

Conditional PDF derivation:
For the region outside the interval $(x_1, x_2]$ , the probability is zero. Inside the interval:

f_X(x | x_1 < X \le x_2) = \begin{cases} \frac{f_X(x)}{P(B)} = \frac{f_X(x)}{\int_{x_1}^{x_2} f_X(u) du} & x_1 < x \le x_2 \\ 0 & \text{otherwise} \end{cases}

Concept: The conditional PDF is simply the original PDF "chopped" to the interval and scaled up (normalized) so the total area remains 1.

6. Problems and Examples

Problem 1: Exponential Distribution (Memoryless Property)

Given: The lifetime of a component is exponentially distributed with parameter $\lambda = 0.1$ .
Find: The probability that the component lasts more than 15 hours, given that it has already lasted 10 hours.

Solution:
Let $X$ be the lifetime. PDF is $f_X(x) = 0.1e^{-0.1x}$ for $x>0$ .
We need to find $P(X > 15 | X > 10)$ .

Using the definition of conditional probability:

P(X > 15 | X > 10) = \frac{P(X > 15 \cap X > 10)}{P(X > 10)}

Since

X > 15

implies

X > 10

, the intersection is just

X > 15

= \frac{P(X > 15)}{P(X > 10)}

For exponential distribution, $P(X > t) = e^{-\lambda t}$ .

= \frac{e^{-0.1(15)}}{e^{-0.1(10)}} = \frac{e^{-1.5}}{e^{-1.0}} = e^{-0.5} \approx 0.6065

Note: Using the memoryless property, $P(X > 10+5 | X > 10) = P(X > 5) = e^{-0.1(5)} = e^{-0.5}$ .

Problem 2: Conditional Density on Uniform Distribution

Given: A random variable $X$ is uniformly distributed in the interval $[-2, 2]$ . Let event $B = \{X > 0\}$ .
Find: The conditional PDF $f_X(x | B)$ .

Solution:

Original PDF:
Since range is $[-2, 2]$ , width is 4.
$f_X(x) = 1/4$ for $-2 \le x \le 2$ , else 0.
Probability of Event B:
$B = \{0 < X \le 2\}$ (within the valid range).

$P(B) = \int_{0}^{2} \frac{1}{4} dx = \frac{1}{4}[x]_0^2 = \frac{2}{4} = 0.5$
Conditional PDF:

$f_X(x | B) = \frac{f_X(x)}{P(B)} = \frac{1/4}{1/2} = \frac{2}{4} = 0.5$
This is valid only where $X \in B$ (i.e., $0 < x \le 2$ ).

Result:

$f_X(x | X > 0) = \begin{cases} 0.5 & 0 < x \le 2 \\ 0 & \text{otherwise} \end{cases}$

Problem 3: Gaussian Probability

Given: $X$ is a Gaussian random variable with $\mu = 10$ and $\sigma = 2$ .
Find: $P(8 < X < 12)$ .

Solution:
Transform $X$ to the Standard Normal variable $Z$ using $Z = \frac{X - \mu}{\sigma}$ .

For $x_1 = 8$ : $z_1 = \frac{8 - 10}{2} = -1$
For $x_2 = 12$ : $z_2 = \frac{12 - 10}{2} = 1$

P(8 < X < 12) = P(-1 < Z < 1)

= F_Z(1) - F_Z(-1)

Using standard normal properties where $F_Z(-x) = 1 - F_Z(x)$ :

= F_Z(1) - (1 - F_Z(1)) = 2F_Z(1) - 1

From standard Z-tables, $F_Z(1) \approx 0.8413$ .

P = 2(0.8413) - 1 = 1.6826 - 1 = 0.6826

(This confirms the empirical rule that roughly 68% of data falls within one standard deviation of the mean).