Unit 7: Standard Distribution

ECAP790 7 min read

I. Orientation — Probability Distributions

A probability distribution specifies how probabilities are assigned to the possible values of a random variable. Standard distributions provide mathematical models for recurring random phenomena, such as repeated trials, event counts, waiting times, and continuous measurements.

Defining framework:

Random variable: A numerical function (X) whose value depends on the outcome of a random experiment.
- A discrete random variable takes countable values, such as (0,1,2,\ldots).
- A continuous random variable can take any value within an interval.
Probability mass function (PMF): For discrete (X), (p(x)=P(X=x)), where:
- (p(x)\geq 0) for every (x);
- (\sum_x p(x)=1).
Probability density function (PDF): For continuous (X), a function (f(x)) satisfying:
- (f(x)\geq 0);
- (\int_{-\infty}^{\infty}f(x)\,dx=1);
- (P(a\leq X\leq b)=\int_a^b f(x)\,dx).
Cumulative distribution function (CDF): (F(x)=P(X\leq x)). It applies to both discrete and continuous variables.

Expected value: The distribution’s long-run average:

TEXT

Discrete:   E(X) = Σ x p(x)
Continuous: E(X) = ∫ x f(x) dx

Variance: The spread about the mean:
TEXT
```
Var(X) = E[(X − μ)²] = E(X²) − μ²
```
Here, (\mu=E(X)), and the standard deviation is (\sigma=\sqrt{\operatorname{Var}(X)}).
Model selection: The binomial, Poisson, and negative binomial distributions are discrete; the normal distribution is continuous.

II. Binomial Distribution — Number of Successes in Fixed Trials

A. Binomial Distribution and its properties

The binomial distribution models the number of successes obtained in a fixed number of independent and identical Bernoulli trials.

Conditions: A random variable (X) has a binomial distribution when:
- the experiment contains a fixed number (n) of trials;
- each trial has exactly two outcomes, conventionally called success and failure;
- trials are independent;
- the probability of success (p) remains constant;
- the probability of failure is (q=1-p).
Notation: The model is written (X\sim\operatorname{Bin}(n,p)), where (n) is the number of trials and (p) is the success probability.
Probability mass function:
TEXT
```
P(X = x) = C(n,x) pˣ qⁿ⁻ˣ,    x = 0,1,2,…,n

C(n,x) = n! / [x!(n−x)!]
```
Here, (x) is the number of successes, (q=1-p), and (C(n,x)) counts the ways of placing (x) successes among (n) trials.
Mean: The expected number of successes is:
TEXT
```
E(X) = np
```
Each trial contributes an expected success count of (p), so (n) trials contribute (np).
Variance and standard deviation:
TEXT
```
Var(X) = npq
SD(X) = √(npq)
```
Variability decreases as (p) approaches (0) or (1), because outcomes then become more predictable.
Mode: A mode is (\lfloor(n+1)p\rfloor). If ((n+1)p) is an integer, the two modes are ((n+1)p-1) and ((n+1)p).
Shape:
- The distribution is symmetric when (p=0.5).
- It is positively skewed when (p<0.5).
- It is negatively skewed when (p>0.5).
Moment-generating function:
TEXT
```
Mₓ(t) = (q + peᵗ)ⁿ
```
Here, (t) is a real argument and (e) is the exponential constant.
Additive property: If (X\sim\operatorname{Bin}(n_1,p)) and (Y\sim\operatorname{Bin}(n_2,p)) are independent, then:
TEXT
```
X + Y ~ Bin(n₁ + n₂, p)
```
Worked example: If ten independent components each function with probability (0.8), then the probability that exactly eight function is:
TEXT
```
P(X = 8) = C(10,8)(0.8)⁸(0.2)²
         = 45(0.16777216)(0.04)
         ≈ 0.3020
```
Applications and limitations: It suits defect counts, correct responses, and yes/no outcomes. It is unsuitable when trials are dependent, (p) changes between trials, or more than two outcomes must be distinguished.

III. Poisson Distribution — Counts within a Fixed Interval

A. Poisson Distribution and its properties

The Poisson distribution models the number of independently occurring events in a fixed interval of time, area, length, or volume when events occur at a constant average rate.

Conditions: The model assumes that:
- events occur independently;
- the mean event rate remains constant;
- two events cannot occur at exactly the same infinitesimal location or instant;
- the probability of one event in a very small interval is proportional to its size;
- the probability of multiple events in that interval is negligible.
Notation: (X\sim\operatorname{Pois}(\lambda)), where (\lambda>0) is the expected number of events in the specified interval.
Probability mass function:
TEXT
```
P(X = x) = e⁻λ λˣ / x!,    x = 0,1,2,…
```
Here, (x) is the event count, (\lambda) is the average count, and (e\approx2.71828).
Mean, variance, and standard deviation:
TEXT
```
E(X) = λ
Var(X) = λ
SD(X) = √λ
```
Equality of the theoretical mean and variance is a defining Poisson property.
Mode: The mode is (\lfloor\lambda\rfloor) when (\lambda) is not an integer. If (\lambda) is a positive integer, both (\lambda-1) and (\lambda) are modes.
Shape: The distribution is strongly right-skewed for small (\lambda), but becomes more symmetric and bell-shaped as (\lambda) increases.
Moment-generating function:
TEXT
```
Mₓ(t) = exp[λ(eᵗ − 1)]
```
Here, (\exp(u)=e^u), and (t) is the MGF argument.
Additive property: For independent (X\sim\operatorname{Pois}(\lambda_1)) and (Y\sim\operatorname{Pois}(\lambda_2)):
TEXT
```
X + Y ~ Pois(λ₁ + λ₂)
```
Rate scaling: If events average (r) per unit and the observed interval has size (s), then (\lambda=rs).
Worked example: If calls arrive at an average of three per minute, the probability of exactly two calls in one minute is:
TEXT
```
P(X = 2) = e⁻³ 3² / 2!
         = 4.5e⁻³
         ≈ 0.2240
```
Binomial approximation: For large (n), small (p), and (\lambda=np), (\operatorname{Bin}(n,p)) is approximately (\operatorname{Pois}(\lambda)).
Applications and limitations: It models arrivals, accidents, defects, and radioactive emissions. It can fit poorly when events cluster, rates vary, or the observed variance substantially exceeds the mean.

IV. Negative Binomial Distribution — Trials Required for Multiple Successes

A. Negative Binomial Distribution and its properties

The negative binomial distribution models repeated independent Bernoulli trials continued until a specified number of successes has occurred.

Convention: Let (X) denote the total number of trials required to obtain the (r)-th success. Then (X=r,r+1,\ldots), where (r) is a positive integer.
Conditions: Trials are independent, each has success probability (p), failure probability (q=1-p), and experimentation stops at the (r)-th success.
Probability mass function:
TEXT
```
P(X = x) = C(x−1,r−1) pʳ qˣ⁻ʳ,    x = r,r+1,…
```
The final trial must be a success; among the first (x-1) trials, exactly (r-1) must be successes.
Alternative convention: If (Y=X-r) counts failures before the (r)-th success, then:
TEXT
```
P(Y = y) = C(y+r−1,r−1) pʳ qʸ,    y = 0,1,2,…
```
Stating the convention prevents confusion about the variable’s support and mean.
Mean, variance, and standard deviation for total trials:
TEXT
```
E(X) = r/p
Var(X) = rq/p²
SD(X) = √(rq)/p
```
For failures (Y), (E(Y)=rq/p), while the variance remains (rq/p^2).
Geometric special case: When (r=1), the negative binomial becomes the geometric distribution, which models the trial of the first success.
Additive property: If independent variables count trials associated with (r_1) and (r_2) successes using the same (p), their failure-count versions add to a negative binomial variable with parameter (r_1+r_2).
Moment-generating function for total trials:
TEXT
```
Mₓ(t) = [peᵗ / (1 − qeᵗ)]ʳ,    qeᵗ < 1
```
Here, (t) is real, (p) is the success probability, and (q=1-p).
Shape: It is generally right-skewed; skewness decreases as (r) grows or as successes become more likely.
Worked example: With (p=0.6), the probability that the third success occurs on the fifth trial is:
TEXT
```
P(X = 5) = C(4,2)(0.6)³(0.4)²
         = 6(0.216)(0.16)
         = 0.20736
```
Applications and limitations: It models attempts until repeated success and overdispersed count data. The elementary trial model is unsuitable if success probabilities change or trials influence one another.

V. Normal Distribution — Continuous Symmetric Measurements

A. Normal Distribution and its properties

The normal distribution is a continuous, bell-shaped distribution determined completely by its mean and variance.

Notation: (X\sim N(\mu,\sigma^2)), where (-\infty<\mu<\infty) and (\sigma>0).
Probability density function:
TEXT
```
f(x) = [1/(σ√(2π))] exp[−(x−μ)²/(2σ²)],    −∞ < x < ∞
```
Here, (\mu) is the mean, (\sigma) is the standard deviation, (\pi) is the circle constant, and (e) is the exponential constant.
Central properties:
- Mean, median, and mode all equal (\mu).
- Variance is (\sigma^2), and standard deviation is (\sigma).
- The curve is symmetric about (x=\mu).
- Total area beneath the curve is (1).
- The tails approach but never meet the horizontal axis.
- The inflection points occur at (\mu-\sigma) and (\mu+\sigma).
Probability as area: Because (X) is continuous, (P(X=x)=0), while interval probabilities are integrals of (f(x)).
Empirical rule: Approximately (68.27\%), (95.45\%), and (99.73\%) of observations lie within one, two, and three standard deviations of (\mu), respectively.
Standard normal distribution: Standardization converts (X\sim N(\mu,\sigma^2)) into (Z\sim N(0,1)):
TEXT
```
Z = (X − μ)/σ
```
Here, (Z) measures how many standard deviations (X) lies above or below (\mu).
Moment-generating function:
TEXT
```
Mₓ(t) = exp(μt + σ²t²/2)
```
Linear transformation: If (X\sim N(\mu,\sigma^2)), then:
TEXT
```
aX + b ~ N(aμ + b, a²σ²)
```
Here, (a) and (b) are constants.
Additive property: Independent normal variables have a normal sum; their means and variances add.
Worked example: If (X\sim N(100,15^2)), then for (X=130):
TEXT
```
z = (130 − 100)/15 = 2
```
Thus, (130) is two standard deviations above the mean, and (P(X\leq130)=P(Z\leq2)\approx0.9772).
Applications and limitations: It models many biological measurements, errors, and sample statistics. It is inappropriate for strongly skewed, bounded, multimodal, or heavy-tailed data without adequate justification.