Back to Subject

Unit 3 - Notes

MTH302 8 min read

Unit 3: Special Discrete Distributions

1. The Bernoulli Process

A Bernoulli process is a sequence of discrete, random experiments, often called trials, that satisfy the following conditions:

  1. Two Outcomes: Each trial has only two possible, mutually exclusive outcomes, typically labeled "success" (S) and "failure" (F).
  2. Constant Probability: The probability of a success, denoted by p, remains constant from trial to trial. The probability of a failure is q = 1 - p.
  3. Independence: The outcome of each trial is independent of the outcomes of all previous trials.

The distributions derived from the Bernoulli process (Binomial, Geometric, Negative Binomial) are concerned with counting the number of successes or trials in a sequence.

The Bernoulli Distribution

This is the fundamental building block of the Bernoulli process. It represents a single trial.

  • Description: The probability distribution of a random variable which takes the value 1 with probability p (success) and the value 0 with probability 1-p (failure).
  • Random Variable: X ~ Bernoulli(p)
  • PMF (Probability Mass Function):
    TEXT
        P(X = x) = p^x * (1-p)^(1-x)   for x = 0, 1
  • Mean (Expected Value): E[X] = p
  • Variance: Var(X) = p(1-p) = pq
  • MGF (Moment Generating Function): M(t) = (1 - p) + p*e^t

2. Binomial Distribution

  • Description: Models the number of successes (x) in a fixed number of independent Bernoulli trials (n).
  • Intuition: Use this when you know the total number of trials in advance and want to find the probability of a specific number of successes. Think "number of successes in n trials".
  • Random Variable: X ~ Bin(n, p)
  • Parameters:
    • n: The fixed number of trials. (n is a positive integer)
    • p: The probability of success on a single trial. (0 ≤ p ≤ 1)

Probability Mass Function (PMF)

The probability of observing exactly x successes in n trials is:

TEXT
P(X = x) = C(n, x) * p^x * (1-p)^(n-x)

Where:

  • C(n, x) or (n choose x) is the binomial coefficient: n! / (x! * (n-x)!). It represents the number of ways to choose x successes from n trials.
  • p^x: The probability of getting x successes.
  • (1-p)^(n-x): The probability of getting n-x failures.
  • x: The number of successes, where x = 0, 1, 2, ..., n.

Key Properties

  • Mean (Expected Value): The average number of successes.
    TEXT
        E[X] = n * p
  • Variance: The spread of the number of successes.
    TEXT
        Var(X) = n * p * (1-p) = n * p * q
  • Standard Deviation: σ = sqrt(n * p * q)

Moment Generating Function (MGF)

TEXT
M(t) = (q + p*e^t)^n

Where q = 1-p.

Example

Problem: A fair coin is flipped 10 times. What is the probability of getting exactly 6 heads?

  • Identify: This is a binomial experiment.
    • n = 10 (fixed number of flips)
    • p = 0.5 (probability of success, i.e., getting a head)
    • x = 6 (desired number of successes)
    • q = 1 - 0.5 = 0.5
  • Formula: P(X = 6) = C(10, 6) * (0.5)^6 * (0.5)^(10-6)
  • Calculation:
    • C(10, 6) = 10! / (6! * 4!) = (10 * 9 * 8 * 7) / (4 * 3 * 2 * 1) = 210
    • P(X = 6) = 210 * (0.5)^6 * (0.5)^4 = 210 * (0.5)^10
    • P(X = 6) = 210 * 0.0009765625 ≈ 0.2051

3. Negative Binomial Distribution

  • Description: Models the number of trials (x) required to achieve a fixed number of successes (k).
  • Intuition: Use this when the experiment stops once a target number of successes is reached. Think "number of trials to get the k-th success".
  • Random Variable: X ~ NB(k, p)
  • Parameters:
    • k: The fixed number of successes to be observed. (k is a positive integer)
    • p: The probability of success on a single trial. (0 ≤ p ≤ 1)

Probability Mass Function (PMF)

The probability that the k-th success occurs on the x-th trial is:

TEXT
P(X = x) = C(x-1, k-1) * p^k * (1-p)^(x-k)

Where:

  • The k-th success must occur on the x-th trial (the last trial).
  • Therefore, we must have k-1 successes in the first x-1 trials. C(x-1, k-1) accounts for this.
  • p^k: The probability of getting k successes.
  • (1-p)^(x-k): The probability of getting x-k failures.
  • x: The total number of trials, where x = k, k+1, k+2, .... Note that x must be at least k.

Key Properties

  • Mean (Expected Value):
    TEXT
        E[X] = k / p
  • Variance:
    TEXT
        Var(X) = (k * (1-p)) / p^2 = (k * q) / p^2

Moment Generating Function (MGF)

TEXT
M(t) = (p * e^t / (1 - q*e^t))^k

Where q = 1-p, for t < -ln(q).

Example

Problem: A basketball player makes 80% of her free throws. What is the probability that she makes her 5th successful free throw on her 7th attempt?

  • Identify: This is a negative binomial experiment.
    • k = 5 (target number of successes)
    • x = 7 (trial number on which the last success occurs)
    • p = 0.8 (probability of success)
    • q = 0.2
  • Formula: P(X = 7) = C(7-1, 5-1) * (0.8)^5 * (0.2)^(7-5)
  • Calculation:
    • C(6, 4) = 6! / (4! * 2!) = (6 * 5) / 2 = 15
    • P(X = 7) = 15 * (0.8)^5 * (0.2)^2
    • P(X = 7) = 15 * (0.32768) * (0.04) ≈ 0.1966

4. Geometric Distribution

  • Description: A special case of the Negative Binomial distribution where k=1. It models the number of trials (x) required to get the first success.
  • Intuition: Use this when you are waiting for something to happen for the first time. Think "trials until the first success".
  • Random Variable: X ~ Geo(p)
  • Parameters:
    • p: The probability of success on a single trial. (0 < p ≤ 1)

Probability Mass Function (PMF)

The probability that the first success occurs on the x-th trial is:

TEXT
P(X = x) = (1-p)^(x-1) * p

Where:

  • (1-p)^(x-1): The probability of having x-1 failures in a row.
  • p: The probability of the first success occurring on trial x.
  • x: The trial number of the first success, where x = 1, 2, 3, ....

Note: An alternative definition of the Geometric distribution models the number of failures Y before the first success. In that case, Y = X - 1, the PMF is `P(Y = y) = (1-p)^y pfory = 0, 1, 2, ..., and the mean isq/p`.* We will use the first definition (number of trials) for consistency.

Key Properties

  • Mean (Expected Value):
    TEXT
        E[X] = 1 / p
  • Variance:
    TEXT
        Var(X) = (1-p) / p^2 = q / p^2
  • Memoryless Property: This is a unique and important property. It states that the probability of a future success is independent of past failures. Mathematically:
    P(X > a + b | X > a) = P(X > b)
    For example, if you've already flipped a coin 5 times and haven't gotten a head, the probability of needing more than 2 additional flips to get the first head is the same as the initial probability of needing more than 2 flips. The process "forgets" the previous failures.

Moment Generating Function (MGF)

TEXT
M(t) = p * e^t / (1 - q*e^t)

Where q = 1-p, for t < -ln(q).

Example

Problem: The probability of a defective component being produced is 0.05. What is the probability that the first defective component is the 8th one tested?

  • Identify: This is a geometric experiment.
    • p = 0.05 (probability of success, i.e., finding a defective component)
    • x = 8 (trial on which the first success occurs)
    • q = 0.95
  • Formula: P(X = 8) = (0.95)^(8-1) * 0.05
  • Calculation:
    • P(X = 8) = (0.95)^7 * 0.05
    • P(X = 8) ≈ (0.6983) * 0.05 ≈ 0.0349

5. Poisson Distribution

  • Description: Models the number of times an event occurs within a specified interval of time, area, volume, or distance. The events occur independently and at a constant average rate.
  • Intuition: Use this for counting occurrences over a continuous interval, not discrete trials. Think "number of events in an interval". Examples: number of calls to a call center in an hour, number of typos per page of a book, number of mutations in a strand of DNA.
  • Random Variable: X ~ Poisson(λ)
  • Parameters:
    • λ (lambda): The average number of events in the given interval. (λ > 0)

Probability Mass Function (PMF)

The probability of observing exactly x events in an interval is:

TEXT
P(X = x) = (λ^x * e^-λ) / x!

Where:

  • e is the base of the natural logarithm (≈ 2.71828).
  • λ is the mean number of events in the interval.
  • x: The number of occurrences, where x = 0, 1, 2, ....

Key Properties

  • Mean (Expected Value):
    TEXT
        E[X] = λ
  • Variance:
    TEXT
        Var(X) = λ

    A defining feature of the Poisson distribution is that its mean and variance are equal.

Poisson Approximation to the Binomial

The Poisson distribution can be used to approximate the Binomial distribution when:

  1. n is very large (e.g., n > 50).
  2. p is very small (e.g., p < 0.1).
  3. The mean np is moderate (e.g., np < 10).

In this case, you can set λ = n * p and use the Poisson PMF. This is useful because n! can be computationally difficult for large n.

Moment Generating Function (MGF)

TEXT
M(t) = e^(λ * (e^t - 1))

Example

Problem: A customer service center receives an average of 10 calls per hour. What is the probability of receiving exactly 3 calls in a given hour?

  • Identify: This is a Poisson experiment.
    • The interval is one hour.
    • λ = 10 (average number of calls in that interval)
    • x = 3 (desired number of calls)
  • Formula: P(X = 3) = (10^3 * e^-10) / 3!
  • Calculation:
    • P(X = 3) = (1000 * e^-10) / (3 * 2 * 1)
    • e^-10 ≈ 0.0000454
    • P(X = 3) = (1000 * 0.0000454) / 6 ≈ 0.00757

Example (Adjusting the Interval): Using the same data, what is the probability of receiving 5 calls in 30 minutes?

  • Adjust λ: The original rate is 10 calls/hour. For a 30-minute (0.5 hour) interval, the new average rate is λ' = 10 * 0.5 = 5.
  • x = 5
  • Formula: P(X = 5) = (5^5 * e^-5) / 5!
  • Calculation: P(X = 5) = (3125 * e^-5) / 120 ≈ 0.1755
Previous
Unit 2
Next
Unit 4