Unit 4: Discrete Probability-I

1. Finite Probability

Finite probability deals with experiments that have a finite number of possible outcomes.

Key Definitions

• Experiment: A procedure that yields one of a given set of possible outcomes.
• Sample Space (): The set of all possible outcomes of an experiment.
• Event (): A subset of the sample space ().

Laplace's Definition of Probability

If the sample space consists of a finite number of equally likely outcomes, the probability of an event is defined as the ratio of the number of outcomes in to the total number of outcomes in .

Example: Rolling a standard fair 6-sided die.

• Sample space . .
• Event (rolling an even number) . .
• .

2. Assigning Probabilities

In many real-world scenarios, outcomes are not equally likely. In these cases, we must assign a specific probability to each individual outcome.

Probability Axioms (Kolmogorov's Axioms for Discrete Spaces)

Let be a sample space with outcomes . A probability distribution assigns a real number to each outcome such that two conditions are met:

1. Non-negativity: for each .
2. Normalization: The sum of the probabilities of all outcomes in the sample space must equal exactly 1.
   

Probability of an Event

For any event , the probability of is the sum of the probabilities of the individual outcomes contained in :

3. Probabilities of Complements and Unions of Events

The Complement Rule

The complement of an event , denoted as (or ), consists of all outcomes in the sample space that are not in .
Since , we have:

Use case: This is extremely useful when calculating the probability of is complex, but calculating the probability of it not happening is simple (e.g., "finding the probability of getting at least one heads in 10 coin flips" translates to ).

Unions of Events

To find the probability that either event or event (or both) occurs, we look at the union .

• Mutually Exclusive Events: If and cannot occur at the same time (), then:
  

• General Addition Rule (Inclusion-Exclusion Principle): If and are not mutually exclusive, simply adding their probabilities double-counts the outcomes in their intersection. We must subtract the intersection:
  

4. Conditional Probability

Conditional probability measures the probability of an event occurring given that another event has already occurred. This effectively restricts or "updates" our sample space to the condition that has been met.

Definition and Formula

Let and be events, with . The conditional probability of given , denoted , is:

• Intuition: becomes the new "sample space" (the denominator), and we are looking for the fraction of that space where also occurs ().

5. Independence

Two events are independent if the occurrence of one does not affect the probability of the other.

Definition

Events and are independent if and only if:

Relation to Conditional Probability

If , independence can also be defined using conditional probability:

This means knowing that happened gives no new information about the likelihood of happening.

6. Pairwise and Mutual Independence

When dealing with more than two events, independence becomes more nuanced.

Let be a set of events.

Pairwise Independence

A set of events is pairwise independent if every pair of events within the set is independent.
For all :

Mutual (Complete) Independence

A set of events is mutually independent if every possible finite sub-collection of the events satisfies the multiplication rule.
For any subset of indices :

• Crucial Distinction: Pairwise independence does not imply mutual independence. It is possible for every pair of events to be independent, yet for all the events together to be dependent. Mutual independence is a much stronger condition.

7. Bernoulli Trials and the Binomial Distribution

Bernoulli Trials

A Bernoulli trial is an experiment that has exactly two possible outcomes. These are typically designated as:

• Success (): Probability denoted by .
• Failure (): Probability denoted by .

The Binomial Distribution

When we repeat a Bernoulli trial times, and each trial is independent of the others, we are dealing with a binomial distribution. We are usually interested in the probability of getting exactly successes in those trials.

Binomial Formula

The probability of exactly successes in independent Bernoulli trials, with probability of success and probability of failure , is given by:

• represents the number of ways to choose exactly successes out of trials.
• is the probability of the successes.
• is the probability of the failures.

8. Random Variables

Despite the name, a random variable is neither random nor a variable. It is a strictly defined mathematical function.

Definition

A random variable is a function that assigns a real number to each outcome in the sample space of an experiment.

• Discrete Random Variable: A random variable that takes on a finite or countably infinite number of values (e.g., the number of heads in 10 coin tosses).

Probability Mass Function (PMF)

The probability distribution of a discrete random variable is described by its probability mass function (PMF). The PMF defines the probability that the random variable equals a specific value :

The PMF must satisfy two conditions:

1. for all .
2. (summing over all possible values can take).

Example: Let be the random variable representing the sum of two rolled dice.

• .