Basics of Law of Total Probability:

Probability is a concept used in Mathematics especially in statistics to predict the likelihood of occurrence of an event. Probability gives an idea of whether an event will happen or not and if the event is predicted to happen, how much can we rely on the occurrence of an event. The probability of an event is a dimensionless quantity. It may be any numerical value that ranges from zero to one. If the probability of an event is 1, then the event is sure to happen and hence is called a sure event or a certain event. If the probability of an event is ‘0’, the event will not happen at all and hence is called an impossible event. The total probability theorem is a theorem that relates the conditional probability with the marginal probability.

Total Probability Theorem:

Consider two events A and B as indicated in the Venn diagram shown in figure 1. Let us consider that these two events are associated with the same sample space represented as ‘S’. The entire sample space can be divided into the following portions indicating the subsets of sample space ‘S”: (A ⋂ B’), (A’ ⋂ B’), (A ⋂ B), and (A’ ⋂ B). These subsets are mutually disjoint sets because they are disjoint pairwise. For any pair combinations of the four sets, they remain disjoint. However, the probability of occurrence of any of these events depends on the occurrence of the other events in the sample space. In such cases where the probability of an event is dependent on the probability of the other events in the same sample space, the total probability theorem is used to find the probability of the event. 

How to State and Prove Total Probability Theorem:

The total probability theorem states that “if A1, A2, A3 …… An are the partitions of the sample space S such that the probability of none of these events is equal to zero, then the probability of an event ‘E’ occurring in such a sample space is given as:

P(E) = \[\sum_{i=1}^{n}\] P(A_i).P(E/A_i)

Total Probability Theorem Proof:

Consider a sample space shown in the above figure such that C1, C2, C3 ….. Cn is the partitions of the sample space ‘S’ such that Cp ⋂ Cq = ∅ (Null set). i.e. the partitions are disjoint when p ≠ q where p and q = 1, 2, 3, 4 …. n. It is also true that P (Cm) ≠ 0. i.e. no event in the sample space has a non zero probability. The sample space can be represented as: 

S = C1 U C2 U C3 U ………… U Cn → (1)

For any event “A’ in the sample space S, the event ‘A’ can be denoted as:

A = A ⋂ S

Substituting (1) in the above equation, the above equation gives

A = A ⋂ (C1 U C2 U C3 U ………… U Cn)

A = (A ⋂ C1) U (A ⋂ C2) U (A ⋂ C3) U ……… U (A ⋂ Cn)

It is clear that A ⋂ Cp and A ⋂ Cq are the subsets of Cp and Cq respectively. Hence it is true that 

A ⋂ Cp and A ⋂ Cq are also disjoint for p ≠ q. So, the probability of event ‘A’ can be calculated as:

P(A) = P [(A ⋂ C1) U (A ⋂ C2) U (A ⋂ C3) U ……… U (A ⋂ Cn)]

P(A) = P (A ⋂ C1) + P (A ⋂ C2) + P (A ⋂ C3) + ……… + P (A ⋂ Cn) → (3)

However, the multiplication rule of probability states that,

P (A ⋂ Cp) = P (Cp) . P (A | Cp)  → (4) 

Substituting (3) in (4), we get

P (A) =  P (C1) . P (A | C1) +  P (C2) . P (A | C2) +  P (C3) . P (A | C3) + …. +  P (Cn) . P (A | Cn)

The above equation can be shortly written as:

P(A) = \[\sum_{i=1}^{n}\] P(C_i).P(A/C_i)

Total Probability Theorem Examples: 

1. Sharon has three bags with 100 marbles in each bag. The first bag has 25 blue and 75 red marbles, the second bag has 40 blue and 60 red marbles, and the third bad consists of 55 blue and 45 red marbles. A marble is randomly chosen from one of the bags. What is the probability that it is a red marble? (Hint: Consider it as a total probability theorem example)

Solution:

Let ‘A’ represent the event of choosing a red marble and Ci represents the event of choosing the bag. From the given data, we can infer the following.

Probability of choosing a red marble from bag 1 is 

P (A | C1) = 75 / 100 = 0.75

Probability of choosing a red marble from bag 2 is 

P (A | C2) = 60 / 100 = 0.6

Probability of choosing a red marble from bag 3 is 

P (A | C3) = 45 / 100 = 0.45

The probability of choosing one among the three bags remains the same.

P (Ci) = ⅓ 

The formula representing total probability theorem proof is 

P(A) = \[\sum_{i=1}^{n}\] P(C_i).P(A/C_i)

So, the probability of choosing a red ball is calculated as:

P(A) = \[\sum_{i=1}^{3}\] P(C_i).P(A/C_i)

P(A) = P(C₁).P(A/C₁) + P(C₂).P(A/C₂) + P(C₃).P(A/C₃)

P(A) = ⅓(0.75) + ⅓(0.6) + ⅓(0.45) = 0.25 + 0.2 + 0.15 = 0.6

Therefore, the probability of choosing a red ball at a random from one of the three bags is 0.6

Fun Facts about Total Probability Theorem-proof:

• In the theory of probability, conditional probability is calculated when the occurrence of one event is possible only when the other event has already occurred.

• Marginal probability is calculated when the events in a sample space are independent of each other. The events do not depend on each other in case of marginal probability. 

• Total probability theorem-proof establishes a relationship between the conditional probability and the marginal probability and defines the probability of an event as the sum of the probabilities of other events in the sample space.