Knowledge that one event has occurred can change the probability of another. Let’s take our example of rolling a die and consider the following trivial example. If we know that an even number has been rolled, the probability it is odd is zero!😃
Here is a less trivial example.
Example 1.
$$ A=\{2,4,6\} \; \text{and} \; B=\{3,4,5,6\} $$
\(P(A) = \frac{3}{6} = \frac{1}{2}\) and \(P(B) = \frac{4}{6} = \frac{2}{3} \)
If we know that event B has occurred then the probability of event A changes since we know that a 2 has not been rolled. Mathematically we denote this new probability as \( P(A|B) \).
Clearly \(P(A|B) = \frac{1}{2}\) since A contains 2 elements from B which contains 4 elements.
How we calculate conditional expectations in general is given by the following expression
if \(P(B)\) > 0 \begin{align} P(A|B) = \frac{P(A \cap B)}{P(B)}\\ \end{align}
Intuitively independence of 2 events means that the knowledge that one event occurs does not affect the probability of the other event occurring or more precisely
Independent events
For two independent events \(A\) and \(B\), \(P(A|B) = P(A)\).
From above this gives
\begin{align}
P(A) P(B) = P(A \cap B)
\end{align}
Example 2.
Suppose you toss a coin twice. Let A be the event a head is tossed on the first go and B the event a head is tossed on the second go. Intuitively you would not expect the probabilities of the second toss to be affected by knowledge of the outcome of the first toss so we would expect A and B to be independent. Let’s check that A and B satisfy the definition of independence above.
\begin{align}
\Omega &=\{HH,HT,TH,TT\} \\
A &= \{HH,HT\} \\
B&=\{HH,TH\} \\
\end{align}
so \( P(A) = \frac{1}{2}\) \( P(B) = \frac{1}{2} \)
\( A \cap B = \{HH\} \) so \( P(A \cap B) = \frac{1}{4} \)
We have shown \( P(A \cap B) = P(A) P(B) \) by direct calculation. Therefore A and B are independent.
How about independence of more than 2 events?
Independence of more than two events
Let \(A_1,…,A_n \subseteq \Omega \) be events in a probability space \( \Omega \), the events are mutually independent if for every subset of indices \( I \subseteq \{1,…,n\}, I \neq \emptyset \) we have \( P_{i \in I}(\cap A_i) = \prod P(A_i) \)
Pairwise independence does not mean the events are mutually independent.
Example 3.
Consider these events when tossing a coin twice
\( A =\{\text{Heads on the first toss}\} \)
\( B = \{\text{Tails on the second toss}\} \)
\( C = \{\text{One Head in total}\} \)
Clearly \(A \cap B \) and \(C \) are not independent. However the sets A,B and C are pairwise independent.
Exercise Check this!!