Independence and Conditional Probability

October 25, 2008

Independence and Conditional Probability

Suppose that $(\Omega, \mathcal{F}, P)$ is a measure space, $A$ a measurable set with $P(A) \in (0, \infty)$ . We can create a new measure space $(A, \mathcal{F}_A, P_A)$ where

$\displaystyle \mathcal{F}_A = \{ S \cap A : S \in \mathcal{F} \}$

$\displaystyle P_A = \frac{P(E)}{P(A)}$

Note that $(A, \mathcal{F}_A, P_A)$ is a probability space, as $P_A(A) = 1$ , and any other set $E \in \mathcal{F}_A$ is a subset of $A$ .

Supposing $(\Omega, \mathcal{F}, P)$ is a probability space, we can use this new probability space in our definition of conditional probability. The probability $P_A(E)$ represents the probability of $E$ occurring, where we already know $A$ has occurred. Normally, instead of going through the trouble of writing out a new sigma-algebra and probability measure each time, we simply take $P(B|A)$ to be the probability of $B \cap A$ using the $P_A$ measure defined above. Of course, our measure and sigma-algebra are so simple that we can just write this in one line as

$\displaystyle P(B|A) = \frac{P(B \cap A)}{P(A)}$

We call this the probability of $B$ given $A$ . Now if $P(B|A) = P(B)$ , we say that $A$ and $B$ are independent events. If this is the case then we have

$\displaystyle P(B) = \frac{P(B \cap A)}{P(A)}$

$\displaystyle \implies P(A) P(B) = P(B \cap A)$

This is certainly a useful property as it makes proofs of interesting facts fall out easily when we consider sequences of independent random variables (a related idea) later.

Now consider the fact that $P(A|B) = \frac{P(A \cap B)}{P(B)}$ implies $P(A \cap B) = P(A|B)P(B)$ . Plugging into the formula for $P(B|A)$ we arrive at the following, known as Bayes’ theorem.