Mathematics Prelims

October 25, 2008

Independence and Conditional Probability

Filed under: Measure Theory,Probability Theory — cjohnson @ 6:39 pm

Suppose that (\Omega, \mathcal{F}, P) is a measure space, A a measurable set with P(A) \in (0, \infty).  We can create a new measure space (A, \mathcal{F}_A, P_A) where

\displaystyle \mathcal{F}_A = \{ S \cap A : S \in \mathcal{F} \}

\displaystyle P_A = \frac{P(E)}{P(A)}

Note that (A, \mathcal{F}_A, P_A) is a probability space, as P_A(A) = 1, and any other set E \in \mathcal{F}_A is a subset of A.

Supposing (\Omega, \mathcal{F}, P) is a probability space, we can use this new probability space in our definition of conditional probability.  The probability P_A(E) represents the probability of E occurring, where we already know A has occurred.  Normally, instead of going through the trouble of writing out a new sigma-algebra and probability measure each time, we simply take P(B|A) to be the probability of B \cap A using the P_A measure defined above.  Of course, our measure and sigma-algebra are so simple that we can just write this in one line as

\displaystyle P(B|A) = \frac{P(B \cap A)}{P(A)}

We call this the probability of B given A.  Now if P(B|A) = P(B), we say that A and B are independent events.  If this is the case then we have

\displaystyle P(B) = \frac{P(B \cap A)}{P(A)}

\displaystyle \implies P(A) P(B) = P(B \cap A)

This is certainly a useful property as it makes proofs of interesting facts fall out easily when we consider sequences of independent random variables (a related idea) later.

Now consider the fact that P(A|B) = \frac{P(A \cap B)}{P(B)} implies P(A \cap B) = P(A|B)P(B).  Plugging into the formula for P(B|A) we arrive at the following, known as Bayes’ theorem.

\displaystyle P(B|A) = \frac{P(B \cap A)}{P(A)} = \frac{P(A|B) P(B)}{P(A)}

Note that P(A|B) \neq P(B|A).


Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at

%d bloggers like this: