Conditioning and Bayes' Rule

MIT 6.041 Probabilistic Systems Analysis and Applied Probability, lecture 2, explained in plain English

Want this lecture to stick? In LessonBound you get checked practice questions for every part, can ask about anything you don't get, and can listen with your screen off. Free, no ads.

Learn this lecture free

Watch or read the original lecture: MIT OpenCourseWare.

Part 1 · 0:00–10:00

This part explains how probabilities are set up for an experiment, why adding up probabilities works, and why a probability of zero does not always mean impossible.

Key ideas

A model of a random experiment starts with a sample space, which is the list of all possible outcomes. The outcomes must not overlap, and together they must cover every possible result. 1:25
Probabilities are assigned to events, which are subsets of the sample space. For events that do not overlap, the probability that either one happens is the sum of their separate probabilities. 3:17
The adding rule works for a sequence of sets, but it does not work for every way of combining sets. In the unit square example, the sum of zeros argument breaks because individual points cannot form a sequence that covers the whole square. 6:48
Single points in a continuous model have zero probability, yet one of them always happens. So zero probability means extremely unlikely, not impossible. 9:21

Words to know

sample space: The full list of everything that could possibly happen when you do an experiment.
mutually exclusive: Two outcomes that cannot both happen at once. Only one of them can occur.
disjoint: Sets with nothing in common. They do not overlap at all.
uncountable: So large that its elements cannot be lined up in a list, like the points inside a square.

Example

Imagine throwing a dart at a board so that it lands at a random spot. The chance it hits one exact point is zero, but the dart still lands somewhere. That is the same idea as the unit square: any one exact outcome has zero probability, but something still happens.

3 checked practice questions for this part are in the app.

Part 2 · 10:00–20:00

When someone gives you new information, you should update your beliefs, and conditional probability gives the revised chances.

Key ideas

New information about an experiment should change what you believe happened or might happen. You should revise your beliefs when you learn something new. 12:33
A conditional probability is the new probability that applies after you learn some information. It is the probability once your beliefs have been revised. 12:54
To find the conditional probability of A given B, divide the probability that both A and B happen by the probability of B. This gives the fraction of B's probability that also includes A. 16:43
Think of probabilities as frequencies. Look only at the experiments where B happens, then ask what fraction of those also have A. 18:14
Conditional probabilities are still real probabilities. They follow the same rules as ordinary probabilities, but in a new world where the known event has happened. 19:02

Words to know

conditional probability: The chance that something happens, after you have been told that some other thing has already happened.
sample space: The full set of all the possible outcomes of an experiment.
event: A set of outcomes, or a group of possible results, that you are asking about.
disjoint: Two events that cannot both happen at the same time, because they share no outcomes.

Example

Imagine a bag with some red and blue marbles, and you reach in without looking. At first, you guess the chances of each color. If someone tells you the marble you pulled out is red, you change your guess right away, because you now know more. The chance of blue drops to zero, and only the red marbles matter now.

3 checked practice questions for this part are in the app.

Part 3 · 20:00–30:00

This part shows how learning new information changes probabilities, and how to work out chances step by step using a radar and airplane example.

Key ideas

A conditional probability is still a normal probability law. It just works inside a new, smaller world where we already know some event B happened. 20:00
When every outcome starts out equally likely, the outcomes left after learning B stay equally likely. So if B has five equal outcomes, each one has probability 1/5. 23:15
To find the chance of two things both happening, multiply the chance of the first by the chance of the second, given the first. You can draw this as a tree, with each branch carrying its own probability. 27:38
If an event can happen in more than one way, add the probabilities of those separate paths. The radar example adds the plane-detected path and the false-alarm path. 28:46
The lesson ends with the key question of how likely a plane is, given that the radar showed a blip. The lesson text cuts off before giving the answer, so this part does not explain it. 29:52

Words to know

conditional probability: The chance of something happening after you are told that another event already happened.
intersection: The outcomes that belong to two events at the same time, like the overlap of two circles.
uniform distribution: A setup where every possible outcome has the same chance of happening.
false alarm: When a detector shows something is there even though nothing is actually there.

Example

Imagine you are told a card drawn from a deck is red, and you want the chance it is a heart. Only the red cards are left in your new world, so you only look at those. The same idea works for the radar: once you know a plane is there, you only look at what the radar does in that case.

3 checked practice questions for this part are in the app.

Part 4 · 30:00–40:00

This part teaches how to use conditional probability, why a good test can still give mostly false alarms, and two basic tools for working out probabilities in multi-step problems.

Key ideas

Conditional probability asks how likely one thing is when you already know another thing happened. You find it by dividing the chance that both happen by the chance of the known event. 30:19
A radar can be very accurate and still give mostly false alarms when real planes are rare. In the lesson's example, a radar reading means only about a 30% chance that a plane is really there. 31:08
To find the chance that several events all happen in order, you multiply the conditional probabilities along each branch of a tree diagram. 34:55
To find the total chance of an event, you split all possible scenarios into separate groups that do not overlap, then add up the chance of the event within each group. 38:08

Words to know

conditional probability: The chance that something happens, given that you already know something else has happened.
false alarm: When a test or sensor says something is there, but nothing is actually there.
composite event: An event made up of two or more smaller events that all happen together.
partition: Splitting all possible outcomes into separate groups that do not overlap, so every outcome falls into exactly one group.

Example

Imagine a smoke alarm that is very good but goes off mostly when someone burns toast, and real fires are rare. When it beeps, you may still be more likely to be facing toast than a real fire, even though the alarm is accurate most of the time. This matches the radar example: when the thing you are looking for is rare, a positive result is often a false alarm.

3 checked practice questions for this part are in the app.

Part 5 · 40:00–51:10

This part shows how to find the chance of an event that can happen in several ways, and how to update your beliefs about the causes once you see the event happen.

Key ideas

An event can happen in several separate ways, called scenarios. To find its total probability, add up the chance of the event happening together with each scenario. 40:43
Each scenario's chance is found by multiplying its own probability by the chance the event happens inside that scenario. The scenario probabilities add up to one, so the total is a weighted average. 41:27
When equally likely scenarios are used, the total probability is just the simple average of the event's probabilities in each scenario. 44:25
Bayes' rule starts with prior beliefs about each scenario, then uses what you observed to revise them. In the radar example, a 5% chance of a plane became 34% after the radar recorded something. 46:15
The method reverses the direction of a cause-and-effect model: you know the effect was seen and work out how likely each cause was. 49:35

Words to know

partition: A set of separate groups that do not overlap and together cover every possible outcome, like splitting a pizza into slices that make up the whole pizza.
conditional probability: The chance that something happens, given that you already know something else happened. It is the chance of one thing inside a smaller, known situation.
total probability: The overall chance of an event, found by adding up its chances across all the different ways it could happen.
prior probabilities: Your starting beliefs about how likely each scenario is, before you get any new information.

Example

Imagine you come home and find the grass wet. Before you looked, you had a rough idea of how likely it was to have rained. Seeing the wet grass is new information, so you update your belief: rain is now more likely than it was before, though sprinklers could also explain it. This is the same idea as revising a starting guess after you see new evidence.

3 checked practice questions for this part are in the app.

← Lecture 1: Probability Models and Axioms · Lecture 5: Discrete Random Variables; Probability Mass Functions; Expectations →

Notes written by AI (Claude) from the lecture's official transcript, so check anything important against the lecture. MIT OpenCourseWare · CC BY-NC-SA 4.0 · not affiliated with or endorsed by MIT. These notes are shared under the same license.

LessonBound · Free lecture notes · Privacy · Terms · New York