Lecture notes · Probability · Lecture 20

Central Limit Theorem

MIT 6.041 Probabilistic Systems Analysis and Applied Probability, lecture 20, explained in plain English

Want this lecture to stick? In LessonBound you get checked practice questions for every part, can ask about anything you don't get, and can listen with your screen off. Free, no ads.

Learn this lecture free

Watch or read the original lecture: MIT OpenCourseWare.

Part 1 · 0:00–10:00

This part explains the central limit theorem, which says that adding up many random pieces produces a sum whose standardized shape looks like a normal curve.

Key ideas

The setup is a large number of independent, identically distributed random variables, meaning each one has the same distribution and none depends on the others. We look at their sum and ask what its distribution looks like. 0:45
The raw sum spreads out more and more as n grows, so it has no useful limit by itself. We first standardize it by subtracting the mean and dividing by the standard deviation, which gives a result with mean 0 and variance 1. 1:28
As n grows, the standardized sum settles into the shape of a standard normal random variable. The theorem is about probabilities matching the normal table, which is a quick shortcut compared with computing the exact answer. 2:33
The theorem is universal: it works for almost any distribution with a finite mean and finite variance. To use it, you only need to know those two numbers. 3:42
The theorem is about cumulative distribution functions (CDFs) converging, not about the shapes of PMFs or PDFs matching. The lecturer's example showed two very different-looking PMFs whose CDFs were both close to the normal CDF. 7:25

Words to know

standard deviation: A number that measures how spread out values are around the average. It is the square root of the variance.
variance: A number that measures how far values typically spread from their average. Bigger variance means more spread.
cumulative distribution function (CDF): A running total of probability. For any number c, it tells you the chance that a random value is less than or equal to c.
standard normal: A normal (bell-shaped) distribution with mean 0 and variance 1. Its probabilities are listed in standard tables.

Example

Think of a dust speck floating in a liquid, getting bumped by many tiny molecules from every direction. Each bump is small and random, but after one second the speck's total sideways movement follows a bell-shaped pattern. That is the kind of noisy total the central limit theorem describes.

3 checked practice questions for this part are in the app.

Part 2 · 10:00–20:00

This part explains how to use the central limit theorem in practice, including how large a sample needs to be and how a polling problem uses it.

Key ideas

The central limit theorem is stated about the standardized total Zn, but in practice you can treat the sum Sn as if it were normal. This works because Zn and Sn are linked by a simple linear formula, and linear functions of normal values are normal. 10:22
The theorem is a limit result, but it works well for fairly small samples. The lecturer suggests n around 15 to 20 often gives good answers, though no hard guarantee is given. 11:23
How good the approximation is depends on the shape of the starting distribution. If the starting shape is close to normal, a small n is enough. If it is very different, you need a larger n. 12:13
A symmetric starting shape, like a uniform one, gives a normal-looking sum quickly. A lopsided starting shape, like a truncated geometric one, keeps some lopsidedness even at n = 16, and it looks close to normal around n = 32. 13:44
In the polling example, the true standard deviation is unknown and only known to be less than one half. So the lecturer uses the value one half, which is a conservative choice that gives a safe estimate. 19:20

Words to know

standard normal: A normal bell-shaped distribution with an average of 0 and a spread of 1. Standardizing a total turns it into this shape so we can look up probabilities.
limit theorem: A rule about what happens as the number of samples gets very large. Here it is used as a good guide even when the sample size is small.
convolution: A way of combining two probability patterns to find the pattern for their sum. The lecturer uses it to add more and more random values together.
conservative: Choosing a careful, cautious value that makes the result safe. Here, using one half for the unknown spread means the estimate is not too optimistic.

Example

Imagine rolling many dice and adding up the totals. A single die gives a flat, even chance for each face, but the total of many dice piles up in the middle and looks like a bell curve. This shows how adding many separate random values can produce a normal-looking result.

3 checked practice questions for this part are in the app.

Part 3 · 20:00–30:00

This part shows how to use the Central Limit Theorem to estimate error chances and to work backwards to find a sample size that meets a target error rate.

Key ideas

The sigma (spread) of the population is unknown, so the lecturer uses a safe upper bound. That bound is a bigger number, so the real error chance is no higher than the one we calculate. 20:21
Using a sample of 10,000, the calculated chance of error is about 4.5%. That is better than the 5% target, so a smaller sample might work. 23:02
Normal tables give the chance of being less than a value, so to find the chance of being bigger, you subtract from 1. 22:13
Working backwards, a sample of 9,604 gives an error chance of exactly 5%, which is the target. So the theorem can answer both 'what is the error chance?' and 'how big a sample do I need?' 26:47
The same method works for a binomial count, which is a sum of yes-or-no trials. Its mean is n times p, and its variance is n times p times (1 minus p), so we can standardize it. 29:19

Words to know

Standard normal: A bell-shaped curve with an average of 0 and a spread of 1. Tables give its probabilities.
Upper bound: A number that is at least as big as the real value. Using it makes sure we do not underestimate the risk.
Bernoulli: A yes-or-no trial, where success has probability p and failure has probability 1 minus p.
Binomial: The distribution of how many successes you get in a fixed number of yes-or-no trials.

Example

Think of a poll that asks 36 people a yes-or-no question. The number who say yes is a binomial count, and the Central Limit Theorem lets us estimate how likely it is to land in a certain range without checking every outcome. The average count is 18 and the typical spread is 3, so a count below 21 is fairly likely.

3 checked practice questions for this part are in the app.

Part 4 · 30:00–40:00

This part shows how the Central Limit Theorem gives a quick normal-curve estimate of a binomial probability, and how adding a half (1/2) makes that estimate much more accurate.

Key ideas

The Central Limit Theorem lets you pretend a binomial count is normal, using the same mean and variance. Here that gives a rough answer of 0.43 for P(Sn is less than 21), while the exact answer is 0.8785. 30:44
Because Sn can only take whole-number values, the event 'Sn is less than or equal to 21' is the same as 'Sn is less than 21.5'. Using 21.5 as the cutoff gives an answer of 0.879, which is very close to the exact value. 34:46
The same half-step trick works for a single value. To estimate P(Sn = 19), you take the normal area between 18.5 and 19.5, because a normal curve gives zero probability at any exact point. 37:37
Each whole-number value gets the normal area around it. Whatever falls between the midpoints belongs to that value, so this gives a good estimate of the bar height in the binomial chart. 38:25
The Central Limit Theorem was first noticed for binomial counts when p is 1/2, and the idea was later extended to other values of p. 39:27

Words to know

standard deviation: A number that tells you how spread out values are around the average. Dividing by it turns a value into a standard score.
binomial distribution: The pattern of how often you get a yes or success over a fixed number of tries, when each try has the same chance of success.
PMF: Probability mass function. A list or chart that gives the chance of each separate whole-number outcome.
standard normal: A normal bell-shaped curve with an average of 0 and a spread of 1. Tables let you look up areas under it.

Example

Imagine a bag with many coins, and you count heads after tossing them all. The count can only be a whole number, like 18 or 19, but the bell curve is smooth. To estimate the chance of getting 19 or fewer, you give each whole number a slice of the curve, with the boundary halfway between neighbors, like 18.5 and 19.5.

3 checked practice questions for this part are in the app.

Part 5 · 40:00–51:20

This part shows how to estimate a binomial probability with a normal curve, why a Poisson puzzle seems to break the central limit theorem, and how to choose between two approximations.

Key ideas

To find the chance that a count is exactly 19, you treat it as the chance of landing in an interval from 18.5 to 19.5. This half-unit adjustment makes the normal curve line up better with the counting steps. 40:21
You first shift and scale the count so it becomes a standard variable Z. Then you use the standard normal curve and subtract two cumulative values (CDF numbers) to get the probability of the interval. 40:43
The central limit theorem is about cumulative probabilities, not single-point probabilities in general. For a binomial, though, the normal approximation with the 1/2 correction is still very good even for one exact value. 42:07
The Poisson puzzle fails because the pieces change as you cut the interval finer. The central limit theorem needs the distribution of each piece to stay fixed while you add more of them. 46:29
A binomial can be approximated in two valid ways: a normal approximation when p is fixed and n is large, or a Poisson approximation when n is large, p is tiny, and n times p stays small. As a rough guide, use the normal approximation when n times p is above about 10 or 20. 50:36

Words to know

Standard deviation: A number that shows how spread out a set of values is around the average. Dividing by it puts different problems on the same scale.
Bernoulli: A single trial with only two results, such as success or failure. A binomial count is built by adding many of these together.
Poisson process: A way of modeling events that happen at random over time, such as arrivals, at a steady average rate.
Asymptotics: The study of what happens to a formula when a number, like n, gets very large or goes toward a limit.

Example

Imagine flipping a coin 500 times and counting the heads. The count clusters around the middle in a bell shape, so you can estimate the chance of any exact count using a smooth curve instead of working out every single term. The lesson says the normal curve works well here because n times p is large.

3 checked practice questions for this part are in the app.

← Lecture 19: Weak Law of Large Numbers · Lecture 21: Bayesian Statistical Inference I →

Notes written by AI (Claude) from the lecture's official transcript, so check anything important against the lecture. MIT OpenCourseWare · CC BY-NC-SA 4.0 · not affiliated with or endorsed by MIT. These notes are shared under the same license.

LessonBound · Free lecture notes · Privacy · Terms · New York