Overview

The Weak Law of Large Numbers (WLLN) is a fundamental theorem in probability theory that describes the behavior of the average of a large number of independent and identically distributed (i.i.d.) random variables.

Statement

Let $X_1, X_2, ..., X_n$ be a sequence of i.i.d. random variables with finite expected value $\mathbb{E}[X_i] = \mu$ and finite variance $\mathbb{V}(X_i) = \sigma^2$.

Define the sample mean as:

$$\bar{X}_n = \frac{1}{n}\sum_{i=1}^{n} X_i$$

The Weak Law of Large Numbers states that for any $\epsilon > 0$:

$$\lim_{n \to \infty} P(|\bar{X}_n - \mu| \geq \epsilon) = 0$$

Or equivalently:

$$\bar{X}_n \xrightarrow{P} \mu \text{ as } n \to \infty$$

This is called convergence in probability.

Proof Using Chebyshev's Inequality

Step 1: Compute the expected value of $\bar{X}_n$

$$\mathbb{E}[\bar{X}_n] = \mathbb{E}\left[\frac{1}{n}\sum_{i=1}^{n} X_i\right] = \frac{1}{n}\sum_{i=1}^{n} \mathbb{E}[X_i] = \frac{1}{n} \cdot n\mu = \mu$$

Step 2: Compute the variance of $\bar{X}_n$

Since the $X_i$ are independent:

$$\mathbb{V}(\bar{X}_n) = \mathbb{V}\left(\frac{1}{n}\sum_{i=1}^{n} X_i\right) = \frac{1}{n^2}\sum_{i=1}^{n} \mathbb{V}(X_i) = \frac{1}{n^2} \cdot n\sigma^2 = \frac{\sigma^2}{n}$$

Step 3: Apply Chebyshev's Inequality

For any $\epsilon > 0$:

$$P(|\bar{X}_n - \mu| \geq \epsilon) \leq \frac{\mathbb{V}(\bar{X}_n)}{\epsilon^2} = \frac{\sigma^2}{n\epsilon^2}$$

Step 4: Take the limit

$$\lim_{n \to \infty} P(|\bar{X}_n - \mu| \geq \epsilon) \leq \lim_{n \to \infty} \frac{\sigma^2}{n\epsilon^2} = 0$$

Since probabilities are non-negative, the limit must be exactly 0.

Interpretation

The WLLN tells us that as the sample size increases, the sample mean $\bar{X}_n$ converges in probability to the true mean $\mu$. This means that for large $n$, the sample mean will be close to the population mean with high probability.

Applications

1. Statistics: Justifies using sample averages to estimate population parameters
2. Gambling: Explains why casinos have consistent profits
3. Insurance: Forms the basis for risk pooling and premium calculation
4. Quality Control: Validates using sample means to monitor processes

Relationship to Strong Law

The Strong Law of Large Numbers (SLLN) states almost sure convergence:

$$P\left(\lim_{n \to \infty} \bar{X}_n = \mu\right) = 1$$

The SLLN implies the WLLN, but not conversely. The WLLN is sufficient for most practical applications.

Example: Coin Flipping

For a fair coin with $P(\text{Heads}) = 0.5$, let $X_i = 1$ if the $i$-th flip is heads, $0$ otherwise.

• $\mu = \mathbb{E}[X_i] = 0.5$
• $\sigma^2 = \mathbb{V}(X_i) = 0.25$

The proportion of heads in $n$ flips is $\bar{X}_n$. By WLLN:

$$\lim_{n \to \infty} P(|\bar{X}_n - 0.5| \geq \epsilon) = 0$$

This means that as we flip the coin more times, the proportion of heads will approach 0.5.

For more details on expectation and variance, see Expectation and Variance.