Chapter 13 — Probability and Statistics

Probability models uncertainty; statistics summarizes data. This chapter covers random variables, common distributions, descriptive statistics, and two cornerstone results — the Law of Large Numbers and the Central Limit Theorem.

Learning Objectives

Compute descriptive statistics.
Work with common distributions via scipy.stats.
Understand PMF/PDF and CDF.
Compute covariance and correlation.
Demonstrate the Law of Large Numbers by simulation.
Demonstrate the Central Limit Theorem by simulation.

Prerequisites / Imports

In [1]:

import numpy as np
import matplotlib.pyplot as plt
from scipy import stats

1 Descriptive Statistics

Mean, median, variance, standard deviation, and quantiles summarize a sample.

In [1]:

data = np.array([12, 15, 14, 19, 22, 18, 14, 21, 30, 16])
print('mean:', data.mean())
print('median:', np.median(data))
print('variance:', data.var(ddof=1).round(2))
print('std:', data.std(ddof=1).round(2))
print('quartiles:', np.percentile(data, [25, 50, 75]))

mean: 18.1
median: 17.0
variance: 27.88
std: 5.28
quartiles: [14.25 17.   20.5 ]

2 Random Variables and Distributions

scipy.stats provides distributions with pdf/pmf/cdf and random sampling.

In [1]:

# Normal distribution: PDF and CDF at a few points
xs = np.linspace(-4, 4, 5)
print('normal pdf:', np.round(stats.norm.pdf(xs), 4))
print('normal cdf:', np.round(stats.norm.cdf(xs), 4))
print('P(-1 < Z < 1) =', round(stats.norm.cdf(1) - stats.norm.cdf(-1), 4))

normal pdf: [1.000e-04 5.400e-02 3.989e-01 5.400e-02 1.000e-04]
normal cdf: [0.     0.0228 0.5    0.9772 1.    ]
P(-1 < Z < 1) = 0.6827

3 The Normal Distribution

The bell curve is ubiquitous due to the Central Limit Theorem.

In [1]:

x = np.linspace(-4, 4, 200)
plt.figure(figsize=(7,4))
plt.plot(x, stats.norm.pdf(x), label='pdf')
plt.fill_between(x, stats.norm.pdf(x), where=(x>-1)&(x<1), alpha=0.3, label='within 1 std')
plt.title('Standard normal distribution'); plt.legend()
plt.show()

4 Discrete Distributions

The binomial counts successes in $n$ trials; Poisson counts events in an interval.

In [1]:

print('Binomial(n=10,p=0.5) P(5 heads):', stats.binom.pmf(5, 10, 0.5).round(4))
print('Poisson(mu=3) P(2 events):', stats.poisson.pmf(2, 3).round(4))

ks = np.arange(0, 11)
plt.figure(figsize=(7,4))
plt.bar(ks, stats.binom.pmf(ks, 10, 0.5), alpha=0.6, label='Binom(10,0.5)')
plt.title('Binomial PMF'); plt.legend()
plt.show()

Binomial(n=10,p=0.5) P(5 heads): 0.2461
Poisson(mu=3) P(2 events): 0.224

5 Covariance and Correlation

Covariance measures joint variability; correlation standardizes it to $[-1, 1]$.

In [1]:

rng = np.random.default_rng(1)
x = rng.normal(0, 1, 200)
y = 2 * x + rng.normal(0, 1, 200)
print('covariance:', np.cov(x, y)[0, 1].round(3))
print('correlation:', np.corrcoef(x, y)[0, 1].round(3))

covariance: 1.752
correlation: 0.903

6 Law of Large Numbers

As the sample size grows, the sample mean converges to the true mean.

In [1]:

rng = np.random.default_rng(2)
sample = rng.random(10000)
running_mean = np.cumsum(sample) / np.arange(1, len(sample) + 1)
plt.figure(figsize=(7,4))
plt.plot(running_mean, label='running mean')
plt.axhline(0.5, color='red', linestyle='--', label='true mean 0.5')
plt.title('Law of Large Numbers'); plt.xlabel('n'); plt.legend()
plt.show()

7 Central Limit Theorem

The mean of many samples is approximately normal, whatever the original distribution.

In [1]:

rng = np.random.default_rng(3)
# Uniform population (clearly non-normal)
means = [rng.uniform(0, 1, 30).mean() for _ in range(5000)]
plt.figure(figsize=(7,4))
plt.hist(means, bins=40, density=True, alpha=0.6, color='steelblue')
overlay = np.linspace(0.3, 0.7, 200)
plt.plot(overlay, stats.norm.pdf(overlay, loc=0.5, scale=1/np.sqrt(12*30)), 'r-', label='normal approx')
plt.title('Central Limit Theorem: distribution of sample means'); plt.legend()
plt.show()

Case Study: Simulating a Dice Game

Estimate the expected sum of two dice by Monte Carlo simulation and compare to the exact value 7.

In [1]:

rng = np.random.default_rng(4)
rolls = rng.integers(1, 7, size=(100000, 2))
sums = rolls.sum(axis=1)
print('simulated mean sum:', sums.mean().round(4))
print('exact expected sum: 7')
print('simulated P(sum=7):', (sums == 7).mean().round(4), ' exact:', round(6/36, 4))

plt.figure(figsize=(7,4))
plt.hist(sums, bins=np.arange(1.5, 13.5, 1), density=True, alpha=0.6, color='coral', edgecolor='black')
plt.title('Sum of two dice (simulation)'); plt.xlabel('sum'); plt.show()

simulated mean sum: 7.0039
exact expected sum: 7
simulated P(sum=7): 0.1682  exact: 0.1667

Exercises

Compute the mean, median, and standard deviation of 20 random normal values.
Plot the PDF of a normal distribution with mean 100 and standard deviation 15.
Compute $P(X \le 5)$ for $X \sim \text{Binom}(10, 0.4)$.
Simulate 1000 Poisson(mu=4) values and plot a histogram.
Generate correlated data and compute its correlation coefficient.
Reproduce the Law of Large Numbers plot for a fair die.
Demonstrate the CLT using an exponential population.
Estimate $\pi$ by Monte Carlo using random points in a unit square.
Explain the difference between covariance and correlation.
State in one sentence why the CLT matters for data science.

Python Data Science: From Foundations to Applications — Chapter 13