The undergraduate probability course, concretely: random variables as numerical outcomes of experiments, the distribution zoo everyone must know, and the two limit theorems — Law of Large Numbers and Central Limit Theorem — that explain why averages stabilize and why the normal distribution is everywhere. The measure-theoretic foundations live in Probability Theory; this page is the working layer above them.
A random variable X assigns a number to each outcome. Discrete variables are described by a pmf p(x) = P(X = x); continuous ones by a pdf f(x) with P(a \le X \le b) = \int_a^b f (individual points have probability zero); both by the cdf F(x) = P(X \le x), the universal description. Expectation is the probability-weighted average, E[X] = \sum x\,p(x) or \int x f(x)\,dx; it is linear unconditionally — E[X + Y] = E[X] + E[Y] with no independence needed — which is the single most exploited fact in the subject (indicator-variable arguments live on it). Variance \mathrm{Var}(X) = E[(X - \mu)^2] = E[X^2] - \mu^2 measures spread; it adds across independent variables, and \mathrm{Var}(aX) = a^2 \mathrm{Var}(X) — the deep reason averaging n independent copies shrinks the standard deviation by \sqrt{n}, from which most of statistics follows.
Each standard distribution is the answer to a modeling question:
| Distribution | Models | Mean / Variance |
|---|---|---|
| Bernoulli(p) | One yes/no trial | p / p(1-p) |
| Binomial(n, p) | Successes in n independent trials | np / np(1-p) |
| Geometric(p) | Trials until first success (memoryless, discrete) | 1/p / (1-p)/p^2 |
| Poisson(\lambda) | Counts of rare events at fixed rate | \lambda / \lambda |
| Uniform(a,b) | Complete ignorance on an interval | (a+b)/2 / (b-a)^2/12 |
| Exponential(\lambda) | Waiting time between Poisson events (memoryless, continuous) | 1/\lambda / 1/\lambda^2 |
| Normal(\mu, \sigma^2) | Sums of many small independent effects | \mu / \sigma^2 |
| Gamma, Beta | Waiting times for k events; probabilities of probabilities | — |
The relationships matter more than the rows: Binomial with n large and np moderate \to Poisson; Poisson event counts pair with Exponential event gaps (the two faces of a Poisson process, and the memorylessness of Exponential/Geometric is why Markov and queueing models are tractable); everything with finite variance flows toward the Normal by the CLT.
Joint distributions describe several variables at once; marginals integrate the others out; X and Y are independent iff the joint factors into marginals. Covariance \mathrm{Cov}(X,Y) = E[XY] - E[X]E[Y] and correlation \rho \in [-1, 1] measure linear association — zero correlation does not imply independence (take X symmetric and Y = X^2). Conditioning is the modeling verb: E[X] = E[E[X \mid Y]] (the tower rule) computes expectations by conditioning on what you wish you knew, and is the workhorse of both applied probability and Bayesian reasoning.
Law of Large Numbers: for iid X_i with mean \mu, the sample average \bar{X}_n \to \mu (in probability / almost surely). Averages stabilize — the license behind Monte Carlo methods, insurance, and casinos.
Central Limit Theorem: if additionally \mathrm{Var}(X_i) = \sigma^2 < \infty, then
regardless of the shape of the X_i's distribution. The LLN says where averages go; the CLT says how they fluctuate on the way — normally, at scale \sigma/\sqrt{n}. This universality is why the normal distribution appears in measurement error, sampling distributions, and every confidence interval in statistical inference; the slick proof is one line in characteristic functions (Fourier transforms of distributions — products of transforms are sums of variables). Caveats that matter in practice: heavy tails (infinite variance) break the CLT — sums then converge to stable laws instead, the starting point of robust statistics and of skepticism about normal-tail risk models in finance.