Introduction
This chapter formalises random variables (introduced in Chapter 2) into complete probability distributions, and covers the specific named distributions that recur constantly throughout applied statistics.
1. Discrete and continuous random variables
A discrete random variable takes a countable set of distinct values (e.g., number of successes in a fixed number of trials); a continuous random variable can take any value within an interval (e.g., a measured height). This distinction determines which mathematical tools apply: discrete variables use a probability mass function and sums, while continuous variables use a probability density function and integrals.
2. Probability mass/density and cumulative distribution functions
The probability mass function (PMF), for a discrete variable, gives the probability of each specific value directly. The probability density function (PDF), for a continuous variable, does not give probabilities directly (indeed, the probability of any single exact value is zero for a continuous variable) — instead, the probability of the variable falling within a range is the area under the PDF curve over that range. The cumulative distribution function (CDF), F(x) = P(X ≤ x), applies to both types and gives the probability of the variable being at or below a given value — for continuous variables, the PDF is the derivative of the CDF.
3. Expectation and variance of random variables
The expectation (mean) and variance of a random variable generalise the descriptive-statistics concepts of Chapter 1 to a theoretical probability distribution rather than an observed dataset — computed as a probability-weighted sum (discrete) or integral (continuous) rather than a simple average over collected data, but interpreted identically as a measure of central tendency and spread respectively.
4. Important distributions: binomial, Poisson, and normal
- Binomial distribution: models the number of successes in a fixed number (n) of independent trials, each with the same success probability (p) — applicable whenever an experiment consists of repeated identical yes/no trials (mean = np, variance = np(1−p)).
- Poisson distribution: models the number of events occurring in a fixed interval of time or space, when events happen independently at a constant average rate (λ) — often used as an approximation to the binomial when n is large and p is small (rare events over many trials), with mean and variance both equal to λ.
- Normal (Gaussian) distribution: the symmetric, bell-shaped continuous distribution central to statistical theory, fully characterised by its mean (μ) and standard deviation (σ); its overwhelming importance comes from the Central Limit Theorem (Chapter 6), which shows that averages of many independent random variables tend toward a normal distribution regardless of the original variables’ own distribution.
5. Approximations and applications
Under specific conditions, one distribution can be used to approximate another where exact calculation would be cumbersome — the Poisson approximation to the binomial (above), and the normal approximation to the binomial (valid when np and n(1−p) are both sufficiently large, since the binomial distribution’s shape becomes increasingly bell-like as n grows) are the two most commonly applied approximations at this syllabus level. Correctly identifying which named distribution actually fits a given real-world scenario’s assumptions — rather than defaulting to whichever formula is most familiar — is the central applied skill this chapter tests.
Key Takeaways
- Discrete variables use a probability mass function and sums; continuous variables use a probability density function and integrals, with the CDF applying uniformly to both.
- Expectation and variance of a random variable are the theoretical, probability-weighted analogues of the sample mean and variance from descriptive statistics.
- The binomial (fixed trials, success/failure), Poisson (rate-based event counts), and normal (continuous, bell-shaped) distributions are the three most important named distributions, each with specific defining assumptions.
- The Poisson can approximate the binomial for rare events over many trials; the normal can approximate the binomial when both np and n(1−p) are sufficiently large.
- Correctly matching a real scenario’s assumptions to the appropriate distribution — not just applying a familiar formula — is the essential applied skill.