Normal distribution

Also known as the Gaussian distribution, the normal distribution is the bell-shaped probability model used when many small, roughly independent effects add together.

A continuous random variable $X$ is normal with mean $\mu$ and variance ${\sigma }^2$, written

$$X\sim \mathcal{N}(\mu ,{\sigma }^2),$$

when its probability density is

$$f(x)=\frac{1}{\sigma \sqrt{2\pi }}\exp \left(-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right).$$

Key ideas:

• $\mu$ sets the centre and equals the mean, median, and mode.
• $\sigma$ sets the spread; larger $\sigma$ means a wider, flatter curve.
• The total area under the density curve is $1$, so probabilities are areas.
• Standardisation converts any normal variable to a standard normal:

$$Z=\frac{X-\mu }{\sigma }\sim \mathcal{N}(0,1).$$

The empirical rule is a useful memory aid: about 68% of observations lie within $1\sigma$, 95% within $2\sigma$, and 99.7% within $3\sigma$ of $\mu$.

The normal distribution appears in measurement error, regression residuals, diffusion models, and as an approximation justified by the central limit theorem. It is central to Stats Equations and definitions, Stats Examples, and many ML loss functions, especially least-squares regression.

Standard normal

The standard normal distribution has mean $0$ and variance $1$:

$$Z\sim \mathcal{N}(0,1).$$

Any normal variable can be standardised by

Z=rac{X-\mu}{\sigma}.

This converts questions about $X\sim \mathcal{N}(\mu ,{\sigma }^2)$ into questions about the standard normal CDF. In practice this is what z-scores are doing.