Stats Key concepts

Probability

Probability assigns numbers between $0$ and $1$ to events. For events $A$ and $B$,

$$P(A\cup B)=P(A)+P(B)-P(A\cap B).$$

Conditional probability is

$$P(A\mid B)=\frac{P(A\cap B)}{P(B)},$$

which leads to Bayes’ rule:

$$P(A\mid B)=\frac{P(B\mid A)P(A)}{P(B)}.$$

Random variables

A random variable is a numerical outcome of a random process. Discrete variables use probability mass functions; continuous variables use probability densities, where probabilities are integrals/areas.

Expectation and variance

The expectation $E[X]$ is the long-run average value. The variance

$$\mathrm{Var}(X)=E[(X-E[X]{)}^2]$$

measures spread. Standard deviation is the square root of variance.

Samples and estimators

A sample is finite data from a population or process. An estimator is a rule that turns data into a parameter estimate, such as $\bar{x}$ for a mean. Bias, variance, and consistency describe estimator quality.

Likelihood

For parameter $\theta$ and observed data $x$, the likelihood $L(\theta \mid x)$ treats the data as fixed and asks which parameter values make them plausible. Maximum likelihood estimation chooses the $\theta$ with largest likelihood.

Inference

Confidence intervals, hypothesis tests, and Bayesian posteriors are different ways to quantify uncertainty. Normal distribution approximations are common, but checking assumptions matters.