A free, self-paced textbook in 8 chapters. Read a chapter, then drill it with the 120 companion flashcards using spaced repetition.
A probability distribution is a rule that assigns probabilities to outcomes in a sample space, with the total probability summing (for discrete cases) or integrating (for continuou...
A family of closely related discrete distributions arises from independent Bernoulli trials. The Bernoulli distribution itself models a single binary outcome: \(P(X = 1) = p\), \(P...
Many continuous distributions arise from natural generative processes. The Uniform distribution on \([a, b]\) has constant density \(f(x) = 1/(b - a)\) over the interval, with mean...
The Normal (Gaussian) distribution is the centerpiece of continuous probability for ML, with PDF \(f(x) = \frac{1}{\sigma\sqrt{2\pi}} \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)\...
Parameter estimation can be framed either in frequentist or Bayesian terms. Maximum likelihood estimation (MLE) chooses parameters \(\theta\) to maximize the log-likelihood \(L(\th...
Mixture distributions combine several component distributions to model data with heterogeneous structure: an index \(z\) is drawn from Categorical(\(\pi\)), then \(x\) is drawn fro...
Classification algorithms can be organized by how they treat the joint distribution of features and labels. Generative classifiers model the joint \(p(x, y)\) by specifying \(p(y)\...
When exact posterior computation is infeasible, Markov chain Monte Carlo (MCMC) provides a way to sample from a target distribution \(\pi\). A Markov chain satisfies the Markov pro...