Skip to content

Chapter 6 of 8

Mixtures, Latent Variables, and Variational Bounds

Mixture distributions combine several component distributions to model data with heterogeneous structure: an index \(z\) is drawn from Categorical(\(\pi\)), then \(x\) is drawn from \(p(x \mid z)\), giving marginal density \(p(x) = \sum_k \pi_k p_k(x)\). A Gaussian Mixture Model (GMM) is the most common instance, mixing \(K\) multivariate Normals with weights \(\pi_k\), means \(\mu_k\), and covariances \(\Sigma_k\). The component assignment \(z\) is a latent variable — an unobserved quantity that influences observed data — and is learned rather than observed directly. The standard algorithm for fitting GMMs is Expectation-Maximization (EM): the E-step computes posterior responsibilities \(\gamma_{nk} = \pi_k N(x_n; \mu_k, \Sigma_k) / \sum_j \pi_j N(x_n; \mu_j, \Sigma_j)\), and the M-step re-estimates \(\pi_k\), \(\mu_k\), \(\Sigma_k\) from \(\gamma\)-weighted sufficient statistics. EM is a general recipe for maximum likelihood in latent-variable models, and it underlies countless applications from clustering to speech recognition.

Information theory provides the tools to compare distributions and bound quantities that arise in latent-variable modeling. KL divergence \(D_{\text{KL}}(p \| q) = E_{x \sim p}[\log p(x)/q(x)]\) is non-negative, asymmetric, and not a true metric, but it captures how one distribution \(p\) differs from a reference \(q\). Jensen's inequality states that for a convex function \(\varphi\), \(\varphi(E[X]) \le E[\varphi(X)]\); this simple fact drives many variational bounds. Most importantly, the Evidence Lower Bound (ELBO) is defined as \(\text{ELBO} = E_q[\log p(x, z)] + H(q) = \log p(x) - D_{\text{KL}}(q(z) \| p(z \mid x))\); it lower-bounds the log evidence \(\log p(x)\) and is tractable whenever the joint \(p(x, z)\) and a tractable \(q\) can be evaluated, making it the workhorse of variational inference.

Variational inference (VI) approximates an intractable posterior \(p(z \mid x)\) with a tractable \(q_\phi(z)\) by minimizing \(D_{\text{KL}}(q_\phi \| p(\cdot \mid x))\), equivalently maximizing the ELBO. Mean-field VI further restricts \(q\) to a factorized form \(q(z) = \prod_i q_i(z_i)\); this leads to coordinate-ascent updates where each \(q_i\) is set to the log-normalized conditional expectation of \(\log p\) under the current values of the other \(q_j\)'s. When closed-form optimization is impossible, importance sampling offers an alternative: \(E_{p(x)}[f(x)]\) is approximated by \(\frac{1}{N} \sum_i f(x_i) p(x_i)/q(x_i)\) with samples drawn from a proposal \(q\). Variance depends critically on the choice of \(q\), which should ideally have heavier tails than \(|f| p\).

All chapters
  1. 1Foundations of Probability Distributions
  2. 2Discrete Distributions for ML
  3. 3Continuous Distributions
  4. 4The Normal Family and Asymptotics
  5. 5Bayesian Inference and Conjugate Priors
  6. 6Mixtures, Latent Variables, and Variational Bounds
  7. 7Classification Models from a Distributional View
  8. 8Sampling, MCMC, and Modern Generative Models

Drill it

Reading is not remembering. These come from the Probability Distributions For Ml deck:

Q

What is a probability distribution?

A function (or rule) that assigns to every outcome in a sample space a non-negative probability, with the total probability over all outcomes summing (or integr...

Q

What is a discrete probability distribution?

A distribution defined on a countable set of outcomes, specified by a probability mass function (PMF) p(x) with p(x) ≥ 0 and Σ p(x) = 1.

Q

What is a probability mass function (PMF)?

A function p(x) that gives the probability of a discrete random variable taking the value x; satisfies p(x) ≥ 0 for all x and Σ_x p(x) = 1.

Q

What is a probability density function (PDF)?

A function f(x) for a continuous random variable such that P(a ≤ X ≤ b) = ∫_a^b f(x) dx, with f(x) ≥ 0 and ∫_{-∞}^{∞} f(x) dx = 1.