Probability Distributions for ML

A function (or rule) that assigns to every outcome in a sample space a non-negative probability, with the total probability over all outcomes summing (or integrating) to 1.

A distribution defined on a countable set of outcomes, specified by a probability mass function (PMF) p(x) with p(x) ≥ 0 and Σ p(x) = 1.

A function p(x) that gives the probability of a discrete random variable taking the value x; satisfies p(x) ≥ 0 for all x and Σ_x p(x) = 1.

A function f(x) for a continuous random variable such that P(a ≤ X ≤ b) = ∫_a^b f(x) dx, with f(x) ≥ 0 and ∫_{-∞}^{∞} f(x) dx = 1.

A function F(x) = P(X ≤ x). It is non-decreasing, right-continuous, lim_{x→-∞} F(x) = 0, and lim_{x→∞} F(x) = 1.

F(x) = ∫_{-∞}^{x} f(t) dt and, where f is continuous, f(x) = dF/dx.

E[X] = Σ_x x · p(x), assuming the sum converges; it is a measure of central tendency.

E[X] = ∫_{-∞}^{∞} x · f(x) dx, assuming the integral converges absolutely.

Var(X) = E[(X − E[X])^2] = E[X^2] − (E[X])^2; it measures the spread around the mean.

The square root of the variance: σ = √Var(X); it has the same units as X.

The distribution of a single binary trial: P(X=1) = p, P(X=0) = 1−p. Its mean is p and variance is p(1−p).

The number of successes in n independent Bernoulli(p) trials: P(X=k) = C(n,k) p^k (1−p)^{n−k}, with mean np and variance np(1−p).