A probability distribution is a rule that assigns probabilities to outcomes in a sample space, with the total probability summing (for discrete cases) or integrating (for continuous cases) to one. For discrete random variables, this rule is called a probability mass function (PMF), written \(p(x)\), which must be non-negative and satisfy \(\sum_x p(x) = 1\). For continuous random variables, the analogous concept is the probability density function (PDF), written \(f(x)\), such that the probability of the variable falling in any interval is \(P(a \le X \le b) = \int_a^b f(x)\,dx\). Both PMFs and PDFs are non-negative and integrate or sum to one, but only the PDF can take values greater than one, because it represents density rather than probability directly.
Beyond the immediate description of probabilities, distributions can be characterized through cumulative and aggregated quantities. The cumulative distribution function (CDF) is defined as \(F(x) = P(X \le x)\); it is non-decreasing, right-continuous, starts at 0 as \(x \to -\infty\), and approaches 1 as \(x \to \infty\). For continuous random variables, the CDF is the integral of the PDF up to \(x\), and where \(f\) is continuous, the PDF is recovered as the derivative of the CDF: \(f(x) = dF/dx\). The expected value (or mean) summarizes central tendency: for discrete \(X\) it is \(E[X] = \sum_x x \cdot p(x)\), and for continuous \(X\) it is \(E[X] = \int_{-\infty}^{\infty} x \cdot f(x)\,dx\), with absolute convergence assumed.
Dispersion around the mean is captured by the variance, defined as \(\mathrm{Var}(X) = E[(X - E[X])^2]\), equivalently \(E[X^2] - (E[X])^2\). The standard deviation is the square root of the variance, restoring the original units of \(X\). These summary statistics — mean, variance, and the underlying CDF — provide the basic vocabulary for describing any distribution, whether discrete or continuous, single-variable or multivariate.