Probability provides the mathematical language for uncertainty. A probability is a number between 0 and 1 that quantifies how likely an event is to occur. Several rules govern how probabilities combine. The complement rule states that \(P(\text{not } A) = 1 - P(A)\). The addition rule gives \(P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)\), and the multiplication rule states \(P(A \text{ and } B) = P(A) \cdot P(B \mid A)\). Two events are independent if \(P(A \text{ and } B) = P(A) \cdot P(B)\), meaning that knowing one tells you nothing about the other. They are mutually exclusive if \(P(A \text{ and } B) = 0\), because they cannot occur together. Conditional probability, \(P(A \mid B) = P(A \text{ and } B) / P(B)\), expresses the probability of A given that B has occurred.
Bayes' theorem, \(P(A \mid B) = P(B \mid A) \cdot P(A) / P(B)\), formalizes how evidence updates beliefs. In Bayesian language, the prior \(P(A)\) is the probability of a hypothesis before seeing the data, the likelihood \(P(\text{data} \mid \text{hypothesis})\) measures how plausible the data are under that hypothesis, and the posterior \(P(A \mid B)\) is the updated belief after combining the prior with the observed evidence. The law of total probability, \(P(A) = \sum_i P(A \mid B_i) \cdot P(B_i)\) over a partition \(\{B_i\}\), ties conditional and unconditional probabilities together. A random variable is a numerical outcome of a random phenomenon; it is discrete when it takes countable values and continuous when it takes any value in an interval. For discrete variables, the probability mass function (PMF) gives \(P(X = x)\) for each possible value, while for continuous variables the probability density function (PDF) has the property that the area under its curve over an interval equals the probability of falling in that interval. The cumulative distribution function \(F(x) = P(X \le x)\) describes either case, and the expected value \(E[X] = \sum x \cdot P(X = x)\) for discrete variables represents the long-run average. The variance \(E[(X - E[X])^2]\) is the expected squared deviation from this mean. When arranging outcomes, permutations \(_nP_r = n!/(n-r)!\) count ordered selections, while combinations \(_nC_r = n!/(r!(n-r)!)\) count unordered selections.