Probability provides the mathematical framework for reasoning about uncertainty. Every probability problem starts with a sample space, the set of all possible outcomes, and events, which are subsets of that space. The complement of an event A, denoted A' or A^c, contains all outcomes not in A, and the complement rule \(P(A') = 1 - P(A)\) is particularly useful when it is easier to compute the probability of "none" rather than "at least one." For combining probabilities, the addition rule governs unions: \(P(A \cup B) = P(A) + P(B) - P(A \cap B)\), where the intersection term is subtracted to avoid double-counting outcomes that belong to both events.
When two events are mutually exclusive (disjoint), they cannot occur simultaneously, so \(P(A \cap B) = 0\) and the addition rule simplifies to \(P(A \cup B) = P(A) + P(B)\). The multiplication rule governs intersections: \(P(A \cap B) = P(A) \times P(B \mid A)\). If A and B are independent, then \(P(B \mid A) = P(B)\) and the rule reduces to \(P(A \cap B) = P(A) \times P(B)\). Note that mutually exclusive events are generally not independent (except when one has probability zero). Conditional probability, \(P(A \mid B) = P(A \cap B) / P(B)\) with \(P(B) > 0\), updates the probability of an event based on new information.
Bayes' theorem, \(P(A \mid B) = P(B \mid A) \times P(A) / P(B)\), links the posterior probability \(P(A \mid B)\) to the prior \(P(A)\), the likelihood \(P(B \mid A)\), and the marginal probability \(P(B)\). When \(P(B)\) is not directly known, the law of total probability expands the denominator, giving \(P(A \mid B) = P(B \mid A) P(A) / [P(B \mid A) P(A) + P(B \mid A') P(A')]\). A useful alternative formulation expresses results through the likelihood ratio \(P(B \mid A) / P(B \mid A')\), where posterior odds equal prior odds multiplied by the likelihood ratio.
Bayes' theorem corrects the base rate fallacy, the tendency to ignore prior probabilities. For example, with disease prevalence \(P(D) = 0.01\), sensitivity \(P(+ \mid D) = 0.95\), and false positive rate \(P(+ \mid D') = 0.05\), the probability of disease after a positive test is \(P(D \mid +) = (0.95 \times 0.01) / (0.95 \times 0.01 + 0.05 \times 0.99) \approx 0.16\). Even an accurate test applied to a low-prevalence population can leave the probability of disease surprisingly low after a positive result. Finally, when counting outcomes, permutations \(P(n, r) = n! / (n - r)!\) count arrangements where order matters, while combinations \(C(n, r) = n! / [r! (n - r)!]\) count selections where order does not: choosing a president and vice president is a permutation, while choosing a committee is a combination.