Most consequential decisions are made without full information, so reasoning well under uncertainty is a core skill. Frank Knight drew the foundational distinction between risk, where probabilities are known, and uncertainty, where they are not. Under risk, expected value thinking guides choice: weighing likely upside and downside by both impact and probability, with expected value computed as \(EV = \sum_i p_i \cdot v_i\). Under uncertainty, when probabilities cannot be assigned, the same arithmetic fails and different rules apply.
Bayesian reasoning formalizes learning under uncertainty. A prior probability expresses initial belief about a hypothesis; after new evidence arrives, Bayes' theorem updates that belief into a posterior probability: \(P(H|E) = \frac{P(E|H)P(H)}{P(E)}\). The likelihood ratio, how much more likely the evidence is under the hypothesis than its negation, is the engine of that update. A common medical pitfall illustrates the value of this discipline: a positive test with 99% accuracy can still mean a low probability of disease if the disease is rare, since base rates dominate. The most important question to ask of any forecast is: what would change my mind? Setting a falsification standard exposes confirmation-only thinking.
Calibration measures how well probabilities track reality. A well-calibrated forecaster says 70% when right 70% of the time. The Brier score quantifies accuracy as the average squared error between predicted probability and outcome, \(\frac{1}{N}\sum_{i=1}^N (p_i - o_i)^2\), with zero being perfect. Tetlock's research on superforecasters shows that good judgment under uncertainty is a learnable combination of evidence weighing, calibrated probability estimates, and frequent updating. Their Ten Commandments include triaging hard questions, decomposing problems, balancing inside and outside views, and using granular probabilities. Ambiguity aversion (the Ellsberg paradox) describes preferring known risks to unknown ones even at worse expected value, a feature of Knightian uncertainty rather than quantifiable risk.
Beyond probabilities, decision-makers must distinguish types of unknowns. A known unknown is a risk you can name but cannot quantify; it warrants scenario planning, which explores how a choice performs across multiple plausible futures rather than one forecast. An unknown unknown, in Rumsfeld's phrasing, is a risk you do not even know to look for; black swans, rare high-impact events, live here. The McNamara fallacy warns against trusting only what is measurable, and Goodhart's law cautions that when a measure becomes a target it ceases to be a good measure. The streetlight effect describes looking for answers where it is easy to look rather than where they actually are.