the missing data set; the planes that did not come back and the founders who did not make it
Study this properly
Free flashcard deck: Cognitive Biases Quick Reference For Decision Making - 231 cards
Start studyingThe advice you never hear is to seek out the people the plan didn't work for, because success stories are a biased sample and cannot tell you what actually causes success. If you study only the winners, every habit they share looks like a magic ingredient, even when the same habits killed the people who quietly disappeared.
Survivorship bias is a systematic filtering of the record. Failures tend to vanish: startups liquidate, athletes fall off the roster, papers go unpublished, funds get closed. The survivors remain visible, so the dataset silently fills with the lucky or the robust. When you infer a rule from that dataset, you are comparing survivors only to the thin part of the record, not to the full population of attempts. The causal signal you think you see may be pure coincidence of selection.
The classic wartime example is the armor-plating decision. When engineers mapped bullet holes on returning bombers, they saw clusters on the wings and tail. Their first instinct was to reinforce those areas. But the planes that came back were the ones that survived those hits. A hit to the engine or the cockpit likely prevented return, so those clean spots were actually the fatal ones. The correct response was to armor the areas with no holes in the surviving sample—the zones where a hit meant the plane never made it home.
The bias does not say that habits are irrelevant or that success is pure luck. It says you cannot test a habit by looking only at the people who kept it and won. Both winners and losers may have worked long hours, made cold calls, or launched at the same time; the losers simply are not in your spreadsheet.
Survivorship also flatters survivors. They built a narrative around their choices because those choices are the only evidence they have. Read a founder's memoir and you see deliberate actions; read the bankruptcy files of the same era and you see the same actions with different outcomes. Without a control group, the story is a rationalization, not a proof.
If you have data on every attempt, the bias is less of a problem. A randomized trial tracks all participants, including the ones who drop out or fail, so treatment effects can be estimated without this filter. Census-based studies of an entire industry, adjusted for firms that entered and exited, can also avoid survivorship. By contrast, looking at today's index funds or the current list of unicorns without counting delisted funds and failed startups will always overstate performance.
The lesson is to ask, before trusting a success recipe, what did the competitors who failed do? The missing cases are not a footnote; they are the measurement you need.
Cram To make it big, I just copy the habits of people who already made it, right?
Rep That path is a trap. You can only copy the winners you can see. The thousands who did the same and crashed are simply not in the picture.
Cram But the winners did something obviously right.
Rep Maybe, or maybe only their plan survived. The failures and the winners often did the very same things. You just never get to meet the failures.
Cram So all those success stories read like formulas, but they are not?
Rep Right. The story you read is the one where the dice landed well. Survivor bias feeds on the vanished cases, and it quietly flatters the survivors.
Cram Like the planes bullet story. They armor the holes, right?
Rep That is where survivor bias bites. The military wanted to armor the bullet holes on the planes that came back. Wald said armor the clean spots. Planes hit there never returned.
Cram The holes were proof of survival, not of danger.
Rep Yes. You armor what you cannot see. The survivors tell you what kills less, never what kills more.
Cram So when I read about the founder who swears by cold calls and early mornings...
Rep Ask how many did the exact same thing and went bankrupt. If only the winners stick around, the habit looks magical when it is just survivorship.
Cram So the real lesson is hunting for the people it did not work for.
Rep Exactly. Build your model from winners and losers alike. The missing cases are the real classroom, and they teach what nobody says out loud.