Survivorship bias: the planes that came back

“My product is better than their product, so I’ll succeed.”

“If I focus on my unhappy customers, I’ll be able to retain them and decrease my churn rate.”

Seems right, but it's a reasoning error called survivorship bias.

Survivorship bias?

See, back during World War II, the RAF lost a lot of planes to anti-aircraft fire. So they decided to armor them up. But where to put the armor?

The obvious answer was to look at planes that returned from missions, count the bullet holes in various places, and then put extra armor in the areas that attracted the most fire.

Obvious … but wrong.

See, if a plane makes it back safely even though it has bullet holes in its wings, it means that bullet holes in the wings aren’t very dangerous. So armor up the areas that don’t have any bullet holes. Why? Because planes with bullet holes in those places never made it back.

Clever… but not obvious.

So, think about the evidence you are looking at and the evidence you are not looking at:

“My product is better than their product, so I’ll succeed.”

How about your acquisition? Very often the crappy product wins.

“If I focus on my unhappy customers, I’ll be able to retain them and decrease my churn rate.”

How about letting them go sooner and focusing on your happy customers?