Thursday, March 10, 2016

Words and Numbers

From Superforecasting: The Art and Science of Prediction, by Philip E. Tetlock and Dan Gardner:
In the late 1940s, the Communist government of Yugoslavia broke from the Soviet Union, raising fears that the Soviets would invade. In March 1951 [US intelligence under Sherman Kent reported there was a “serious possibility” of a Soviet attack.] But a few days later, Kent was chatting with a senior State Department official who casually asked, “By the way, what did you people mean by the expression ‘serious possibility’? What kind of odds did you have in mind?” Kent said he was pessimistic. He felt that the odds were about 65 to 35 in favor of an attack. The official was startled. He and his colleagues had taken “serious possibility” to mean much lower odds.

Disturbed, Kent went back to his team. They had all agreed to use “serious possibility” in the [report], so Kent asked each person, in turn, what he thought it meant. One analyst said it meant odds of about 80%. Another thought it meant odds of 20% – exactly the opposite. Other answers were scattered between those extremes. Kent was floored. A phrase that looked informative was so vague as to be almost useless…

In 1961, when the CIA was planning to topple the Castro government by landing a small army of Cuban expatriates at the Bay of Pigs, President John F. Kennedy turned to the military for an unbiased assessment. The Joint Chiefs of Staff concluded that the plan had a “fair chance” of success. The man who wrote the words “fair chance” later said he had in mind odds of 3 to 1 against. But Kennedy was never told precisely what “fair chance” meant and, not unreasonably, he took it to be a much more positive assessment.
Via Slate Star Codex

1 comment:

G. Verloren said...

I would imagine the problem is that the same degree of probability can have entirely different qualities of "seriousness" associated with it in different contexts. The ramifications of the undesired outcome themselves influence our notion of how serious a possibility is.

If you're heading to the beach and the weather report says there's a 20% chance of rain, you don't really consider that to be too "serious" a probability to concern yourself with. It'll probably be fine, and if it isn't, oh well.

But if you're defusing a bomb and you're told that cutting a certain wire has a 20% chance of detonating the device instantly, that exact same chance of failure is treated as a much more deathly serious risk, because the consequences of failure are themselves so much greater.

When life and death are on the line, smaller percentages end up being assigned much greater value than otherwise. Fifty-fifty odds are actually pretty deceny when you're betting on a hand of poker among friends, but absolutely terrifying when you're considering survival rates for major surgery, or the likelihood of the outbreak of war.