Human Bias vs Data Reporting

Why Marketers Should Start Listening to Data

March 17th, 2013

Armed with mountains of data, most experts put in charge of predicting outcomes still can’t come up with accurate answers. Why? Because most people have a human bias to manipulate data to deliver the outcomes they really want to see.

Philip Tetlock, a psychologist at the University of California at Berkeley, published a book in 2005 titled “Expert Political Judgment — How Good Is It?” For the book, a team of researchers interviewed 284 political experts and extracted 80,000 predictions from them over 15 years, rating the probabilities that each event would occur (they gave each prediction three possible outcomes). The result was surprising: the experts’ predictions were worse than those made simply by assigning equal probabilities to all three outcomes. Dart-throwing monkeys could have performed just as well. It’s easy to make fun of talking heads and political pundits, but what about experts in other fields?

Another psychology professor and a past President of the American Psychological Association, Paul Meehl, has conducted similar research in other fields. In his work, he studied whether a simple algorithm could predict freshmen grades at the end of a school year better than expert college counselors could. The algorithms were only given SAT scores and grades. The expert counselors, on the other hand, were given the same data as well as a 45-minute interview and a personal essay. Again the simple algorithm won.


In his book, “The Signal and the Noise, ” Nate Silver tells a great story about how the PECOTA system he built to predict pitcher performance in baseball was beat by expert scouts. In 2006, Nate’s program PECOTA produced a list of 100 prospects from the minor leagues. He then pitted that list against the Baseball America Top 100 prospects list, which was constructed by scouts. After watching six years of actual performance, and worrying their jobs would disappear due to PECOTA, the scouts performed better than the PECOTA list by a good margin. It turns out the scouts had learned to tone down their own biases and instead “listen” to the data. They learned they could combine their expertise with the reams of public data available.

Marketers can learn to separate the signal from the noise. Marketers are dedicated storytellers who believe their own stories, downplay luck and impose cause on randomness. In the world of Big Data, marketers have to learn to to let the data speak for itself. Otherwise, all the big data in the world won’t help marketers understand their customers better.

Omar is the co-founder and CEO of BlueKai, the data activation system

Source: Why Marketers Should Start Listening to Data – Omar Tawakol – | AllThingsD