Observational bias

Market research is a nuanced field, and to suggest that the people leading it find what they want is fraught with peril. Yet observational bias exists. Here’s one potential example (emphasis on “potential”). ListenLogic, a social media intelligence and analytics company, has published a report that suggests that 25% of shopping conversations are posted online while consumers are in a store, via mobile handsets. Egads, you think — 1 out of 4 consumers is talking about my brand while IN THE STORE! We must improve customer service! And learn how to monitor real-time consumer conversations!


Social media intelligence has extreme value, and we don’t suggest you not hire services such as Radian6 to see what the public thinks about you. But dig into this study and you’ll find of “16,500 public online and social consumer conversations” what was observed were consumers “discussing their shopping experiences.” As in, honey, I’m shopping, can you meet me at the store? ListenLogic reports, “Conversations in the Q1 2011 study ranged from comparing pricing, seeking assistance, checking-in, ‘meet me here,’ and interacting with staff.” The kinds of things you’re most likely to do in the store.

It is important to note that this stuff happened. But to conclude that 25% of all shopping-related conversation happens beside the mannequins at Macy’s … well, that is not the case here, when what we’re listening to is a very specific subset of consumer online posting behavior.

We’re certain the study is well-intentioned, as most research is. The takeaway for marketers is: Are you allowing your bias on what you want to find to discolor your research results?

HT Dirk Singer of the UK agency Rabbit.

Ben Kunz is vice president of strategic planning at Mediassociates, an advertising media planning and buying agency, and co-founder of its digital trading desk eEffective.

One thought on “Observational bias

Leave a Reply

Your email address will not be published. Required fields are marked *

* Copy This Password *

* Type Or Paste Password Here *