Category Archives: research

Observational bias

Market research is a nuanced field, and to suggest that the people leading it find what they want is fraught with peril. Yet observational bias exists. Here’s one potential example (emphasis on “potential”). ListenLogic, a social media intelligence and analytics company, has published a report that suggests that 25% of shopping conversations are posted online while consumers are in a store, via mobile handsets. Egads, you think — 1 out of 4 consumers is talking about my brand while IN THE STORE! We must improve customer service! And learn how to monitor real-time consumer conversations!


Social media intelligence has extreme value, and we don’t suggest you not hire services such as Radian6 to see what the public thinks about you. But dig into this study and you’ll find of “16,500 public online and social consumer conversations” what was observed were consumers “discussing their shopping experiences.” As in, honey, I’m shopping, can you meet me at the store? ListenLogic reports, “Conversations in the Q1 2011 study ranged from comparing pricing, seeking assistance, checking-in, ‘meet me here,’ and interacting with staff.” The kinds of things you’re most likely to do in the store.

It is important to note that this stuff happened. But to conclude that 25% of all shopping-related conversation happens beside the mannequins at Macy’s … well, that is not the case here, when what we’re listening to is a very specific subset of consumer online posting behavior.

We’re certain the study is well-intentioned, as most research is. The takeaway for marketers is: Are you allowing your bias on what you want to find to discolor your research results?

HT Dirk Singer of the UK agency Rabbit.

Ben Kunz is vice president of strategic planning at Mediassociates, an advertising media planning and buying agency, and co-founder of its digital trading desk eEffective.

TVs and microwaves: Consumers no longer need you

A new Pew study has gotten some buzz over the fact that consumers now view microwaves as more a necessity than television sets. The scary finding for marketers, though, is that most categories of household goods saw a 20-point decline in perception in the past three years as consumers began tightening their belts. You can almost hear the sales of big flat panels screeching to a halt. Marketers of all stripes need to rethink their messaging — because what people needed yesterday is now just a luxurious add-on.

Via Brandflakes.

Too busy to read a Nielsen web report? Click here.

London’s Dirk Singer digests the latest Nielsen internet report so you don’t have to. A few of his highlights: More people use online video today than email, but watch for only 6 to 10 minutes a day, so TV is still video king. Time spent using social media bypassed email in February of this year, so it’s time your boss stopped snickering about Twitter. And mobile internet remains Peanuts’ Great Pumpkin, the exciting dream that Linus stayed up late to greet but has not yet quite arrived.

What’s black and white and dead all over?

View more presentations from Dirk Singer.

Dirk Singer at London PR agency Cow has created an insightful report on the potential death of the newspaper industry. The full PDF, found here, offers more details than the slides above and points out the UK press faces the same challenges as papers in the U.S.

We’re just linking to it because Dirk quoted us next to Arianna Huffington. Finally, our thoughts align with The Huffington Post.

Black swans, Schrödinger’s cat, and your own crossroads

We spoke today with a gentleman working on a business plan for a very clever, and potentially lucrative, business. Research is required to tune the concept, and especially to predict which types of customers will be most interested. So we suggested that instead of focus groups or quantitative studies, he instead stage a small web campaign, insert a snippet of Quantcast code into the banners and landing pages, and use it to match inbound visitors to the vast data sets of user behavior online — which would pinpoint the exact demographics of the audience who likes his offer.

You know. Open the door and observe who walks through it.

Data is dangerous because it lulls us into false security — we often want to predict what will happen based on erroneous theories, and then fail to see the reality transpiring before our eyes. Wired notes this week the meltdown on Wall Street was tied to a single math formula that allowed investors a shortcut to assess hugely complex risks (um, big mistake). Our Twitter colleague Max Zeledon points out Nassim Taleb’s thesis that major events are really unpredictable; humans in hindsight try to make sense of the disorder in the universe by linking data points into logic flows, when the reality is Black Swans — things that shouldn’t exist — often just pop up. You can bend your mind thinking about the paths of fate; see Schrödinger’s cat and then ask which of yourselves is going in to work tomorrow morning.

Sometimes data can predict events, if screened carefully; Google does this beautifully with its little known Flu Trends site, collecting user Google searches for flu remedies to predict outbreaks in the United States two weeks before the Centers for Disease Control. Tel Aviv University professors are sorting Gnutella music searches by the location of consumers to predict when small bands will spike into bell curves of popularity. And in our favorite example, a simple chart comparing home prices vs. rent over the past 28 years indicates clearly that the U.S. housing market was due for a massive headache in 2009.

So are we humans vain to try to see the future, based on the data at hand? Or does randomness really make forecasting impossible? We ordered Taleb’s Black Swan tonight to learn more. Until then, we’ll keep trying.

Photo: Dietrich

Ads failing, Facebook turns to research

Facebook advertising has some of the worst performance of any online ad formats. Click-through rates (the percent of ads that are actually clicked on) are among the lowest of any online ad buy, and while the ads can be purchased on an efficient cost-per-click basis, target populations tend to be tiny. The service now has 150 million active users, but if you drill in to reach a given demographic the numbers shrink — for example, advertisers hoping to reach men over age 25 with college degrees in New York City end up with only 70,280 targets. Multiply those people by the 0.07%, say, who click through ad impressions, and you see the problem.

One ray of hope for social media sites such as Facebook is to give up on advertising models — you know, pushing third-party messages to people who don’t want them — and instead move to learning models. Facebook announced at Davos this week that it is launching a market-research service to turn its vast subpopulations around the world into online focus groups. 150 million members suddenly become the world’s largest market-research database. The service moves beyond marketing; for example, political leaders could ask populations in Israel and Palestine their take on whether current events will lead to peace, and adjust diplomacy (or, egads, wars) accordingly.

Flaws remain. The real-time reporting capability is offset by the skew toward young demos and an affluent, tech-savvy audience. But if millions of people are chatting and ignoring you, perhaps the best utility of the network is simply to listen in.

Marketers, boost the GRPs on sunny days

Dirk Singer points us to a brilliant psychological study on the effect of weather on consumer memory and judgments. Researchers put 73 subjects in a shop in Sydney and tested their ability to recall objects; half were tested on sunny days and half in rainy weather. As you’d expect, rainy-day subjects were in dour moods, but they had much better memories — recalling 3 times as many objects — and scrutinized objects carefully.

The British Psychological Society sums up: “The theory is that a bad mood triggers a more sceptical, careful mode of processing, in contrast to the less vigilant, conceptual thinking style that characterises a good mood.” If you hope to sell to consumers on a whim with vague, rosy product promises, we suggest you beef up the media schedule on bright, sunny days.

Secret to online targeting: Hello, neighbor

One of the great myths of online advertising is that publishers can accurately dissect the demographics of a given web site’s users. Quantcast uses cookies and comScore uses research panels, but all major online metrics companies monitor only a fraction of the millions of web sites that consumers actually consume. And even of sites they do measure — how do they know who you are, at the PC screen? Or if it is you or your spouse or kid or the dog somehow perched by the home office desk pawing at the keyboard?

David Honig suggests that the simplest way to improve online ad targeting may be to recognize the relationships between similar consumers. He wrote in OMMA Magazine that AT&T Labs Research and NYU ran a study in 2004 looking at response rates from telephone “network neighbors,” or people who communicate with each other frequently:

“If they found one network neighbor to have responded to a particular direct mail offer, then sending the same offer to his network neighbors resulted in a three- to fivefold lift above any targeting technique not informed by this network-neighbor data.”

OK, the study used phone networks and direct mail, but put the media formats aside for a moment. This is an amazing finding, if you think about it. These consumers had nothing else in common except that they communicated with each other regularly via the telephone; when direct mail hit them, they tended to act like a clone of their friendly counterpart. The researchers called this “homophily,” in which people attract friends with similar interests who like the same products or causes — and have the same Pavlovian responses.

Honig suggests this approach of mapping “neighbors” online for ad targeting may eventually replace demographic targeting; rather than pitching an ad for diapers to women ages 35-44 with children and HHI above $100k, you’ll serve the ad for diapers to friends of women who have recently bought diapers. Birds of a feather shop together.

Photo: Estherase

What McCain and Obama hostility means for Amtrak

Tim Siedell, the design guru behind the Bad Banana Blog, shared this “vote with your gum” poster that strikes a chord as Americans come to grips with who won and who lost. What we like about it is the pure form of collecting anger — pick what’s wrong, please. Very few organizations have a simple complaint-discovery system in place.

If you think about it, almost all marketing systems are designed to track responses — but not avoidances. How do you uncover the people who hate you and may pass that message on to others?

A few years ago we got a phone survey from Amtrak, after taking the Acela express train from New Haven to Washington D.C. on a regular basis. The research firm asked questions on food, comfort, prices, then ended the call. We gave Amtrak high marks in every category. What the survey forgot to ask was about the conductors, whom we found the rudest of almost any customer-facing personnel in the universe. You see, Amtrak has a simple business process problem — it expects passengers to close the overhead bins after putting luggage in them, but passengers are trained by airplanes to have flight attendants walk the aisles and close the bins for them. So instead of understanding this confusion, Amtrak conductors loudly chastise the train full of adults to snap to attention and close the bins — rudely.

As the train lurches forward, you can see business professionals and parents looking at each other in surprise — wow, those conductors sound hostile.

Amtrak, like many organizations, could use a simple tool to figure out who hates what, and perhaps it’s something simple it could fix. A little complaint discovery goes a long way.