Category Archives: advertising measurement

The false dichotomy of results

“America, love it or leave it” is an example of a false dichotomy — a statement so black-and-white that it leaves no room for nuance in the middle. Do you not like George Bush? Or Barack Obama? Either way, you’d have to move out of the U.S. Liberals who believe all oil companies are evil, or conservatives who think all government programs are bad, ignore the logical reality that the phone you love is made out of petrochemicals and the interstate highway system is pretty darned useful. To riff on Ralph Waldo Emerson, false dichotomies are the hobgoblin of foolish minds, adored by little debaters to make their own opinions seem more valid.

Marketers often fall into the same trap, however, in the impulse to measure every campaign discretely for results. It is tempting, nay, even necessary to try to measure revenue vs. costs, investment vs. return, from any given expense. We do this for clients (and have published much of our methodology for free here and here). But results are never isolated into black or white response. A radio campaign may have a cost per inquiry of $800, but be driving additional results in through Google search at a fraction of the response cost; measuring integrated long-term media impact is almost impossible. Advertising campaigns build brands and awareness as well as leads and sales; so monitoring ad spend solely on short-term sales results ignores the longer-term lift in consumer inclination. And the most important addition to media in decades, social media, builds platforms in which consumers can interact with each other … not necessarily leading to sales today, but rather communication forums that can be leveraged for customer service, complaint resolution, market research, competitor evaluation, product innovation, and yes, advertising response.

Forests catch fire because they’ve been dried out over time, not because one person lit a match.

So here’s a simple thought experiment: For each marketing campaign component you’re trying to isolate into ROI, draw a bull’s-eye, put that component and its key metrics in the middle, and then in the outer rings define all the other factors that program element will influence. Consumer awareness. Competitor position. Product research. Multivariate testing. Future campaign risk avoidance. Are any of them vital to your long-term business strategy? If so, what is the opportunity cost of missing the waves reaching those important peripheral goals?

Image: Toni Blay

Facebook’s $0.001 business model

“Facebook Just Blew Away the Competition in Display Ads!” bellows Business Insider. Yes! Ah, hyperbole and the agents of change who follow it. You’d think Facebook is crushing Yahoo and Google with ad revenue from that headline, right?

Um, no. Here’s a calculus secret — and we know you flunked math in high school which is why you went into advertising, so we’ll type slowly here — advertising impressions don’t exist. Impressions are really a currency by which media planners judge options on where to place ad investments. Take Facebook. Social media site users refresh their streams constantly; unlike a reader pouring over, an FB newsfeeder is clicking again and again, looking for the latest old girlfriends or Bejeweled update. Silicon Alley Insider suggests Facebook has a run rate of 720 billion display ad impressions a year, which works out to 1,800 per user. If you’re a heavy Facebook user, one of the top 10% who really check the site, given a 10-to-1 skew in usage (math, sorry) you may receive 18,000 display ads pitched at you annually.

How many of those Facebook display ads do you recall?

Can you name five?


Right-O. In our agency’s client work, Facebook ads can perform brilliantly if purchased on a cost per click basis — meaning you only pay if someone sees the ad, likes it and clicks through to learn more — but fail miserably when evaluated on impressions. Click-through rates, a good apples-to-apples proxy indicator of whether users are interested in an ad offer, average 0.02% on Facebook vs. the 0.08% average of banner ads in the U.S. Do the math (sorry, sore point) and Facebook users are one-quarter as likely to actually see the banner image in the back of their retina vs. those other colorful boxes at the top of Or, (more math) Silicon Alley Insider estimates Facebook made $700 million back in 2009, about $2 per user at the time. Given 720 billion annual display “impressions,” that equals (yes, math, but wait for it!) $0.001 revenue per ad served.

Facebook advertising, WOOT!

We know. OK. Can you name just one Facebook display ad?

.001 business model' st_url='' class='st_twitter_large'>

Dare we say it? Social media measurement defined.

The world of advertising is full of silly metrics, probably because many who work there are suppressed novelists or film producers who didn’t carry calculators in college. Alas, the problem of foolish marketing math has gotten more severe with the advent of social media, where you’ll find numerous numb-headed attempts to explain how to “measure” “engagement” for “ROI.”

Data guru Anna O’Brien got so ticked off by the hyperbole she created the chart above to explain exactly what goes into true measurement. She writes: “It hit me. Adding qualitative aspect to a previously primarily quant-based world has thrown some people so far for a loop that they are willing to accept complete gibberish as a viable marketing solution as long as it has words like ‘tweets’, ‘likes’, and ‘posts’ built into the equation.” O’Brien’s resulting post is a brilliant kingdom-phylum-class-order-family-genus-species hierarchy of the measurement universe, and she keenly draws the distinction between monitoring and true measurement.

Funnel. Return. Value.

We suggest the truth is even more simple: There are only 3 basic questions that need to be answered for measuring any marketing performance.

(1) Where is the customer in the marketing funnel? (Is your target moving through awareness, consideration, or action?)

(2) What is the ROI on the marketing investment? (Does the financial gain from your initiative outweigh the cost?)

(3) What is the resulting impact on customer valuation?
(Customers are your real asset base; is their lifetime value rising based on your marketing activity?)

Customers are your source of financial value. Everything in marketing manages customers’ inflow of value with those questions. Everything outside of marketing measures the outflow of organizational costs.

Yes, even Razorfish got it wrong

The problem with most social media measurement is it touches only a subset of (1) above: the awareness stage of the marketing funnel. You can track the number of mentions of a brand online, and rank the people talking by their level of influence, and evaluate whether the “sentiment” is positive or negative. But do all that, even with Razorfish’s lovely Social Influence Marketing score (see page 24 of Shiv Singh’s Fluent report), and all you get is a mood ring for how much customers love your product while they are talking about you. The vibe of awareness within the earliest stages of your marketing funnel tells you nothing, really, about whether they moved to action and bought something, the resulting ROI on campaign investment or long-term customer value.

That’s OK. There’s nothing wrong with measuring small sections of your entire value chain, and social media itself is primarily conversations that do not provide insight into consumers’ true actions. But remember that. Don’t get carried away by software that tells you X brand was mentioned 2,318 times yesterday, and that sentiment was up to 0.79. If you can put a number to something that helps you adjust marketing course for improved results tomorrow, then you know the measurement has value.

Footnote: We are a huge fan of Shiv Singh, one of the sharpest minds and clearest speakers in social media, and this post is not a knock on his insights. We just find the Razorfish social media score a narrow view of one way customer value flows into your organization, even within the confines of social media, and should be labeled as such. Hat tip to Anna O’Brien and the comments on her blog for inspiring this post.

Advertising as a research tool

What if we told you most companies spend millions of dollars each year on a research study but never look at the data?

That’s advertising for you. Wired magazine’s fun little mapping of American vice reminds us that many marketers spend big on advertising, but fail to leverage it as the research tool it is. Marketers typically compare responses and sales to campaign expense to see if the campaign “worked” with an acceptable ROI. If a $500,000 campaign makes 5,000 sales, and the cost per sale is an acceptable $100, then we have success.

But what about all the data hidden in those responses? Where did those calls come from? It’s a simple list exercise to match originating phone numbers to ZIP Codes, and suddenly the marketers could have a heat map, like Wired’s above, showing hot spots in market demand. ZIPs could then be run through PRIZM data to identify psychographic clusters of prospective customers. Sales by ZIP could be compared to responses by ZIP, to see if certain market areas have a low vs. high close rate. Some simple data digging, and you could find exactly which markets to direct future advertising to, where to reallocate your sales force, and where to send those expensive direct mail drops.

Advertising does more than generate a sale; it’s capturing knowledge about what the market thinks about you. If Wired magazine can map sin in the United States, surely you can plot your best future prospects.

Nielsen confirms advertising impressions are often fiction

You’re not paying attention, dammit! Or at least that’s what advertisers are learning about consumers, as shown by the cute triangles on the line at above right.

Nielsen recently enlisted the Ball State University Center for Media Design to spend a whopping $3.5 million and observe 476 U.S. subjects directly for a combined total of 952 days to see how they really acted in front of TV sets, computer screens, mobile phones and other media devices. The study found Americans on average:

– watch 8.5 hours of content on all screen devices each day,
– spend most of that time, 5 hours and 9 minutes, in front of live TV,
– pay the most attention to television and video games (and the least to other media, such as radio, which tends to be on while John and Jane America do other stuff).

All sounds good for advertisers until, alas, Page 46 of the report shows that when spots run on TV, the “concurrent media exposure indices” go through the roof. Um. Yikes. You see the triangles above? In layman’s terms, that means when ads appear for dish soap, consumers change their environment — go to the bathroom or kitchen, pick up a magazine, call someone on the phone, or toy with a laptop computer, things other than watch the 30-second spot on TV. “During commercial breaks,” the report says, “people were observed shifting their primary attention.”

Impressions as currency

We call this the fallacy of impressions. “Impressions,” if you don’t work in advertising, are the currency used to price advertising. When you’re spending ad dollars, you compare cable network A vs. magazine option B based on the cost to make impressions on the target audience. Impression estimates, like prices in a store, help marketers judge which media thing to buy. But just as the dollar bills in your wallet used to be tied to the gold standard but now represent fictional digits in a bank computer, “impressions” in advertising are often more currency used to price media … than any actual imprint on the retinas of a consumer.

This is not really news. Numerous studies, such as this one by Mediamark Research, show consumers’ attentiveness to advertising slips and slides based on the channel format and time of day. Of course advertising still works, but the question, as our esteemed agency chief says, is “how much?” The solution we recommend for clients is to build some form of direct measurement into your advertising media plan. Impressions may or may not happen. The only way to evaluate your real impact in the market is to monitor the response results.

What to do with all that advertising data

So with newspapers dying and even banner ads becoming problematic, it’s astounding that most marketers still don’t measure. Sure, you may have metrics in silos — CTRs on the web, GRPs in broadcast, CPMs in print — but typically each marketing silo is tweaked independently for performance.

Could you be missing the big picture?

The table above shows just this — a $1 million advertising schedule that is kicking off about 80% in return. What’s that, you say? The return seems to vary in each line? Well, of course. If you grouped your advertising media into 10 equal buckets (called “Media Groups” above) based on their performance in generating sales, you almost always find a 20-to-1 range in what works. Media Group 1 above performs horribly, losing money. Media Group 10 is rocking.

If this were an investment, you’d call your broker and say buddy, move my money around to what works. Kind of like this …

Much better. We’re still investing $1 million in advertising, but now the total return has jumped from 80% to 204%. We’re making more than $2 million in return instead of just $795,000.

We didn’t spend a dime more. We just made measurement meaningful.

So here’s a question for your marketing team. Are you caught looking at CTRs and conversion rates and CPMs and GRPs and reach and frequency, and not doing a damn thing about it? In this economy, it may be time to rebalance your entire investment portfolio.

Find more thoughts on ad measurement in our whitepaper here.

Memo to publishers: It’s all about controlling the outcome

The death of print has become such a cliché Paul Dailing recently penned an instruction manual on How to Become a “Death of Newspapers” Blogger. Magazine ad pages are off 26%, ad revenue at many larger newspapers is down 30%, NYT is chopping sections and the Chicago Tribune is cutting one-fifth of its editorial staff.

Why are newspapers dying and magazines gasping with asthma? It’s not just about lost readers — publishers have failed to help advertisers predict results, sort of like this card trick.

Advertisers provide magazines and newspapers more than 90% of their revenue, yet it is astounding how little publishers have done to help advertisers get more return from their dollars. Imagine if publications offered advertisers measurement systems to monitor the cost-per-response from every ad format. Imagine if publishers gave advertisers a host of case studies, results forecasts, planning tools, ad formats, tracking phone numbers, and testing schedules to make advertising work better over time. Imagine a print sales rep going over an advertising results report, showing a 20% improvement in marketing ROI and a series of specific steps for boosting ROI another 20% from the print ads in the next quarter.

You haven’t heard of these solutions from your newspaper or magazine sales reps? Funny. Neither have we. Until publications help advertisers control the outcome, expect the death watch to continue.

Usain Bolt: Sprinting toward macro trends

Wired notes that when Usain Bolt ran the 100-meter sprint in 9.69 seconds he startled biostatisticians who have been predicting for years how human speeds accelerate. It seems number-crunchers who look at macro trends can forecast new records with startling accuracy, using simple “curve fitting” shown above.

Bolt ran so fast he fell off the curve.

What’s remarkable to us is how few marketers look at such macro trends. Marketing campaigns are often structured tactically — set up integrated ad channels, web sites, call centers and field sales, all with forecasts for triggers and response. Spend $1 million at $100 cost per sale and you’ll end up with 10,000 customers. Simple.

But macro forces such as recession, changing energy prices, emerging media and aging demographics rumble the ground under such calculations. What worked yesterday won’t perform tomorrow. The internet is awash in free resources to watch such shifts: Google Trends shows global or regional search volumes for products and services; the brilliant Iowa Electronic Markets uses the wisdom of crowds to foresee political winners with 1.37% accuracy.

How are you predicting your own race finishers?

Social media explained, pardon the language

First, forgive the F*** bomb. We’d edit these slides for your delicate sensibilities, but don’t know how.

Second, extreme credit goes to Marta Z. Kagan for collapsing all the hyperbole about new communications into a pithy, inspiring slide show. Hit the right arrow to play. And get scared when you read Marta’s stats on the flaccid disempowerment of traditional advertising media. If you aren’t learning and testing new communication formats, well, get on the bus.

Via Shannon Paul.

Disney’s new ad research kingdom

Mickey Mouse is watching you. Or at least, how you watch ads.

Disney has launched a new biometric test program to monitor how people react to advertising, using research subjects in a controlled living room setting to measure heart rate, skin conductivity, eye movements, and potentially popcorn consumption. The goal is to try to figure out how nuances in TV or web or mobile creative change responses; for example, does a ticker crawl at the bottom of a video screen help or harm the impression?

It’s all very qualitative and the first analysis, for ABC and ESPN, isn’t due until 2009. But as advertising gets more fragmented and consumers tune out to play their social media games, any data that determines what sticks is welcome. Mickey, give us a call.