In 2006 Netflix offered a $1 million prize for anyone who could improve its movie preference recommendations by 10%. Netflix, at the time, made most of its money sending DVDs in the mail to users’ homes (Internet streaming had yet to take off), and personalization offered two major advantages as customers built their “movie queue” on the Netflix website. First, if the recommendations seemed to make sense, Netflix consumers would be happy as they searched for films online. And second, once the DVDs came in the mail, users might actually enjoy the movie — since a truly personalized prediction would be more likely to meet your taste than your guess based on a movie’s cover image and brief description. Happy ordering and happy watching built Netflix customer loyalty.
To spur improvement, Netflix did more than offer big bucks in the competition. It made public a dataset of 100 million-plus ratings on 17,000 movies, which included the customer rankings from 1 to 5 stars and the sequence in which customers watched films, and allowed competitors to play with the data. The cleverest part was a subset of the data was hidden blind, and Netflix would run the proposed new algorithms against that to see if the prediction models matched how customers really behaved in film rankings.
Mathematicians went wild. The competition was lauded by business pundits as an example of crowdsourcing genius. Because this was damned hard math, the project took years. And then in 2009, a team of mathematicians called “BellKor’s Pragmatic Chaos” actually cracked the code, achieved a 10% lift, and Netflix gave them the $1 million.
And then … Netflix never implemented the winning algorithm. Because personalization at that point no longer mattered.
What happened? Netflix at the time said the technical work of implementing the new personalization would be too costly for the anticipated return. These seems like a rather lame excuse, since bundling a new math model into a computer system surely doesn’t cost more than a bit of coding. Other observers noticed that, by 2010, Netflix’s business model had changed, moving away from DVDs-by-mail to instant streaming. When you can order any movie instantly online, personalization isn’t as valuable — since if the movie is a dog, you simply click over to another movie. Today, in 2014, Netflix’s online interface has a series of rows of film titles, and most of them aren’t personalized recommendations at all.
You learned my needs. But I don’t really care.
The deeper issue is that personalization is not as exciting as many once believed. In the 1990s, Don Peppers built a consulting business on the concept of “1to1 marketing,” where new computer systems would learn individual preferences and businesses would respond with customized offers. Don’s concept was that personalization would create an unbreakable competitive advantage — because once a consumer trained a company to anticipate her needs, she would be reluctant to go through the same process with a competitor. Don was observant enough to note that such customization wouldn’t be a fit for every business model — but companies that had customers with a wide range of needs (such as Netflix movie watchers) or a wide range in value (say, financial advisors courting investors) would benefit by deploying 1to1 personalization.
Despite the noble dream of giving customers more utility and companies more brand loyalty, personalization never took off. Amazon was really the best case study … but it struggles still to offer truly relevant personal recommendations on its website (the core challenges being it cannot easily recognize multiple users on the same Amazon account, or differentiate between your modality as you shop for your spouse one day and yourself the next). Twitter has a personalization engine behind its “Discovery” tab to push news or links to you based on your observed Twitter profile. That site section has so little utility, most Twitter users don’t use it. And Facebook, which arguably has the greatest trove of data on human personal interests, is really at the mercy of the advertisers who wish to target you; this is why you, guys, get ads for men’s underwear whether you really want them or not.
Why is personalization so difficult? Why is it so hard to anticipate what people want, and use that for business advantage? The challenge is personalization is at odds with a core driver of consumer purchase behavior — novelty. Consumers are constantly hungry for something new, something improved, something that will stimulate their endorphins in a manner unseen before. The iPhone 5S had marginal improvements over the prior models, but people lined up in droves for the new OSX, excited by — wait for it — a thinner Helvetica font. Most cable on-demand movie rentals are “new releases,” when logically the utility you derive from a film should have little to do with whether it was released in 2013 or 2003. Retail stores make a business of rotating in new fashions that don’t keep your body any warmer, but spring your desire to shop.
Psychologists have termed our love for newness “the novelty effect,” and it has both positive and negative implications. Humans have the highest stress response when first faced with a threat, likely a survival mechanism that spurs our fight-or-flight reflexes when a mountain lion appears over the hill (and which also explains our grumpiness on Monday mornings at work), so new things can make us angry or upset. But people also have the highest interest when a new person enters their life, a new service is launched, or a new technology is offered. Teachers have noticed, for instance, that when educational information is presented in a new medium — say, tablets — students’ test scores initially rise; the information could be the same as that presented in a history book, but the novelty of the new approach lifts interest and recall.
There is also an evolutionary bias toward novelty in human relationships, both emotional and sexual. A 2012 study by The Journal of Marital and Family Therapy found that at least one spouse in 41% of marriages admitted to marital infidelity. The reason, psychologists believe, is that physical and emotional excitement often diminish in long-term relationships. This could be tied to an ancient instinct for humans to be sexually promiscuous to spread their genes as randomly as possible, ensuring the survival of our species. Even with love, people will trade proven history for risky novelty.
And that is the trouble with personalization. Finding something new is likely at odds with our old interests — because newness by definition is a break from the past. It’s an interesting lesson for marketers now playing with big data. Yes, you can learn and model the past needs of your customers. But just as Netflix didn’t implement a 10% improvement in personalized recommendations because its analysts didn’t predict much value from it, you may find that launching a new product with different sheet metal or miracle fiber gets customers more excited tomorrow than a personalized version of what they wanted yesterday.
No one ever asked for a phone with a camera on it. But today, we can’t live without it.