Numbers, Recommendations and Unforseen Connections

11 Feb

Two blog posts caught my eye today. They touched on a similar theme — how to do you get a computer to account for taste?

The first was Stephen Baker’s BusinessWeek blog post with a take on the Apple iTunes Genius recommendations.  His initially incendiary claim (apparently unresearched beyond his own iTunes collection) is that the Genius recommendations — are rigidly race-based. Asking iTunes to make a playlist based on Aretha Franklin, he says, generates a playlist composed entirely of black R&B musicians because the database was compiled with that category rating highest. Taking a step back from his first blog post, Baker writes: 

“When I do a search on Aretha, the system starts with what it knows about her: R&B. And it lines up an entirely R&B playlist. How does it know she’s R&B? Many of the commenters say that the analysis starts with no groupings provided by the programmers, and that it is based entirely the study of user behavior. (Ie. What other artists do we group with Aretha on our playlists, and what other music do Aretha buyers purchase?)

“Based on what I’ve seen so far, I’m sticking my initial guess: The programmers started out Genius by putting the artists in their boxes—R&B, Folk, Classic Rock, World music, etc. It’s a little crude, but you have to start somewhere.”

Then there was a post by George at Fast Horse on the Idea Peepshow blog. George reports on the challenge NetFlix faced building customer recommendations based on the movie Napoleon Dynamite.  Apparently, it is almost impossible to create an algorithm to predict accurately what movies you’d like based on a high rating of the movie Napoleon Dynamite.  NetFlix has gone so far as to offer a $1 million prize to anyone who can improve their movie recommendation engine, and much of the challenge is accounting for taste in quirky movies.  The common thread: the drive to get computers to understand our behavior well enought to predict what we’ll do next — or at least what we’ll buy. 

It’s nothing new, of course. Much of Amazon’s initial success came from a recommendation engine based on collaborative filtering — the idea that population of people who purchased the one book might be interested the other  books that same population has purchased. The techniques have continued to grow in sophistication. And they have continued to    please and confound marketers–seeking the perfect pitch–and consumers–simply seeking something new.    

Baker says that this is the goal — to put us in categories to see how we are similar in what we buy, what we choose to do    and even what our risks might be for disease.  He writes:   

“Traditionally, marketers and politicians have organized us along traditional demographic lines: Income, ethnicity,    neighborhood, etc. But with more data about our activities, they can start to create new “behavioral tribes.” The old    boundaries break down…In music, I imagine the same thing will happen. Once the data comes in, Aretha and others will break    out of their boxes.”   

Maybe so…more likely, they’ll break out of the old boxes only to hop into new ones.       

But there is a lot of fun in all this.  Search and social media are helping people find like-minded partners-in-crime in ways  no one could have predicted back when I was in college in the latter half of the 80s, wondering why I could never hook up with fellow Neil Diamond-listening sci-fi fans.   

On the other hand, the more we seek and find people and experiences “like us”, the more we yearn to be surprised.  The reason Napoleon Dynamite fascinates, Boing Boing is a wonderful blog, Super Bowl ads disappoint, and why print news media may well live on is that they give us something we didn’t ask for.  Something we didn’t specify in our search terms, preferences or    previous actions. Something we didn’t even know we wanted.  

Crunching the numbers is worth the effort — the better they get, the less money will be wasted on marketing to the wrong    people at the wrong times. But breakthroughs happen when you surprise people.   I’d make sure there’s a seat at the table for creativity, ingenuity and intuition.  You never know what’s going to happen … and that’s the point.

2 Responses to “Numbers, Recommendations and Unforseen Connections”

  1. Adam Singer February 12, 2009 at 10:26 pm #

    Interesting analysis. Okay this totally goes against what Amazon and all those big sites but say you were a small business – what if you decided to manually suggest things?

    I know this sounds like it would be a lot of work but think about the level of interaction that would be, and how strong your recommendations and relationship building could be.

    Or instead of letting robots do it, somehow get users to recommend them to each other – not just reviews but actual recommendations. I don’t like having the robots of Amazon recommend me stuff (I used to be a music columnist and wouldn’t want to be referred music simply because others made the same selections).

    You bring up great points here Ken…hmmmm, may be worth deeper thought for a better solution.

  2. Ken Kadet February 13, 2009 at 11:45 am #

    Manual recommendations make a lot of sense for a small, specialty retailer. They certainly do offline — I love seeing “staff recommends” notes while wandering through the bookstore. And you should have see the job the local pet store owner did for us when recommending how we could turn a birdcage into a home for agile young rodent pets.

    Building on your idea, the real key would be two things: 1) for those who recommend to be seen as independent of whatever product delivers the best profit margin; and 2) for people to be able to get to know those who recommend products to them — because the best recommendation (survey says) is from “someone like me”.

    Think about Dell. I once called them when I wanted to buy an external hard drive. Told them what I was and asked how big a drive made sense. The conversation was short — he asked no questions and just sent me to the drive Dell was featuring that week.

    What if I could have chosen from a list of 25 “sales consultants,” each with a profile that told me who they were, their training and life experience. Might have found someone who has a home office or small business whose recommendations would carry some weight with me.

    Re: Amazon, I think they’re as close as you can get … there are users out there who are creating their own lists of favorites, telling you why they like them, a little on themselves.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: