Tom Chi  

Getting Personal

January 14th, 2005 by Tom Chi :: see related comic

There was a period in the history of the web when personalization was going to transform *everything* — making the buying experience vastly more inviting for customers and lucrative for business. I’m not really sure what happened.

Right now, the standard bearer for personalization is still Amazon. More to the point, most everyone else has fallen off. So to ‘research’ this article I went to Amazon and looked at all 720 of my recommendations. This took a very long time, and resulted in me adding a grand total of 4 items to my wish list (2 of which were albums I already had on tape but wanted to update to CD). To their credit, there were about 20 or so items that they recommended that I had already bought through other channels, but on the whole, the experience wasn’t too useful for ‘discovering’ new items.

So what killed personalization?

1. Search
It turns out that it is much faster to search for what you want than it is to browse through even an elegantly presented collection of what someone else thinks you maybe might want. Search has greatly reduced our patience when it comes to accessing data.

2. Out of Channel Interactions
Personalization would work a lot better if there were no other places to get these goods. If Amazon was the only source for all my music, books and DVDs, then their training set would be optimal. Luckily for consumers and unluckily for personalization code, there are many outlets to get these products. Everytime I make a purchase anywhere else, I slightly degrade the quality of the Amazon training set.

There are also privacy concerns. If a company were able to collect all this data, many people would start to get very very nervous about it. Credit card companies probably have the most complete records currently, and we certainly don’t get warm fuzzies when we get ‘targeted’ junk mail based on our spending habits.

3. Unsophisticated Product Relationship Mapping
Looking through my recommendations, I noticed that a great deal of them were just other albums/books from artists/authors I had purchased before. While this is a ’safe’ strategy, it removes a lot of serendipity and discovery from the browsing process. With so much purchasing data, it seems like there would be ways of creating more interesting recommendations. Perhaps making use of a model similar to musicplasma would be a step in the right direction.

Amazon faces some interesting problems on this front, however. One problem though is that popular items tend to distort the data. If someone has bought a Harry Potter book, it tells you almost nothing about them. Similarly, person X who bought the Outkast album might also have Wu-Tang’s 36 Chambers, while person Y who bough the same Outkast album has a sizable Britney Spears collection. Suffice it to say that X does not want recommendations from Y or vice versa.

The result is that Amazon can easily type-cast a person incorrectly. It’s current guess is that I *love* psychedelic rock. In previous instantiations it was convinced that I was a big fan of cooking, Top 40 R+B, and world music… so I can see why they have chosen the ’safe’ route, but really both ends of the spectrum are pretty suboptimal. Having recommendations which are totally predictable and boring is only slightly better than having recommendations which are completely random and wrong.

4. Not Enough Content
It’s sounds wierd to say, but most sites don’t have enough original content to make personalization meaningful. For example, CNN.com is a pretty sizable and content-rich site, but if I want to personalize my CNN to follow only news about renewable energy, I will be sorely disappointed. I would get about one story a month. Most content sites have a *lot* less content than CNN, so they fare even worse in this regard. The only sites which have the depth of content to pull off programmatic personalization are the huge aggregators — the Amazon, the Google, etc. But being aggregators, they have a completely different relationship to the content. Since they are not the producers of the content, if they want to move beyond basic metadata, they need a lot of infrastructure to determine the quality and nature of the content. Amazon does this via reviews and buying statistics, Google does it via pagerank and various heuristics.

So if you are smaller than a CNN, a personalized site is often not meaningful. If you are bigger than a CNN, it might be meaningful, but it requires a *lot* of work. Is it worth the cost? In most cases no. For a Google or Amazon, sure. For an advertiser who is willing to pay more for a targeted ad? Sure. But for most sites deciding whether to install a personalization framework to splice up the content they have? Not really.

Despite all these drawbacks, there is still a prevailing sense that personalization is a valuable goal to strive for in the future. I’m still waiting for someone to do it right. If and when it happens, the creators will be walking a tightrope between predictability and serendipity, between privacy and accuracy.

22 Responses to “Getting Personal”
David Heller wrote:

Great points Tom!

An example of a place where I like and use personalization is NetFlix. You have to take the time to get dirty and tell it what you like, don’t like, but after that it really does start to help me find things that I wouldn’t have thought to search for.

Ya see, when it comes to renting DVD’s NetFlix is a single channel for me and even if it isn’t the way that recommendations are presented, if I see somethign I saw elsewhere, I just tell it what I think and move on. Giving ratings is pretty non-disruptive to the browsing process. In fact, rating and browsing are one and the same.

To your other points about Amazon. I totally agree. I find that Amazon’s recommendations are usually pretty useless to me. What I do find a lot more useful are the Listmania pages. I look up a book, I get a list of people’s lists where that book is relevant. In essence this is a type of “human” faceting. The human’s intuition about grouping things is an analog process, so it has a certain level of perception mapping that “digital” process of trying to do algorithmic corelations doesn’t quite get.

The way I am comparing analog to digital in this sense is out of N. Negroponte’s “Being Digital” where he expresses how exceptions in digital are a lot more disruptive than exceptions in analog. our minds are more prepared, atune, to analog exceptions so we can process them better. (at least that is what I got from that book.)

Bob Salmon wrote:

For recommendations, I think it would be good to keep a completely random element i.e. pure serendipity. It probably would be necessary for the user to allow the degree of this, and also to sample results before committing money or lots of time e.g. listen to a bit of the track.

As an example, my room-mate at college and I had basically disjoint music collections, but both turned out to like the other’s music. There was probably something deeper than musical taste linking us i.e. friendship, but basically it was serendipity. Ditto the music that your parents/brother/next door neighbour listened to - outside your control and unrelated to you, but you might like it.

ingo wrote:

In Information Retrieval this is called the “Novelty Ratio”: unknown-results/total-results. Of course, ‘unknown-results’ being user-specific and not knowable, its hard to optimize for.

In this case, though, things could be improved simply because Amazon already offers many common avenues of exploration directly (e.g., click on an authors name to see other books by her) and because the sales-count or similiar distortions are easy to correct for. All these things are not that hard.

However, I guess its a matter of goals: Amazon wants to maximize sales, not make you happy. Recommending bestsellers might do that in the short term. Even if it does turn you off reading all that boring stuff in the long term (and maybe it doesn’t), thats hard to account for.

Andrei Sedelnikov wrote:

Personalization was nearly abandoned also because the costs of personalization outweighed its benefits. It is extremely hard to get the really personalized content, because you need to gather a lot of information and to analyze it properly according to the context. We still have few sources of input and little experience with its understanding.

Google’s GMail context ads are on the way to the real personalization: they have a very rich source of information and they get more and more experience with analyzing. I even suppose that Adverting Industry will be the first who would get a perfect personalization because it is what they had always wanted. Now they have and will have technologies that would allow to know your living place, current location and income…

Bob wrote:

I remember an Amazon rep saying that the “Gold Box” was not designed to be good matches at all, but rather to expose you to things that you wouldn’t know Amazon even sold. The NEXT time you needed a vacuum cleaner or set of silverware, you might turn to them instead.

Like ingo said, it’s a business.

Henrique Ahnfelt wrote:

There is a saying “It is unbelievable what you can find when persistently looking for something else”

The personalization as Amazon does it has another dimension, which is presenting us with a constantly updated, ‘fresh’ and friendly site - what you see may bee what you just bought (sympathy recognition smile in your eyes, warm and proud owner satisfaction feeling in your heart), what you might buy (sympathy), etc. It gives you a friendlier environment and it is complemented by a usable environment to which you like to return to solve your quests. You buy only 2-4 extra items a year? Multiply that by the number of customers and it might not be all that bad - increases of 2-3% in sales due to cross-selling initiatives are what we work with on CRM progams for a bank, and they result in a nice ROI.

The bottom line is that it isn’t what you buy from what is promoted that counts - it is how much you buy overall, with and without the feature. A site of a client added a server which boosted performance. The number of new visitors doubled in 3 months - same content, same design, same whatever. It’s logical, but not straightforward.

Sites may also have less content but be targeted at one shot visits - if you have a site from a public health service advising teenagers on relationships and protection, it is enough if they learn the message the first time - you don’t need them there every day for eight hours. A tourism site may want to qualify a destination but you can’t expect people to travel to the same country every year. Etc.

Tom Chi wrote:

Well the whole spiel behind personalization was that it was a classic ‘win’-'win’. The customer would have a great experience, which would benefit the business with cross and upsell possibilities, all while creating a stickier site.

But as Amazon has duly shown, even with personalization running, there are real differences in what is good for sales and what is optimal for customer experience.

Tom Chi wrote:

Henrique, the theory sounds good, but in practice, I do not typically go through 720 recommendations. While I can imagine that someday this will be done right (i.e. there are probably hundreds of albums that I would buy if presented and recommended properly), currently getting 2 interesting results out of 720 seems a little low. I’m sure there are more powerful ways to expose me to 2 interesting results without running a personalization engine, and without having me sift through so many results.

The little bundles that they propose on item pages are a great example of this. Buy book X with Book Y today for only $28! This seems much more topical and to the point, but it is notable that it does not require ‘personalization’ to work.

Tim Tucker wrote:

I’ve argued with others over this one in the past, but I really like Amazon’s recommendations system.

I found myself initially somewhat amused to see the recommendations that it would give me when I started using (probably at least 2 years ago). Things like: “You like Star Wars, therefore we think you’d also like this coffee maker”.

Well, I started thinking to myself: what would happen if I started feeding data into the system? And so began a little hobby of mine: seeing just how accurate the system could get if you feed it massive amounts of data.

I started by checking off everything on the recommendation page that I owned and rating things I had an opinion on. I then went on to rate anything else in the list that I had a concrete opinion on one way or another.

Things improved, but I found something that I still consider one of the flaws in the system: it often gives far too much preference to things that you’ve rated or marked that you’ve owned more recently.

I also came across another flaw in the system: flagging an item as “not interested” doesn’t seem to have much effect on whether or not Amazon offers similar items (though flagging things with a negative rating seems to have some effect).

That said, I continued on to search out other things that I had opinions on…
- I rated all the books that I could ever remember reading
- I rated all the movies I could remember watching
- I rated any software that I’d ever used
- I rated any video game I’d ever played
- I rated any board game I’d ever played
- I even went on to rate classic toys that I’d had when I was growing up

End result:
• Items you own (565)
• Items on your Wish List (45)
• Items you’ve rated (1794)
• Items you’ve marked “Not interested” (338)

I now get extremely accurate ratings for book, music, dvds, games, and other media — Amazon could probably do a better job of buying me gifts than many of my friends and relatives.

The ratings in the “Kitchen and Housewares” category still make absolutely no sense, though.

Chris McEvoy wrote:

Here is an exercies that I use to try and find new books that I wouldn’t otherwise read as I don’t find my personalised recommendations very useful:

1. Take a single book from Amazon as a root.
2. Take the recommended books for that book.
3. Repeat until you have reached a distance of 20 from the original book.
4. Add up the number of recommendations for each book.
5. Calculate the BookRank from the Distance, Recommendations and SalesRank
6. Display the 50 books with the highest BookRank scores

Now this can take quite a long time to do with pen and paper, so I wrote some software that does it for me using the Amazon Web Services.

You can see the results at BookRank.net

For example, if I start with Readings in Information Visualization it isn’t very long before I get to a book about Mindmapping. I am sure that this would not appear in my personalised recommendations.

The further the distance from the root book the more tenous the link and it seems that every book will eventually lead to “The Da Vinci Code”.

If more companies made their data freely available we would be able to come up with a much wider range of approaches to personalisation.

John Lam wrote:

Above Tim has it right. Amazon uses many methods to present personal recommendations, some of which work better than others.

Specifically, your post began on personalization and rating things, but then later failed to distinguish between impersonal data mining and collaborative filtering. Devoid of the closed loop of feedback, recommendations based upon what others buy works worse than recommendations based upon what your like-minds like and dislike.

For a taste of a recommender system that uses only collaborative filtering, try MovieLens and be sure to rate items through your spectrum of love to hate. Back at Amazon, since you have already rated plenty, rate more of your dislikes that are similar to your likes. This users wrongly ignore. With a sufficient personal base, this improves recommendations more than rating yet more items you like. In collaborative filtering, not only does this improve predictions for yourself, but also helps Amazon discriminate between similar items of a topic or category, and also helps your like-minds find the otherwise unpredictable faves they seek.

Louise Ferguson wrote:

I’ve had the same experience with Amazon. Despite having bought perhaps 400 books through the site over the years, maybe 300 CDs and a few dozen DVDs (not to mention computer and kitchen stuff), and despite getting stuck into rating etc., I’ve found its suggestions pretty dud. As you say, it’ll come up with another disc by the same artist or another book by the same author…which is precisely when I *don’t* want to buy something else from the same artist/author. (I prefer variety, and even if buying 20-30 CDs will not opt for the same artist twice.)

I find looking at what people I know are buying much more useful.

Tom Chi wrote:

I don’t doubt that Amazon will do better with more data. This is the nature of training sets generally. But I’ve been buying things from Amazon for 6 years. It really has a *lot* of data about me already. For most people, spending a significant amount of time rating a list of hundreds of suggestions just doesn’t seem worth it. Afterall, I must have bought over 100 items in all these years, why can’t it use it’s considerable database to help me?

The reason I did not distinguish between personal consumer history, collaborative filtering, and meta-content mapping, is that is doesn’t matter. The consumer doesn’t give a crap what technology gets used to provide the good experience. Just give it to me. As an example, if I poke a couple levels through Real Rhapsody, I can very quickly find related artists that I know less about and review their songs. It’s completely awesome and it’s based on essentially no personal data about me. If Amazon could provide a more varied set of recommedations in this fashion along with a quick way to review the quality of those recommendations, it would be far more effective.

For example, let’s say I select an album from the Digable Planets. Along the bottom of the page it list other what other customers who liked Digable have bought. This list includes ‘Subject’ by Dwele and ‘Coming from where I’m from’ by Anthony Hamilton. Why not make it so I can click a listen link right there? Instead I have to navigate out to a item detail page, scroll a ways down to get to the track listing and to find a listen link and only then launch the player. If there was a listen link on the front page next to each entry, I could just click, click, click and experience 3 different artists I didn’t know much about.

Anyhow, perhaps we can talk more about how Amazon can improve without a ton of user intervention. I think it’s not that interesting to talk about how if you spend 2 or 3 hours filling in data for them they can do a better job.

David Hawdale wrote:

Pine and Gilmore in The Experience Economy” talked about different models of personalisation (in fact they call what a system serves up for you without your explicit request customisation, but lets not go there…). One model they call presentational personalisation and one they call product personalisation. A very useful definition, damn obvious once stated!

So, obviously, Amazon does presentational customisation. The thing you buy isn’t affected by the personalisation. Whereas CNN do product personalisation - the thing itself, the content, is altered by the personalisation. Its quite different with quite different mechanisms (and like Tom I don’t care what they do as long as its the Right Thing!).

I guess what we havent seen much of so far in the presentational space is the personalisation of the customer journey itself. Amazon simply serves up the set of things that it thinks we might like, but in the same place, same wrapper, for everyone.

I wonder what steps Amazon might take to personalise the customer journey … or is what they do right now is, in fact, as far as they can go with personalisation? I mean they can’t alter the product can they? And therefore when we look to the future perhaps we should be looking elsewhere for our inspirations?

Andy Holyer wrote:

I find Amazon’s suggestions a lot less usable that they used to be - I think part of the problem is that it “muddies” the knowledge base over time.

The one really useful thing Amazon found for me was Kernighan & Pike’s “The Practice of Programming” - probably a bit low level for a lot of this site’s readers, but really good if you want to inprove your programming. This was suggested when the book was new, in 2000.

In 2002 we went on holiday to the Dordoigne in France; didn’t like it much, too touristy. However I’m still getting travel guides to the Dordoigne suggested to me.

I’ve often bought videos and books for my son as he was growing up: however Amazon needs to learn that six-year-olds don’t want to watch Teletubbies any more….

Andy Lloyd wrote:

Maybe sites like Amazon are not essential enough for personalisation to offer the benefits we all expected. If you make use of an online groceries store (e.g. I use www.tesco.com) then personalisation can be incredibly useful - to me as a customer (discovering new items, or replacements when the things I want are out of stock) as well as to the store (they get to sell me more stuff!). However, the best use of personalisation is the ability to re-order the same thing (not a priority for Amazon…)

Eddie Capstick wrote:

What sort of things can Amazon realistically offer that I would like, or can I imagine some others might like? Or even removing some of those items from long lists – what set of rules would you create for yourself. Here’s some unordered, unpriortised ideas you’ve inspired me to write up.

This author recommends: Myself as an aspiring author, if a bookseller/publisher said, ‘Please recommend a title (or more) for your readers, that you enjoy’ – I think I’d put a couple in – maybe even based upon what age I was and why I liked it.
This also has two other elements of author participation, whereby books are shown where this author has written a positive review (negative are rarely published), and introductions to other titles (mainly non-fiction here). It may be an urban myth, but I hear a few authors are more hooked upon looking at their reviews on Amazon than anyone else, so why not let them add to their ‘brand’ by recommending to their readers books that they like too.

A bit of humour: Show me books I’d really not want to buy. If I’m to make ‘friends’ with Amazon itself, well quite simply, I like humour/wit in my relationships. Make me laugh (or cry) at what my ‘friend’ thinks I wouldn’t buy. Would this make me rate more books ? Very probably. If I found this very funny, might I send this book title to a real friend, therefore adding more weight to my friendly feelings towards Amazon ?

Author future based: I’d like to think I’m quite up on the latest names of new releases (I still visit real bookshops), but I’d always like to be notified of pre-ordering
future releases of my favourite authors – at least every 4 months I type in their names and do this manually – even just allow me to add names to a list to check.

Friends based: We can already type in names and look at wish based lists (for anyone, not just my own friends - anyone with a stalker beware). If I/my friends ‘allow’ - Why can’t I add my ‘reading’ friends and see what they have been viewing / buying, and especially reviewing. This feature would be useful for a book club I know.

My rules for not showing: like don’t show me audio CDs (unless I re-ask them to be shown). Don’t try to upsell / cross sell to me when I’m looking for a particular booktitle / author. Don’t show these things (or at least click/scroll away from these choices) to me when what I really want is to find something specific. I don’t want to be sidetracked. I need this book as soon as possible, and only this book. Don’t confuse me with too many options, allow me to configure which ones are shown, and if I don’t configure them, sequentially show them through different sessions.

Keyword generation/generators: I don’t pretend to know how Amazon works, even though I first saw the site in ’96, and I’ve been part of teams that have built global sites selling books / cds / items with personality matching engines. Does anyone know how this site builds keywords? Is it a balance between: words in titles, descriptions & keywords put forward from publisher / category based keywords / previous user clicks (from keywords previous users, used to search and found (shown or clicked) this item)? I wonder how many different kinds of training sets they have trialled throughout different cultures/age groups etc. And which have been dismissed due to lack of data given by the users. How much % off the next purchase could I be bought for to give the info they desire ? Or am I self-interested enough not to be bought off, but after being shown the benefits, sit there and do it for own altruistic nature of helping myself, but also helping others (through Amazon) to submit this data.

By the way, I’ve just checked ‘what’s new for you’ section, for me, how did ‘Monsieur Hulot’s Holiday [1953], the video, get in there. Maybe my idea of showing me something I never want to buy has just been implemented. (*chuckle)

Mary Branscombe wrote:

The cost of gathering enough information to be personal to me is high either in user time or privacy costs of mining what I do. The ‘what my friends do’ idea is interesting - I know the folks over at Eurekster - but even my husband doesn’t have the same tastes as me. Taste covers organisation and navigation as well as products; how about sites that streamline themselves to prioritise the areas I visit most? Well, they’d like me to buy from all the other areas too so it’s still like having to walk all the way to the back of the store for the bread - maybe something will tempt me along the way.

Bob G wrote:

Another factor not mentioned above (unless I missed it) is gift shopping. The items I purchase for others may have nothing to do with my personal tastes and as such skew the training set in wild ways. The only hope it has, is that I buy a lot of stuff for myself and gift purchases can be treated as outliers.

Greg Linden wrote:

Hi, Tom. Great article. But I think you’re judging personalization on the wrong standard.

Personalization doesn’t have to be perfect. It just has to be better than the alternative.

What is the alternative? Unpersonalized content. How many useful and interesting items do you see on the front page of an unpersonalized e-commerce site? Almost none, I’d guess. The content they pick is almost entirely irrelevant and useless.

Personalization doesn’t have to be perfect. It doesn’t have to read your mind. It doesn’t have to always guess what you want. It just has to be more useful and more relevant to you than generic content.

[Full disclosure: I worked at Amazon and now lead Findory, a personalized news site]

Tom Chi wrote:

Those are some great points, Greg. I would add one caveat though. If a system is personalized and advertises this fact to the user, it creates the expectation that valuable personalized data is being presented. Given this expectation, a user will naturally raise the standard of what they expect, and they will use the site in a different way.

For example, if I go to Google News, the front page is not personalized, so it is up to me to do a combination of searching and browsing to find my news — this works pretty well. Now, if Google News were personalized, I would experience the site with a different mindset. I would probably spend less time searching since that work is supposed to have been done for me anyhow. As I’m browsing, I’d implicitly make value judgements on the quality of personalization, and ultimately the expectation that the content is personalized might actually slow me down compared to just searching for interesting stories.

So an interesting conundrum arises. Even though the personalized content might be better than completely unpersonalized content, the difference in user expectation can color errors more negatively than an unpersonalized system without those expectations. While the technology may be giving you better results in an absolute sense, you feel *worse* about it.

4rentinla wrote:

Few days ago i read an article about personalisation of websites. Nowadays more and more compnies trying to get personal with their website visitors in hopes of increasing sales. But unfortunately it showed up like very expensive thing.
The Jupiter research center arrived at a conclusion that “Given flexible, usable navigation and search, website visitors will be more satisfied with their experiences and will find fewer barriers to the profitable behaviour sought by site operators. In fact, good navigation can replace personalisation in most cases”.
In my opinion the personalisation of websites would be great if somehow we coudl make it in lower resources.


Leave a Reply


OK/Cancel is a comic strip collaboration co-written and co-illustrated by Kevin Cheng and Tom Chi. Our subject matter focuses on interfaces, good and bad and the people behind the industry of building interfaces - usability specialists, interaction designers, human-computer interaction (HCI) experts, industrial designers, etc. (Who Links Here) ?