A fascinating article in last Sunday’s New York Times documents a recent study in which it is shown that predicting the success of cultural products (such as movies or music) is impossible, and that a phenomenon called “cumultive advantage” — where people prefer something largely because other people already prefer it — will usually overcome any empirical qualitative preference individuals may have for one product over another.

Marketers, for all their reliance on research, have long suspected this, which is why for years they have been looking to “coolhunters” to help them locate emerging tidal waves of coolness while the “cumultive advantage” is still building up steam. Instead of trying to create new products that will succeed because they were designed to meet a known and measurable consumer demand, they try to emulate products that are ascendant and that reveal previously-unknown consumer preferences.

This phenomenon may seem perfectly reasonable when it comes to movies and music, but I think it’s also true for user interface design: To the extent that any given UI can be called a “cultural product”, it is vulnerable to the wild unpredictability of culture. We may not always recognize it, but almost every UI is a type of cultural product.

This might seem hard to accept. Obviously, Justin Timberlake and Star Wars are cultural products, but the iPod, too, is a cultural product. The Nintendo Wii is a cultural product. Windows Vista is a cultural product. Amazon.com is a cultural product. These products have particular timeliness, particular aesthetics, and particular creative voices — thus they are cultural.

All of these cultural products have pure usability components to their user experience, but the cultural component — the product’s style — is often a major factor in the product’s success or failure. Sometimes it is the predominant factor, outweighing usability and feature-richness, as I think is the case with the iPod.

The ability to predict the success or failure of a UI design before a product is released is the foundation for the entire careers of many of us in the user experience design profession, so this argument may be troubling to many of us who think that there are empirically right and wrong ways of designing a UI. It’s hard to accept that a product’s hot color scheme, seductive finish, or ornamental trimmings — not to mention the brand name, ad campaign, or celebrity spokesmodel — could be far more important to the product’s success than the product’s long feature list or elegant ease-of-use.

I see the Times article as further evidence that no matter how many tests we do to show that one UI convention is better than another, when it comes to cultural products the “it depends” option is so overwhelmingly dominant that no conclusive best practices can ever be stated with confidence. Until you actually build something and have people use it, you will never know. And until then, the product development team’s resident “coolhunter” may have better insights into the product’s potential for success than anyone on the user research team.


3 responses to “Predicting User Experience Success”

  1. Eric Gauvin Avatar
    Eric Gauvin

    Yes. I think you’re right. In my experience, it requires more inspiration, passion and unquantifiable factors than objective measurable factors to arrive at a good user interface. The complex dynamic of communication (message, voice, audience, context), visual design aesthetics an interaction design is pretty hard to quantify. It’s nice to know about some of those aspects that can be measured, but the main part of what I do involves analysis, inspiration and creativity. However, since good UIs tend to rely heavily on existing conventions, they usually don’t look wildly creative.

  2. Terence Fitzgerald Avatar
    Terence Fitzgerald

    The points in the Times article seem to me accurate and are well-taken, as are your comments on it. However, business itself is a cultural construct, and often the way one achieves buy-in within that construct is through testing. Analysis, inspiration, creativity, and innovation are absolutely essential to the success of any venture. But producing (quasi)quanitifiable predictors of that venture’s success are equally responsible for the venture’s success.

  3. Eric Gauvin Avatar
    Eric Gauvin

    Yes. But I think that’s just saying it’s a good idea to consider usability best practices when designing a UI. I think far too many people think usability best practices define a good UI, but I think they’re really a pretty small part of the whole picture.