You can’t really validate or invalidate a design idea just by looking at it and declaring it a success or failure because of some best practice or design heuristic that usually works. You’re just talking about theories. Ultimately, no design disputes can be settled convincingly without making a model and testing it out.
But a theoretical debate about the strengths and weaknesses of a design is a critical first step that design teams must pass through before actually going off and testing their designs. To me, a good design critique is a kind of low fidelity user testing: using our imaginations instead of using a lab and test subjects.
Regarding the inspiration for my previous post, the absurd-looking application “Bulk Rename Utility“, the debate was almost predominantly based on “gut opinions” — albeit by many people with expertise in UI design. And I think that’s great. In my particular gut, I suspected that this app would surprise people and do well in testing. Others felt in their guts (and present compelling arguments, too) that this app would fail miserably in a user test.
What’s great about having this kind of hypothetical discussion at all (especially when the debate might seem to be easily and quickly settled by simply testing the application with real users) is that we learn more about the kinds of things we would need to think about and the questions we should be asking when we actually do test the application. Without debating the options we might not have uncovered (for example) these kinds of questions to ask during testing:
- Would different types of users react to the app in different ways?
- Do different apps with the same stated purpose serve different use cases?
- Is efficiency important, or a feeling of efficiency?
- Is preventing error of primary importance, or permitting error correction?
Sometimes the expert critiques and gut reactions are so compelling (whether positive or negative) that an experienced designer will know right away that formal user research would be a waste of time. Sometimes it’s just obvious — but only becomes obvious once you’ve thought it through, especially if you’re talked it through with other people.
And, of course, testing can be dead wrong. Seinfeld was famously rejected by test audiences, and finished last in the ratings in its first season.
Then there are the “unknown unknowns” (a Donald Rumsfeld-ism that I think is entirely valid and sensible). There are some design decisions that seem so obvious, and may even test well, but fail miserably because of an completely unforeseen factor in real-world practice. Our goal as designers considering design options is to try to minimize the number of unknown unknowns. And again, the best way to uncover unknown potential problems is to imagine as many of them as possible through lively debate from diverse viewpoints.
Whether you user-test your product or not, there’s no doubt that a lively, opinionated and adversarial discussion about any complex design decision, especially a user experience design decision, can only help the overall product development process.