Here’s a great and succint case study of how user research data can be easily misinterpreted, and a great example about why we should always be suspicious of statistics.
The marketing blog at FutureLab (which I do recommend) has a short post today entitled “Study Shows Fear of MySpace Predators is Overblown“.
The research paper (PDF) referenced in the post makes the following conclusions:
- Only 7% of those teens interviewed were ever approached by anyone with a sexual intent and nearly all of them simply ignored the person and blocked him from their page.
- Two-thirds of the parents were sure that there were many sexual predators on MySpace, while only one-third of the teenagers shared this concern.
- When asked about media coverage, 66% of the parents felt that it was either understated or close to the truth.
- Conversely, 58% of the teens felt it was vastly overblown.
At this point, several alarm bells should be going off in your head:
First, what parents or teens think of the media coverage is irrelevant. It’s a red herring. Perception of the media coverage has nothing to do to whether or not sexual predators are an actual problem.
Second, the very first statistic says that 7% of teens were approached by someone with sexual intent. That’s a huge number, really. If you just read the headline and didn’t read the actual body copy, you’d miss this. And if you read this number and didn’t think about it, you’d miss it. But 7% is a huge number!
Third, the study claims that nearly all of the teens who were approached were able to ignore or blocked the predator. If by “nearly all” they mean, say, 99 out of 100, that means that .07% of MySpace teens are approached by predators who are so aggressive that they cannot ignored or blocked. Assuming that there are, say, 10 million teens on MySpace (which is a low estimate, I’ll bet), that’s 7,000 incidents of aggressive predation. That’s a lot of serious incidents!
Finally, if you look at the end of the study’s press release, you will see that the study’s author is, in fact, writing a book about MySpace to “provide a candid view of the benefits and potential hazards of MySpace, and include helpful advice for parents to keep their children safe while allowing them to benefit from their virtual world experiences”.
The author’s conclusion may in fact be correct – MySpace may be perfectly safe. I’ll bet it is. It’s just that these numbers do not support that conclusion at all. Be alert!