A recurring pattern in eReader vs Multi-purpose device reviews is the choice of devices and contexts that support the reviewer’s beliefs and what he/she wants to be true.
If a reviewer thinks the Kindle is better than the iPhone – Her/His comparison is lopsided right from the start. Plus this bias is invisible to the reviewer.
What exactly are we talking about?
- People tend to start with an outcome they think will turn out to be true.
- They approach the comparison from an angle that makes their assumed outcome likelier.
- They then choose devices and situations that make their assumed outcome even likelier.
- Even during the comparison they ignore obvious things that don’t match what they feel is right.
A lot of the times it’s not people being unethical – they literally don’t realize their comparison is lopsided.
Quick example of two lopsided comparisons – iPad vs Kindle
How could you set up a comparison to show iPad is better than the Kindle?
Remember that this is mostly done unconsciously –
- Compare the Kindle DX with the iPad so the iPad’s price becomes a non factor.
- Compare reading on newspapers, magazines, and books.
- Compare cover flow with the random music player on the Kindle.
- Compare Safari with the Kindle’s basic browser.
- Do the comparison in low light or normal light.
- Choose a huge PDF on the Kindle so that page turns are extra slow.
- Talk about innovative apps and show some truly innovative apps.
- Berate the Kindle App Store and point out the limitations i.e. bandwidth costs and slow refresh speed.
These are just the obvious things -
If you really want to make comparisons unfair you’d pick a cool person or a hot girl to show off your favorite product.
There are a lot more things that could be done and are done.
These last few tricks are obviously manipulation and much worse than an unconscious bias.
How could you set up a comparison to show the Kindle is better than the iPad?
Again we make unconscious choices that make our desired outcome likelier.
- Choose the $259 Kindle 2 to compare with the $499 iPad.
- Focus mostly on reading books.
- Highlight the free Internet and 60 second downloads.
- Compare the devices in sunlight.
- Focus on book prices – especially for books that are not bestsellers as they are likelier to be more expensive on iPad.
- Talk about distractions and highlight time-wasting apps on iPad.
- Focus on Kindle Apps that are suited to the Kindle i.e. simple games and reading related apps.
It’s really hard to realize a lot of these factors – to the point that you might think you’re being completely fair while being very biased.
Where does that leave us with comparisons and reviews?
Well, we could clearly show that most reviews and comparisons have conscious and unconscious bias.
It means that we have to approach reviews and comparisons from a completely different perspective.
Making reviews and comparisons very accurate
Step 1: Establish a clear purpose for the review and the device
The most important things to establish are -
- What purpose(s) are we using the device for?
- What is the purpose of the review?
The former is essential as it clarifies to the reader that the review meets their needs (or not). It also clears things up in the reviewer’s head.
The latter is important – Are you writing this review to make yourself happy about your choice? Is it to help people? is it to persuade people?
Sometimes establishing the purpose makes the review unnecessary – If you’re just writing something to make yourself happy it might not help anyone.
Step 2: Identify all possible bias and remove it or list it.
Firstly, this is for yourself so that you figure out whether you are just stuck in your beliefs and your assumptions about the world.
Secondly, this is for customers so that they realize what they’re working with.
There are various assumptions we make -
- That readers are all smart.
- That our readers have the exact same needs as we do.
- That a device is the same thing in our eyes as in someone else’s.
- That a company being evil or good affects the device (which is not always true).
It’s worthwhile to figure these out and make sure they don’t mess up your review or comparison.
Step 3: Look at the comparison from multiple perspectives
This is best illustrated with an example.
For a Kindle vs iPhone review you should look at things from at least these perspectives -
- A Kindle owner or someone who loves the Kindle.
- Someone who reads a lot.
- Someone who only reads once in a while.
- An Apple lover.
It’s going to ensure you don’t miss the really big things. You are still going to miss little things like the fact that some people LOVE the shape and transparency of the dialog boxes on the iPhone.
Step 4: Provide the reader recommendations for different scenarios OR let them choose
There will usually be 3 or 4 main scenarios. You have to address these in your review.
There will also be a few dozen additional scenarios – To be able to address these you have to list the strengths and weaknesses of both devices and also compare them along several important dimensions.
At some level you don’t know what factors are most important to a reader – so there has to be flexibility in how a reader can interpret a review.
There is no perfectly fair review
The biggest takeaway is that you can’t assume your review is perfect. You put out something that helps people and has as little bias as possible and that’s the best you can do.
When Peyton Manning can throw an interception in the Superbowl it’s foolhardy to assume something you write or review is perfect.
Where does that leave us?
A review or comparison changes from ‘telling users what to do’ or ‘making a decision for users’ to -
- Helping people make a smarter decision (for them).
- Minimizing the probability of regret.
- Letting them gather enough reasons (rationalizations) to do what’s in their heart.
In a way reviews and comparisons aren’t about getting things right – because ‘right’ is different for every single person.
They’re about helping readers make a decision that’ll make them happy.
Filed under: reviews Tagged: | truth about reviews
This is good information. I can see that you have been applying these ideas to your recent reviews.
Pointless aside: The Peyton Manning analogy doesn’t really work. He ranked 7th this season in most interceptions thrown.