Pono, a Hawaiian word for “righteousness,” is a portable digital media player and music download service for high-quality audio. It was developed by musician Neil Young and his company PonoMusic, which raised over $6 million on Kickstarter.
A PonoPlayer costs $399 and Yahoo! Tech founder David Pogue says that “The Emperor Has No Clothes:”
Is the Pono story a modern-day “Emperor’s New Clothes” fable? Were those famous rock stars just imagining things?
There was only one way to find out: conduct a blind trial, using identical songs on identical headphones, comparing the Pono with a standard audio player — an iPhone. So that’s what I did. You can watch the process in the video [below].
This is a studio “sip test” and as Justin Colletti writes in “The Art of Choosing Speakers:”
In his 2005 book Blink, author Malcolm Gladwell explored this phenomenon with the case of the “Pepsi Challenge”, a series of soft drink taste-tests that seemed to conclusively prove that Americans plain-old liked Pepsi better than Coke.
These soda shootouts made Pepsi a serious contender for the first time, and left Coca-Cola hemorrhaging market-share. Gladwell however, presents evidence that Pepsi’s overwhelming success over Coca-Cola in these tests was not evidence of a real preference, but rather a result of the flawed nature of the “sip test” method itself.
His research shows that when offered a quick sip, tasters generally prefer the sweeter of two beverages – even if they prefer a less sweet beverage over the course of an entire can. Just because a taster prefers a single sip of the sweeter beverage, Gladwell argues, doesn’t mean he’d prefer to have an entire case of it at home.
Coca-Cola found this out the hard way when they introduced “New Coke”, a soft drink completely redesigned to match Pepsi’s success in the sip test. The results were catastrophic.
It’s a lesson that shouldn’t be lost on the studio world. Where soda drinkers tend to prefer the sweeter of two beverages in a brief sip-test, listeners prefer the louder or more scooped of two sources in quick, non-contextual listening tests. Bass, treble, and volume are like sugar to our ears.
This is a demonstration of how the standard research techniques of corporate America are cheating us of many great experiences. Malcolm Gladwell describes this as the “Kenna problem:”
If I ask you why you think what you think, can I trust your answer?
This process [standard market research] is fundamentally flawed. It is completely screwed. We totally overrate the significance of what we find out when we go through this kind of formal process and the consequence of that overreliance on this system is that we are cheating ourselves of all kinds of wonderful experiences that we would otherwise have [such as Kenna].
Allen Farmelo also addresses this in “The Problem With A-B’ing:”
The problem is that these tests assume that because two things are close enough in a quick test that the difference will also be indistinguishable over long stretches of time. However, this assumption totally misses how it is that we tend to actually experience things in our very real lives….
If you want to do a real test of the differences, give people a music collection that’s all MP3s for a month, then give them that same collection as 24bit WAVs for a month, and then ask which one’s which, and I bet you will start to get some correct answers.
Why? Part of the answer is that, if given enough time, subtle differences will reveal themselves to us. Subconsciously at first, and eventually consciously, we become aware of new details, subtleties, nuances. We humans need time to truly come to perceive things in full detail. But details, once revealed, become important features in the big picture.