[I]t’s not uncommon for people to be asking the question, “can you really hear the difference?” This is very good news for music and music lovers.
Rob then goes on to describe a particularly thorough method for conducting audio tests.
But it doesn’t matter.
If you have to squint to hear the difference, the difference doesn’t matter. We’re talking about comparing two different signal chains and the audio difference between them is so slight that you need an ABX test to tell them apart. If that’s the case, the minuscule difference will be dwarfed by other practical considerations. You know, real-life factors like cost, ease-of-use, ability to impress people, availability, etc.
MP3 encoding is a great example. Low-resolution (below 128kbps) MP3 encoding is easy to hear. You don’t need a controlled test to know when you’re down there. But above a certain point (192kbps for me), MP3 is *almost* indistinguishable from uncompressed audio. At that point, it doesn’t matter how close it is, because the fact that it takes a fraction of the time to transfer online is far more important.
In fact, MP3 encoding is a great example because if you’re encoding your audio with a lossy format, you already don’t care about perfect reproduction.
Amp sims are another example. They sound just fine. People doing extensive comparative testing are wasting their time (unless they’re developing amp sims themselves!). Amp sims suck. They don’t suck because they sound terrible. They don’t (they used to, but thankfully we’re past that point). They sound great. Amp sims suck because they don’t give you the experience of playing in front of a kicking amp (also, iso booths suck for the same reason!). A better performing experience will give you a better performance. And that is what matters.
-Kim.
Your concern about the imperceptibility of the difference between two things is exactly the sort of thing that ABX (and other similar methods of testing) exposes. Taking a few moments to be more quantitative in our decision making can free us from all of the silly minutia that you are (rightly) dismissing.
Hi Rob, thanks for dropping by.
Maybe we’re in agreement? I’m not sure.
What I’m trying to say is ‘if you have to ABX-test it to hear the difference, then the difference is not worth worrying about.’
-Kim.
Another interesting perspective on the irrelevance of ABX tests:
http://blog.szynalski.com/2009/07/27/should-we-care-about-abx-test-results/
“The relevance of ABX tests to the lives of music lovers is questionable. Neither does the absence of audible differences imply equal quality, nor does the presence of audible differences imply that the compressed version is inferior. Rather than being the argument to end all debate, the results of ABX tests are just one data point and the relative strengths of various audio formats may well be put in a new light by further research.”
-Kim.
ABX testing rarely reveals small differences between two audio source. Rather, it generally proves their irrelevance. ABX is just one of several accepted ways of actually proving what you’re says about small, nuanced differences not mattering. The difference is the attempt at objectivity.
Here’s a classic piece on ABX from 1990 via the Boston chapter of the AES:
http://bostonaudiosociety.org/bas_speaker/abx_testing.htm
This is a great post, I love your blog for a lot of the no-nonsense attitude abouth things, and I agree. If you have to go through meticulous measures to check, it doesnt even matter.
Nice article – thanks!
Interesting comment about objectivity. I’d say ABX testing is a middle-ground (in objectivity) between an intuitive judgement (which can miss some details) and a technical analysis (which can consider too many details).
-Kim.
Cheers – glad you like it!
-Kim.
Hi Kim,
I like your comment on Rob’s article. I tend to find such topics very interesting and are tempted to do some related testing. However you got it right: if the differences are so small that you need to use such techniques to find out, if there really is a difference, you could/should better use that time for something more productive.
). So I used variable bitrate encoding ranging from 128 kbps to 192 kbps with the highest VBR quality setting. Todays recommendation when using Lame is to use the few presets (V#), but choosing V2, which should come out as ~192 kbps, results in files mostly having 220 kbps or more. This is not necessary in my opinion. The V3 preset results in an average bitrate, which is too low for my taste (far below 192 kbps). So I just thought about using a CBR with 192 kbps at the highest quality setting (-q 0) and stereo mode. I would like to hear what encoding settings you prefer and why. Suggestions are welcome. 
I made some listening test with bitrates for MP3 compression and found the same number to be sufficient to hear no difference (maybe thorough ABX testing would show?
Thanx for sharing your thoughts via this blog. They are very useful, because they often don’t deal with the technical details but with focusing on the big picture and how to actually make more music.
Take care
Frank
Hi Frank,
I almost always use 192kbps CBR. I’ve written a bit about it here: http://blog.kimlajoie.com/more-things-you-cant-hear/
It sounds like your own listening tests have come up consistent with mine!
-Kim.
Great post Kim!
Thanks Justin! Great blog of yours too!
-Kim.