Why would you value a 'blind test' over what an expert recording engineer, such as Barry Diament, thinks?
There are a lot of problems with blind tests, and there has certainly been much discussion about the arguments and counter arguments.
If you wheel in a bunch of untrained listeners off the street and get them to listen to recordings they are not familiar with, using a Hi-Fi that they are not familiar with, in stressful un-relaxed circumstances. Why would you expect to get some kind of definitive answer about 16/44.1 vs 24/192 for instance, that somehow trumps the opinion of highly regarded recording engineers?
I thought I listed some of the possible flaws in blind tests - there is nothing unscientific about that.
If you value the results of any sort of blind test, no matter how badly conducted, over the opinions of recording engineers, then it doesn't seem to be a purely scientific matter to me.
Your methodological criticisms are sound, it‘s just that they don’t seem to apply to the quoted paper (I found the PDF): http://www.mesoscale.nl/aes_article.pdf
If we are talking about whether 16 bits is sufficient dynamic range (the main subject of this Hacker News discussion) they say:
"In one brief test with two subjects we added 14 dB of
gain to the reference level quoted and tested the two
sources with no input signal, to see whether the noise level
of the CD audio channel would prove audible. Although
one of the subjects was uncertain of his ability to hear the
noise, both achieved results of 10/10 in detecting the CD
loop. (We have not yet determined the threshold of this
effect. With gain of more than 14 dB above reference,
detection of the CD chain’s higher noise floor was easy,
with no uncertainty. Tests with other subjects bore this out.)"
To me, this confirms that a bit depth of 16 is insufficient for high dynamic range music such as classical orchestral music. Maybe we don't need more than 20 bits (or about 16 bits plus 14 dB), but as we have the disk space, internet bandwidth and electronics to comfortably handle 24 bits I don't see the problem.
As far as sampling rate is concerned, they aren't comparing 24/192 PCM with 16/44.1 and so it isn't really relevant to a discussion about whether it is possible to hear the difference between these two formats using a current state of the art DAC.
I've no idea about the pros and cons of convertings SACD to 16/44.1 and doing a comparison as I don't personally care about SACD and don't think it has a future in downloadable non-physical formats.
They only talk vaguely about the actual equipment used which isn't normal for a Hi-Fi review. They say they inserted a comparator:
"always in the 16/44.1 signal path. Audio switching was
handled by an ABX CS-5 double-blind comparator"
Have they done a double blind test to ensure that the effects of the comparator were inaudible?
They don't say what DAC or CD player they were using:
"For the CD loop we used a well-regarded professional
CD recorder with real-time monitoring."
I don't have enough to go on here. Certainly DAC and CD players have improved a great deal in the last five years since these tests were made. From the description I can't tell whether of not the CD player and its DAC were state of the art five years ago.
So overall I agree the paper is an interesting read, but hardly the last word in answering the question of whether we should move to 24 bit recordings, or whether a sampling rate of 44.1 KHz is sufficient.
