Geoffkait said:
Re controlled blind tests, a single controlled blind test that has inconclusive or negative results has no significance
To which Analog Scott replied, “That’s simply not true. It is a failure to detect a difference. And that does have significance.”
Well, not necessarily. Your argument is over simplified. As I stated recently there are perfectly good reasons why an audio test fails to reveal what it’s supposed to reveal. Too many things can and do go wrong. There are too many variables. Who can devise a perfect controlled blind test procedure? Does everyone agree on the perfect test procedure?
Here are some examples of what can go wrong in any test, controlled double blind tests. Thanks to George Tice for some of these.
-The listener’s hearing is not up to the level he thinks it is.
- There are errors in the test system. E.g., out-of-polarity, etc. Whoever dictates what system the test is performed on controls the outcome. I.e., The Amazing Randi.
- The test system doesn‘t have the necessary resolution, dynamics or transparency to reveal the truth.
- The components of the test, including cables and power cords were not sufficiently broken and/or warmed up.
- The instructions, if any, were not followed.
- Unfavorable weather conditions
- The listener is overwhelmed by doubt and skepticism.
- Unknown variables.
The truth is acquired slowly over time by many tests on many systems by many listeners. Conclusions in order to achieve real significance must be repeatable and they must be transferrable.