Sure:
X and Z = 0.01 dB
Y and W = 0.001 percent (100 dB down)
So if a cable measures to those values or lower, I'm confident the sound did not change audibly and a DBT will provide further proof.
--Ethan
Double Blind Testing and the threshold of necessity
- Thread starter RBFC
- Start date
Sure:
X and Z = 0.01 dB
Y and W = 0.001 percent (100 dB down)
So if a cable measures to those values or lower, I'm confident the sound did not change audibly and a DBT will provide further proof.
--Ethan
Man, you still don't get it. Preference and absolute measurements for fidelity are not related.
My preference for audio quality is a flat response with low noise and low distortion. So I have no idea what point you're trying to make about my preference "going against" measurements. This further has nothing to do with testing. My preferences are based on a lifetime as a professional musician and recording engineer. Since I was a teenager just starting out recording, the goal for me has always been to achieve the same clarity and presence I hear on good sounding commercials recordings. How you see that as contradictory is totally beyond my understanding.
--Ethan
I have tested lots of wires, mostly for signal integrity at frequencies decades above the audio band. However, I have measured and heard (verified in DBTs) differences in "cables". Only thing is, the real difference was the way the cables interacted with the components. (Anyone remember Van Alstine? His take was essentially the same.) Beyond a certain level of competence, e.g. a speaker cable with low enough RLCG, no differences were apparent to listeners and only minor (barely measurable) differences to test equipment.
FWIWFM, the research I read and participated in during the 70's and 80's put audible thresholds well above those Ethan provided (i.e. I think you're safe, Ethan
). Also, my experience is that few people actually prefer a flat system, and that most will tolerate a lot more distortion than they think before it gets noticeable (let alone objectionable). As an example and IIRC, playing pure sine tones we found the majority in our listening tests could tell when the distortion hit 1%, but the percentage dropped to well below half if the source was music ( a "busy" signal). Virtually nobody could tell 0.1%, even with a sngle tone output. Again IIRC, we used several tones (100, 1000, 3000, 7000, 10000 Hz), and verified the system itself was <0.1% for the tests. We had to use headphones for some of the 0.1% testing.
FWIWFM, the research I read and participated in during the 70's and 80's put audible thresholds well above those Ethan provided (i.e. I think you're safe, Ethan
). Also, my experience is that few people actually prefer a flat system, and that most will tolerate a lot more distortion than they think before it gets noticeable (let alone objectionable). As an example and IIRC, playing pure sine tones we found the majority in our listening tests could tell when the distortion hit 1%, but the percentage dropped to well below half if the source was music ( a "busy" signal). Virtually nobody could tell 0.1%, even with a sngle tone output. Again IIRC, we used several tones (100, 1000, 3000, 7000, 10000 Hz), and verified the system itself was <0.1% for the tests. We had to use headphones for some of the 0.1% testing.Can I assume that if the cable introduces a difference of 0.011 dB or a distortion of 0.0011 percent you already hear it? By other words, what is your realistic threshold for audible differences?
Last edited:
Probably not. See Don's post above. I have learned to be conservative in my estimates because 1) there are variables - mostly related to masking - and 2) in extreme contrived situations artifacts can be heard at lower levels than usual. Of course, some people have better hearing than others, and we can also learn to recognize detail. I doubt anyone could reliably hear FR differences of 0.1 dB, and even 1 or 2 dB is tough to spot at the frequency extremes.
--Ethan
Thanks. And the threshold for distortion? In my opinion it is much more important than FR - even calibrated microphones have wider tolerances than 0.1 dB!
The irony of me argueing the need for DBT regarding listener preference is ot unapreciated. Maybe demanding its advocates be consistent.
It just dawned on me that there is support for the notion that listener preference is subject to DBT. I was so busy arguing I overlooked it. Sean Olive himself counducts extensive DBT for listener preference. Inddeed he advocates that the listener be able to discern differences to the point of educating listerners as what to listen for. I'll have links in the new year.
Happy new Year!
It just dawned on me that there is support for the notion that listener preference is subject to DBT. I was so busy arguing I overlooked it. Sean Olive himself counducts extensive DBT for listener preference. Inddeed he advocates that the listener be able to discern differences to the point of educating listerners as what to listen for. I'll have links in the new year.
Happy new Year!
It's all important. But I addressed distortion in my comment about artifacts. If the sum of all artifacts is 80 dB below the music, nobody is likely to hear them. In many cases artifacts are inaudible even when only 40 dB down.
--Ethan
That's different from the stuff I usually discuss in audio forums. Yes, a blind test is useful when deciding which speakers you like, because then you're hearing the speakers without being influenced by knowing their price. Or maybe you had a bad experience with some brand 20 years ago and are still prejudiced, even though speakers made by that company today are totally different.
My insistence on blind testing is more related to claims that make no sense, such as speaker cable elevators or replacement AC power wires improving clarity and fullness. If someone believes that, and refuses to accept that audio science suggests otherwise, a blind test will put that belief to rest. It amazes me when people oppose such testing. It's as if they don't really want to know.
--Ethan
A blind test does NOT do that [bolded section mine]. If it did, these discussions would no longer exist. You would be able to such conclusive proof and that would be that. Instead, there are either no rigorous tests performed, or what is performed covers one scenario out of a billion possibilities.
Take cable. You go and hook up a pioneer receiver to a bookshelf speaker with a cable elevator and without. You test it on 100 people and none hear a difference. Does that prove that the cable elevator cannot make a difference? If so, please explain how
.PS I don't use cable elevators
.
True.
Should be true but in fact it (regrettably) is not. People reject the scientific method, holding out for the possibility of some as yet unknown revelation.
My hesitance in this area is due to the large opinion of the audiophile masses. There's a saying that if you think something and you're the only one, you're probably wrong. If you think something, and many others feel the same, there may be merit to your position. The negative-feedback THD wars, fought vigorously and later shown to be misguided, are one example where science didn't clearly identify the parameters of the study. Science was correct, in that large doses of negative feedback reduced THD. There is nothing wrong with the scientific method, and it almost always produces irrefutable results. It is when experimenters lose sight of the overall picture that studying minutae can lead one astray.
So, while I believe the scientific method is sound, I wonder about the inferences drawn from studies that are not 100% applicable to the subject at hand.
Therefore, I'm a believer in the scientific method, and a believer in the results when the researchers clearly identify the problem.
Lee
So, while I believe the scientific method is sound, I wonder about the inferences drawn from studies that are not 100% applicable to the subject at hand.
Therefore, I'm a believer in the scientific method, and a believer in the results when the researchers clearly identify the problem.
Lee
Bill Woslow puts this well in the introduction section of his AES paper, Detecting Changes in Audio Signals by Digital Differencing.
http://www.libinst.com/AES Audio Differencing Paper.pdf
"1.1. The Audibility of Signal Differences
Throughout the history of audio engineering and (particularly) of audio component marketing, a number of discoveries have been announced for new types of distortions. Many factors have been said to cause or correct problems and to have noticeable audible effects in sound reproduction systems. New product concepts are commonly promoted as cures for such ills. Some examples include use of special cable geometries, amplifiers with particularly high slew rates, chemical
treatments for CD disks, or devices intended to control electromagnetic interference. In fact, according to some audiophiles, nearly anything in or even near a high resolution audio system can affect its sound.
But while such claims are common, objective evidence seldom can be found showing that these claimed distortions or factors can actually be differentiated by only hearing sound. There is often significant skepticism about whether some of these things really can affect an audio signal at all, much less to any audible extent. Many testimonial descriptions exist, but there is rather little that can be repeatably demonstrated. Even should a researcher choose to accept that such a claimed effect might be real, he would have no certain way during a product development to know whether or not he is improving related performance.
Objective testing methods for audible effects, such as double-blind A/B or ABX, do exist and are capable of verifying audibility of some changes in an audio signal or system [1]. But these methods can be time consuming and expensive to implement rigorously, and while they can confirm an effect to be audible, they can never conclusively prove any one factor to not be audible. A negative (inaudible) result can at best conclude that audibility wasn't demonstrated under the particular given conditions of the test. And should results strongly imply that an effect can not tected by ear, that conclusion is likely to be routinely dismissed by much of the high-end audiophile community. The other components in the system are accused of lacking adequate resolution to preserve subtle changes, or listening conditions during the test may be thought overly stressful or otherwise atypical.
Switch boxes used in the tests are suspected of degrading audio performance and masking the differences being listened for. For these audiophiles, believable conclusions are achieved only through “sighted” listening tests in which the listener already knows what he is listening to at each moment, describes the sound “subjectively” and (at least consciously) trusts only his ears. Such results, though,are of little or no use in engineering developments or scientific research."
Emphasis mine.
http://www.libinst.com/AES Audio Differencing Paper.pdf
"1.1. The Audibility of Signal Differences
Throughout the history of audio engineering and (particularly) of audio component marketing, a number of discoveries have been announced for new types of distortions. Many factors have been said to cause or correct problems and to have noticeable audible effects in sound reproduction systems. New product concepts are commonly promoted as cures for such ills. Some examples include use of special cable geometries, amplifiers with particularly high slew rates, chemical
treatments for CD disks, or devices intended to control electromagnetic interference. In fact, according to some audiophiles, nearly anything in or even near a high resolution audio system can affect its sound.
But while such claims are common, objective evidence seldom can be found showing that these claimed distortions or factors can actually be differentiated by only hearing sound. There is often significant skepticism about whether some of these things really can affect an audio signal at all, much less to any audible extent. Many testimonial descriptions exist, but there is rather little that can be repeatably demonstrated. Even should a researcher choose to accept that such a claimed effect might be real, he would have no certain way during a product development to know whether or not he is improving related performance.
Objective testing methods for audible effects, such as double-blind A/B or ABX, do exist and are capable of verifying audibility of some changes in an audio signal or system [1]. But these methods can be time consuming and expensive to implement rigorously, and while they can confirm an effect to be audible, they can never conclusively prove any one factor to not be audible. A negative (inaudible) result can at best conclude that audibility wasn't demonstrated under the particular given conditions of the test. And should results strongly imply that an effect can not tected by ear, that conclusion is likely to be routinely dismissed by much of the high-end audiophile community. The other components in the system are accused of lacking adequate resolution to preserve subtle changes, or listening conditions during the test may be thought overly stressful or otherwise atypical.
Switch boxes used in the tests are suspected of degrading audio performance and masking the differences being listened for. For these audiophiles, believable conclusions are achieved only through “sighted” listening tests in which the listener already knows what he is listening to at each moment, describes the sound “subjectively” and (at least consciously) trusts only his ears. Such results, though,are of little or no use in engineering developments or scientific research."
Emphasis mine
.
Yup. It is extremely hard to prove there is no difference. It is much easier to prove there is a difference.
I think they are useful in smoking out people who hear large differences when measurements say there shouldn't be any i.e. Nousaine going down to FL. to smoke out the guy (now deceased) who heard huge differences until he was put to the test. Lots of these various "reasons' differences were heard until they didn't know the identity are basically people not being able to admit they have a sight bias. Just my opinion. Oh, and I don't think everything sounds the same, but sometimes you have to say "thou doth protest too much".
Lee,
Very true.
The scientific method has strict rules that go beyond what can be posted in a few lines. Although it asks for experimentation and measurements, it does not mean that as soon as you show some basic measurements your are using the scientific method.
It is ridiculous to claim that our forum discussions use the scientific method. We try to get the help of the people in this forum who are more knowledgeable in audio discussing interesting subjects, such as acoustics, a bit of electronics, why using DBT and why small differences matter, as well of some of our own private addictions in hifi. When possible, it is good that knowledge of science helps to understand them.
Some others use their pseudo scientific methods to mock the high-end audio, knowing perfectly that most of the time we do not have a model to justify our subjective preferences and also taking profit of some existing charlatanism in this business. But they forget that a bad explanation does not eliminate the effect.
Most of the knowledge in audio was gained by experience and practice. Science can provide tools to help development and help to find better solutions, but most of the time can not prove the results are better sounding.
Check this thread. They require some specific information before you make your choice.
http://www.whatsbestforum.com/showt...ally-Evaluate-Sound-Quality&p=28309#post28309
He starts off way off:
I don't know what he means by genetically, beyond personal supposition, but statistically, most women hear better than most men. And by the time they get to be middle-aged men (most audiophiles), most women hear quite a bit better. Of course I'm shocked that the remainder of the article is personal observation, anecdotal information in support of the writer's existing opinion but supported by absolutely no data. But it is, indeed, a different perspective. I still hold that the most informal blind test, while absolutely inconclusive is, if nothing else, a lot more valid than a sighted and, therefore, deliberately biased one. Close your eyes. Trust your ears.
Tim
I hear your point Tim but I am not sure in reality is always true. Here is the problem. As soon as something has the labels "double-blind," AES paper" on it, the whole world accepts it as complete, properly done and conclusive. People run off with the tag line without understanding what they did. It is not their fault. Knowing how to perform a proper test better than the people who conducted them requires considerable amount of expertise with 99% of the Internet population does not have. Once a month I go and answer why the famous Japanese jitter paper doesn't prove Jitter is inaudible for example. But because it was blind, and was published, it must be valid. I much prefer that improper tests are not published or done than done incorrectly as a result.
Here is a concrete example. I can take three people off the street, play them MP3 and CD and have them all say they hear absolutely no difference. If I came here and put that forth, what will you do with that? Will you accept as proof that the two formats are the same as far as fidelity? I hope not
. A smart person would ask me what I used for material. Would know how MP3 algorithm worked and whether said samples were revealing in that regard. And finally, whether the people I found were discriminating about audio or not. I for one, much rather see no tests like this than to have bad ones done....Similar threads
