The fallacy of Expectation Bias
- Thread starter mep
- Start date
(...) Then I must be deaf as a post or dumb. When we acquired Pacific Microsonics, the makers of HDCD, one of their engineers gave me a quick test of HDCD vs CD version of the same. I swore to him that the HDCD version sounded so much more open, had more detail, etc only to have him tell me that he was playing the CD version both times! After he told me this, I took the test again and this time they both sounded the same. You can say that expectation bias of them being the same wiped out the imagined differences
Amir,
When you say "quick test" you are saying it all. No "quick test" will show any small differences in a reliable way, and in these conditions you will be much more open to suggestion.
I have no doubt that expectation bias exists in audio and can influence our perception. To what degree it can affect our choices is open to debates.
Some times the real effect of the small differences in the typical audio challenges are so insignificant that they are not easy to perceive, and any hidden influence can bias us. This happens when a system is poorly "tuned", and the change of one component only changes between two average badly tuned systems. However, when we manage to get the system "in tune" suddenly the whole system sounds much better and small differences become real. In these conditions, the small differences are more easily noticed.
I refer to this as you told us about your experience with HDCD. When it was released I also carried a lot of unsighted tests with friends (I will not call them blind tests as the requirements for valid blind tests were not fulfilled). We found that using the same equipment (a nice Madrigal CDP upgraded with a new HDCD compatible board) we could easily distinguish HDCD form non HDCD versions in some systems and it was almost impossible in others.
It can be... with expectation bias!
Sorry could not resist.
Now that's really funny!
Now was that tap water, bottled or spring?
How much did it cost??
How did it tatse?
Was your opinion influenced by the cost?...........Smile
I've said this many times before, but it bears repeating at least as often as this audiophile myth is repeated. The science says that the opposite is true, that our auditory memory is very short. our ability to adapt our perceptions to meet our expectations is long, and that exactly the opposite is true -- the only reliable way to identify very subtle differences is through quick switching between what's being tested. Long-term listening is completely unreliable for identification of subtle differences.
Now, with that said, I don't doubt that biases change or are replaced by others over time and if you continue to do a lot of comparative listening you may very well find something that compares better and change your original opinion. But can you hear subtle differences better over hours or days than you can by switching rapidly between the two things you're comparing. No one doing the research seems to think so. No one testing hearing seems to think so. Only audiophiles seem to believe this.
Tim
Agreed here Tim, over the many known and unknown variables from which we "rank" a good from a bad system/component, one tends to sort them differently over time, this just happened to me while evaluating a loaned preamp which was mine some years back, now I am finding attributes that are compelling with my tastes which were either unperciptible or niot relevant when I first had it. - this applies for an inexact market definition.
If they are audible now, they were audible then, it is your perceptions, attitude, needs, taste...you...that has changed. The inanimate object, unless it is no longer functioning properly
, is unchanged.Tim
This may be true for research work in sound perception, and developments in audio where you are looking for specific aspects of sound, but not to to satisfy the "audiophile myth", as you say, of having listening pleasure using all the complicated methods that subjective perception and our brains allows us.
Do you know of any loudspeaker manufacturer that develops speakers listening exclusively to three minutes periods in blind conditions?
If they are audible now, they were audible then, it is your perceptions, attitude, needs, taste...you...that has changed. The inanimate object, unless it is no longer functioning properly
Tim
I don’t agree with that statement one whit. Just because information is encoded in the source material doesn’t mean you can hear all of it unless your system is up to the task. For instance, your speakers are missing at least 1 ½ octaves of bass as they are -6dB at 60 Hz. If you didn’t listen to headphones, you would have no idea how much information is there, being played, but not being reproduced.
On the issue of time: it does take considerable amount of time to learn to hear non-linear artifacts and material that best exposes that. So long term evaluation is good that way. But when it comes time to A/B comparison, speed is everything. You can easily test that yourself. Compress some material and do quick AB tests vs longer listening. You will find the quick AB to far more effective.
Perfect. But some audiophiles do not accept that evaluating audio systems is an A/B comparison looking for differences. BTW, IMHO, training people to look for some kind of artifacts makes them less sensitive to others.
IMHO, humans can not focus their attention in too many aspects simultaneously. If you train someone to become very sensitive to some specific aspects of sound, he will discard the others.
Sometimes I am just looking for certain details in a recording - then I release the appreciation of other aspects.
Not to beat a dead horse but I recall the examples of long term a/b tests showed no difference between long and short a/b tests. If I am wrong could someone cite a test to the contrary. E..G I recall the test where audiophiles were allowed to listen at home for lon terms with no significant difference. Of course long term listening is most effective in acquaintin one self with the characterisitcs of the components under evalaution. Objectivists often invite us to "listen as lon as we like." Once one is familiar with the chahracteristics then proceed to to the a/b/ test.
This perfectly encapsulates what's wrong with all this DBT business, it very mistakenly believes that you can take the unpredictable, untrustworthy "human" element of decision making out of the process, but all it really does is compound the issue in many, if not most, situations. The nature of the the human organism is that it gets very irritated, very irked at setups which cripple its ability to understand precisely what's going on, so frequently in these environments I'm sure a lot of the answers are made from the perspective of "I couldn't care less any more whether I'm right or wrong; I just want to get out of this situation!" ...
Frank
Last edited:
And herein lies a real dilemma in this audio business. Tim is exactly right, that quick switching will identify real differences between 2 versions with genuine, but slight differences. What undoes the value of this is that the differences are not static, they depend upon the relatively long term stabilisation of the system to manifest properly, and the better the system the more this is relevant. You're caught between the devil, and the deep ...
Frank
Should this thread be renamed "Why I reject science"? I mean, I've read a lot of bold claims but have seen no evidence; mythology and opinion for sure but no fact.
Expectation bias never, ever takes a vacation and one exhibits not a shred of humility by even suggesting one is immune from it. This forum has been active for 1 year and 9 months and I continue to read the same misunderstandings and straw man arguments regarding blind testing and bias, both of which are universally accepted as gold standards in measuring the perceptual world. Only capital A Audiophiles proclaim themselves with superhuman abilities. What a serious disservice to the audio hobby it is. What's Best Forum? Nah, at least in this thread this is Audio Asylum.
Expectation bias never, ever takes a vacation and one exhibits not a shred of humility by even suggesting one is immune from it. This forum has been active for 1 year and 9 months and I continue to read the same misunderstandings and straw man arguments regarding blind testing and bias, both of which are universally accepted as gold standards in measuring the perceptual world. Only capital A Audiophiles proclaim themselves with superhuman abilities. What a serious disservice to the audio hobby it is. What's Best Forum? Nah, at least in this thread this is Audio Asylum.
I'm sure DBT will work beautifully when the situation being assessed is static, that is, absolutely nothing varies in the state of the 2 situations, A and B: before the switching takes place, during the period of the changeover, and following the switching, for an indefinite period of time. Most research matters do deal in such situations, so there are no real problems.
But if you want to play science, I remember a very enlightening series of lectures where an academic who specialised in the field of measurement pointed out the extremely rocky road you go down if you want to be sure, absolutely sure that you're measuring exactly what you think you're measuring. Many times, he made clear, you were in fact measuring your limits of understanding of the matter, the subtle but extremely important inaccuracies in the equipment, and the fact the measuring process itself was impacting what you were measuring: you were contaminating the very process you were attempting to measure by your interaction with it ...
Frank
Similar threads
