The fallacy of Expectation Bias

No, but it should, perhaps, make us a little less sure of them. At least that's my opinion :).

Maybe those of us on the "science" side of the argument might recognize that access to measurements comprehensive enough to be meaningful are actually pretty rare, and that the stuff that we expect to be neutral, based on incomplete measurements, brand, reviews, etc, might be just as compromised, or more, than some boutique gear we expect to be deliberately colored.

Maybe those on the "I trust my ears" side might recognize that their ears cannot be divorced from their perceptions, and all they can tell us is what they like. And, in recognizing no objective metric or standard beyond what they hear, it cannot ever be any more than that. They have chosen to make it personal, therefore it cannot be more natural, musical...fill in the pseudo-objective blank. It can only be what they like.

Tim

And here we need statistics. When groups of people share the same desire to get more of the “something that was missing” from the stereo reproduction, and using the technological means available reach it, with the help of the industry, we have the audiophile community as I see it. When I read the opinions of several people who were fortunate enough to listen to Paul Stubblebine great system separately and do not know each others, and report similar experiences I have two possibilities - thinking that their experiences were real or thinking Paul Stubblebine was a great hypnotist. As, in some conditions, but in a smaller degree :( I could experiment similar perceptions I choose the first hypothesis.

Steve wrote: It was tantamount to being at the symphony and hearing it live.

Others tell me: It is not possible, just listen to the recording

I pick the first, although I know it is risky, most of the time science will not support me and I will have to rely on the empirical knowledge of many friendly experts and amateurs. Besides it will allow me to enjoy reading some audiophile forums and magazines. :)
 
I am actually reasonably serious about this: so, again, as an example, has there been a study where the clarity or intelligibility of sound, any sort of sound, is seen to be dependent on how much, in this case, deliberate distortion is added to the sound? Not noise, that's the easy one, but quite precisely known levels of non random, non-linear distortion. Anyone?

Frank

Go to Earl Geddes site and look up his white papers on distortion and his Distortion Metric paper. You will be very surprised.

Rob

http://gedlee.com/Papers.htm
 
Last edited:
Since proper blind tests about most relevant aspects of sound are not within the reach of common audiophiles should we restrain ourselves from having opinions? :(
This is just tiring. This also is just patently false. Are you serious? Even without knowing what you consider the "most relevant aspects of sound", are you telling me that you cannot arrange a proper blind test in your room with your selection of source material at your leisure? People are doing these kinds of tests all the time in order to compare gear, codecs, source material, etc. Not too long ago our own expert Gary undertook a single blind test with his audiophile society in the Seattle/Pacific Northwest area, results of which he posted in our forum, and it would have taken only a tad more effort to make that test double blind.

Mark, here is another example.:rolleyes:
 
This is just tiring. This also is just patently false. Are you serious? Even without knowing what you consider the "most relevant aspects of sound", are you telling me that you cannot arrange a proper blind test in your room with your selection of source material at your leisure? People are doing these kinds of tests all the time in order to compare gear, codecs, source material, etc. Not too long ago our own expert Gary undertook a single blind test with his audiophile society in the Seattle/Pacific Northwest area, results of which he posted in our forum, and it would have taken only a tad more effort to make that test double blind.

Mark, here is another example.:rolleyes:

Ron-I have to side with Micro on this one for a number of reasons. I’m going to say that for many people in this hobby, setting up a proper blind test when auditioning *new* gear is just not practical. Even if you could find a stereo store or stores that would lend you gear to audition, you would have to coordinate the arrival of the people that are going to help you with your double blind testing with the arrival of the gear during the short time you are going to have it available.

There are no audio clubs in my area and I barely know any local people who give a crap about audio. I have no local high-end stores that I can drive to and borrow gear for a weekend. I have to find a store that has a return policy and pay shipping both ways if I don’t like the gear and return it. Throw in a double blind test on top of all that, and you can see that it makes life difficult. And that’s just electronics. Let’s try that with speakers that weigh 500 lbs per side.

Now mind you that doesn’t discredit the validity of performing double blind tests in order to pick out the best sounding gear. But at the end of the day even with a double blind test, you still have to rely on your ears to tell you the truth. And let’s be honest here. How many people on this forum do not purchase any audio gear until they have performed a double blind test that would meet the requirements of the scientific community? I bet it’s damn few, but I could be wrong.
 
And here we need statistics. When groups of people share the same desire to get more of the “something that was missing” from the stereo reproduction, and using the technological means available reach it, with the help of the industry, we have the audiophile community as I see it. When I read the opinions of several people who were fortunate enough to listen to Paul Stubblebine great system separately and do not know each others, and report similar experiences I have two possibilities - thinking that their experiences were real or thinking Paul Stubblebine was a great hypnotist. As, in some conditions, but in a smaller degree :( I could experiment similar perceptions I choose the first hypothesis.

Steve wrote: It was tantamount to being at the symphony and hearing it live.

Others tell me: It is not possible, just listen to the recording

I pick the first, although I know it is risky, most of the time science will not support me and I will have to rely on the empirical knowledge of many friendly experts and amateurs. Besides it will allow me to enjoy reading some audiophile forums and magazines. :)

I doubt that we need statistics at all. The odds are very, very good that all we need to know is what recordings were listened to and how they were recorded to know that the system was very, very good, but hte impression of it being "tanatamount to being at the symphony and hearing it live" had everything to do with perception, because the odds are very, very good that information was not in the recording at all -- an illusion created by the recording, enabled by an excellent playback system. But I'm not sure how this fits in this conversation at all. Are we supposed to conclude that, because several people, including Steve, were very impressed by Stubblebine's system that it is objectively superior and the impression was not a matter of opinion? How does that work? I like Steve; I have great respect for him. But I'm not quite ready to concede that his perceptions and objective reality are one and the same.

Tim
 
Hi Mark.

Ron-I have to side with Micro on this one for a number of reasons. I’m going to say that for many people in this hobby, setting up a proper blind test when auditioning *new* gear is just not practical. Even if you could find a stereo store or stores that would lend you gear to audition, you would have to coordinate the arrival of the people that are going to help you with your double blind testing with the arrival of the gear during the short time you are going to have it available.
This is true for sure, but only for some gear. Yes, speakers ideally would be evaluated blind but as you state it is highly impractical. Having stated that, there are many people who have evaluated speakers blind. And speaker cables, digital sources, preamps, etc., pose far less of a hurdle (or should I say hernia:)) for blind evaluation.

I sympathize with the issues you face due to remoteness and isolation. But to state that "blind tests about most relevant aspects of sound are not within the reach of common audiophiles" is just patently false.

Now mind you that doesn’t discredit the validity of performing double blind tests in order to pick out the best sounding gear. But at the end of the day even with a double blind test, you still have to rely on your ears to tell you the truth.
That's the whole point, at least as it pertains to this thread. One certainly can attempt to minimize and hopefully eliminate expectation bias. I am in complete agreement with the notion of trusting one's ears; the effort to minimize and hopefully eliminate expectation bias is to avoid misplacing that trust.

And let’s be honest here. How many people on this forum do not purchase any audio gear until they have performed a double blind test that would meet the requirements of the scientific community? I bet it’s damn few, but I could be wrong.
This forum? Agreed.
 
I am in complete agreement with the notion of trusting one's ears; the effort to minimize and hopefully eliminate expectation bias is to avoid misplacing that trust.

I'm of the opinion that if you're unwilling to listen blind, you don't trust your ears, and that if you've never listened blind, you don't know if they are trustworthy. Mark is a perfect example; he seemed absolutely convinced of the superiority of his tube-based system until circumstances knocked him out of his groove. A little casual blind listening may have created that event years, and thousands of dollars earlier.

Tim
 
I sympathize with the issues you face due to remoteness and isolation. But to state that "blind tests about most relevant aspects of sound are not within the reach of common audiophiles" is just patently false.

Ron-I have no idea who you are quoting, but it’s not me. I never said that. You need to find the person that made that quote and attribute it to them because it sure appears you are attributing it to me.


This forum? Agreed.

Ron-Never mind this forum, what percentage of audiophiles in the entire world do you think base their purchasing decisions on a double blind test? I bet it’s miniscule, but I don’t have any scientific polling data to back it up. It’s just that I have never met anyone in my many years in this hobby that ever based a purchasing decision on a double blind test, let alone one that would pass the muster of the scientific community. Even if your experience is different and every audiophile you know doesn’t buy any new gear unless it undergoes a scientifically approved rigorous double blind test, I think it’s the exception and not the norm.
 
I'm of the opinion that if you're unwilling to listen blind, you don't trust your ears, and that if you've never listened blind, you don't know if they are trustworthy. Mark is a perfect example; he seemed absolutely convinced of the superiority of his tube-based system until circumstances knocked him out of his groove.

Tim-I hope you read my post above and saw that it is not practical in the least for me to listen double blind in my room in my house. I got knocked out of my groove based on what my ears told me, and not because of a double blind test.

A little casual blind listening may have created that event years, and thousands of dollars earlier. Tim


Is there really any such thing as “casual blind listening?” Who the hell ‘casually’ listens to music that way? You are either in a locked down mode under pressure to make the right decision while people are exercising/evaluating your ability to make decisions about sound quality while meeting the criteria for a double blind test that will stand up to the scrutiny of the scientific community or you are not.
 
Go to Earl Geddes site and look up his white papers on distortion and his Distortion Metric paper. You will be very surprised.

Rob

http://gedlee.com/Papers.htm
Thanks for your response, Rob, I appreciate that. As you can see, I ended up in the right area anyway, so now have a better handle on where the "academics" are ...

Cheers,
Frank
 
Hi again, Mark.

Ron-I have no idea who you are quoting, but it’s not me. I never said that. You need to find the person that made that quote and attribute it to them because it sure appears you are attributing it to me.
It was Micro. You then responded to my post in which I stated, amongst other things, that his claim was patently false: "Ron-I have to side with Micro on this one for a number of reasons".

Ron-Never mind this forum, what percentage of audiophiles in the entire world do you think base their purchasing decisions on a double blind test? I bet it’s miniscule, but I don’t have any scientific polling data to back it up.
Nor do I and if I were a betting person I would not take your bet.

It’s just that I have never met anyone in my many years in this hobby that ever based a purchasing decision on a double blind test, let alone one that would pass the muster of the scientific community. Even if your experience is different and every audiophile you know doesn’t buy any new gear unless it undergoes a scientifically approved rigorous double blind test, I think it’s the exception and not the norm.
I have and I believe Amir has. Frantz has posted several times about his Nordost Valhalla speaker cable blind test and I believe other tests as well. I'm not sure that is either here or there. I'm trying to keep my discussion within the confines of the topic of this thread: expectation bias. That some/many/most audiophiles do or do not purchase gear without first undertaking some level of blind testing, double blind or otherwise, is not in any way dispositive of the validity of the scientific method. This is not a popularity contest. We don't vote for a winner here. Indeed the substance of your proposed bet is highly supportive of the rampant failure of Audiophiles to account for expectation bias.
 
This is just tiring. This also is just patently false. Are you serious? Even without knowing what you consider the "most relevant aspects of sound", are you telling me that you cannot arrange a proper blind test in your room with your selection of source material at your leisure? People are doing these kinds of tests all the time in order to compare gear, codecs, source material, etc. Not too long ago our own expert Gary undertook a single blind test with his audiophile society in the Seattle/Pacific Northwest area, results of which he posted in our forum, and it would have taken only a tad more effort to make that test double blind.

Mark, here is another example.:rolleyes:

Unhappily you can not resist to question people intentions with an insulting and condescend chord.

Any properly carried blind test needs several people, calibration and/or proper positioning each time.
If we choose cables we have to change all of them - interconnects and speaker cables are sets .

Completely different from evaluating file formats sitting at a computer desk with headphones.

If you have objective data about properly carried tests in which you participated involving systems please describe them completely and we will learn from you.
 
The audio community is divided into 2 camps: those who have experienced audio replay at a very high level or believe others have done so; and those haven't or don't. The two sides are obviously somewhat at loggerheads, it's mighty close to a political left and right thing, so it's not clear how or if ever the two groups can ultimately reconcile. Tim's at the point where even if you expose him to such a system working well he'll refuse to accept the experience: for him it will be the effects of a "magic" room, or "hypnotic" room treatments, it most certainly won't be because he's actually hearing more of what's on the recording, he's convinced himself of that. It simplifies life of course thinking this way because it means that you don't have to make extra efforts to improve the quality of playback, because there is nothing ultimately particularly significant to be achieved by following this route.

But for the other camp it's not so easy: they have glimpsed or periodically enjoy the delights of this "higher" experience, so they're commited to the endeavour of pursuing it. All I'll say now is, may good fortune and the rewards of such an exercise be enjoyed by yourselves, because you deserve it ... :b

Frank
 
Last edited:
Micro

I would agree with you that conducting a proper "scientific" Double blind test is a difficult endeavor .. A simple blind test is , for the lack of better words, an eye-opener :) and it is really easy to do. Now, the results may not be as you term them"proper" but such tests however casual that they may be, remove more biases than sighted, I-know-already-what-I-am-about-to-hear tests, that we , audiophiles, myself included regularly conduct ... The point is not so much to eliminate ALL biases , that is a pretty difficult to do, maybe impossible; but to diminish their impact. This is better accomplished with knowledge removed than with sighted evaluation .. And that is a fact, not an opinion.
 
Any properly carried blind test needs several people, calibration and/or proper positioning each time.
If we choose cables we have to change all of them - interconnects and speaker cables are sets .
We need only one person, the person who will be purchasing/evaluating gear. We (or at least I) are (am) talking about that person's expectation bias, no more, no less. Heck, do a single blind test. Do something, anything, other than evaluating with both your eyes and ears, lest we conclude that "trust your ears" is an empty slogan. What positioning does one need to do, e.g., to evaluate 2 DACS? Heck, Gary just posted about a single blind evaluation he and his audio society undertook with 4 DACs.

I will not touch your assertion about cables being sets with a 10 foot pole.

Completely different from evaluating file formats sitting at a computer desk with headphones.
So we cannot compare, e.g., WAV and FLAC files, through speakers?

All of these things have been done and done repeatedly by those who respect the scientific method and who readily acknowledge the potential impact of bias. That one seemingly defends a practice which does nothing to even attempt to minimize bias speaks volumes.

When evaluating gear, why is it necessary to use one's eyes? Does the sound truly change if one knows what gear is in operation? If one won't trust his/her ears and only his/her ears, then why the heck should I? This is an exercise in intellectual honesty.

I'm not out to convince you and those who've already cemented their opinions on the subject of expectation bias. But we may have other members who have not reached any conclusions and we have a lot of guests who read our forum but who have not (yet) joined. It is these people who hopefully may benefit from a more balanced presentation of some of the aspects of expectation bias.
 
Tim-I hope you read my post above and saw that it is not practical in the least for me to listen double blind in my room in my house. I got knocked out of my groove based on what my ears told me, and not because of a double blind test.

They are the same thing. The "double" part is not about covering your ears.

Is there really any such thing as “casual blind listening?”

Sure, I used to do quite a bit of it. It's not about double, there is no X, no one is trying to carry it to a statistically valid sample or terribly worried about the kind of methodology that would satisfy a review board. It's just about finding a way to remove knowledge of what's playing when from the listening experience. And as I've said here many times before, it doesn't "prove" anything, but it sure as hell can provide some strong indications of how significant something is or isn't. And I've never been in a locked down mode under pressure. I've always done it at home, on my own system, on my own schedule. It's just finding a way to switch back and forth between two components without knowing which one is playing. The rest is just listening. It can be as relaxed or as stressed as you're capable of being.


Tim
 
But it's also really good for eliminating expectation bias and revealing preferences without the benefit of sighted knowledge.

I don't recall the exact context but it was said that gross diffefrences blot out bias. You might imagine Tim I remain skeptical.
 
Everyone believes this sort of thing of themselves. I spent my career in advertising. No one is effected by it. We know. We ask them. And no one, especially men, are influenced by ads, especially emotional appeals, in the slightest. Yet the campaigns keep boosting sales. Even the ones targeted at educated males. Especially the ones based on emotional appeals.

Tim

You make a good point Tim. I have been influenced by ads. I have also been groosly diappointed by my purchase decision.
 
.
Any properly carried blind test needs several people, calibration and/or proper positioning each time.
If we choose cables we have to change all of them - interconnects and speaker cables are sets .

Not to mention the universal absence of a control group.
 
This thread is quite something!
With a lot of good content from the great contribution of several well calibrated members.

Many good and elaborated ideas...

...I am learning more here than I can contribute; I like it! :b
 

About us

  • What’s Best Forum is THE forum for high end audio, product reviews, advice and sharing experiences on the best of everything else. This is THE place where audiophiles and audio companies discuss vintage, contemporary and new audio products, music servers, music streamers, computer audio, digital-to-analog converters, turntables, phono stages, cartridges, reel-to-reel tape machines, speakers, headphones and tube and solid-state amplification. Founded in 2010 What’s Best Forum invites intelligent and courteous people of all interests and backgrounds to describe and discuss the best of everything. From beginners to life-long hobbyists to industry professionals, we enjoy learning about new things and meeting new people, and participating in spirited debates.

Quick Navigation

User Menu