The fallacy of Expectation Bias

[Phelonious Ponk]

I don't recall the exact context but it was said that gross diffefrences blot out bias. You might imagine Tim I remain skeptical.

I think there's some truth in that. They blot it out or they clearly define it. If you love vinyl, through low powered SET into horns, you're not likely to suddenly decide you like hi-res digital files through active monitors and subs better, just because your eyes are closed. And if you're listening blind just to see if you can hear the difference, not to choose a preference, well, of course, the more subtle the difference, the more useful the blind listening. If you can't identify a difference between Klipschorns and Magicos with your eyes closed, we have a whole other set of issues.

Tim

 

[mep]

Again, my point to this thread is that we all *suffer* from expectation bias. At least in my case it doesn’t mean that we can’t be intellectually honest with ourselves at the end of the day and admit that our bias was flawed and the more sexy, expensive component doesn’t sound better than its dowdy cheaper cousin. The truth is what the truth is.

 

[Phelonious Ponk]

Again, my point to this thread is that we all *suffer* from expectation bias. At least in my case it doesn’t mean that we can’t be intellectually honest with ourselves at the end of the day and admit that our bias was flawed and the more sexy, expensive component doesn’t sound better than its dowdy cheaper cousin. The truth is what the truth is.

Agreed.

Tim

 

[fas42]

If you can't identify a difference between Klipschorns and Magicos with your eyes closed, we have a whole other set of issues.

Tim

And the joke of it is that if the rest of the system is working correctly, and the two speakers have their worst weaknesses tweaked away, then you won't! Tim's expectation bias is that different styles of speakers will sound different, obviously different; mine is that if you reduce certain types of distortion to a certain level that the "obvious" tonal differences between different components will evaporate in the face of the acoustic signature of the recording dominating the auditory experience. Horses for courses ...

Frank

 

[fas42]

At least in my case it doesn’t mean that we can’t be intellectually honest with ourselves at the end of the day and admit that our bias was flawed and the more sexy, expensive component doesn’t sound better than its dowdy cheaper cousin. The truth is what the truth is.

Could almost get so bad that a "HTIAB" could actually sound as good as a "high end" audio system in some key ways. And that's really scary, scary stuff ... ,

Frank

 

[Robh3606]

Could almost get so bad that a "HTIAB" could actually sound as good as a "high end" audio system in some key ways. And that's really scary, scary stuff ... ,

Possible however an HTIAB system will always be limited by the design compromises imposed on it by it's designers. A true SOTA system will not suffer from those limitations. It may not be obvious based on the program material but at some time it will be painfully obvious when you either run out of frequency extension, dynamic capability or what have you. It's not all that scary either. I would expect mosts systems to handle typical program material quite well in this day and age. It also depends on your expectations which brings us back to the topic at hand.

Rob

 

[NorthStar]

Expectation...

Again, my point to this thread is that we all *suffer* from expectation bias. At least in my case it doesn’t mean that we can’t be intellectually honest with ourselves at the end of the day and admit that our bias was flawed and the more sexy, expensive component doesn’t sound better than its dowdy cheaper cousin. The truth is what the truth is.

Exactly! ...And the same with some very good looking women. :b

* But sometimes our expectations are right on the mark!
...Rarely though. ...And some other times; more! We get more that we bargain for!

Talk about falling from grace! ...Disgrace? :b ...The fallacy of it.

 

[sasully]

The common *wisdom* is that if you think a new purchase is going to sound good (based on looks, cost, perceptions, reviews, etc.), it will sound good and meet your expectation bias. The only way to rule out expectation bias is to sit you in a chair, blindfold and gag you, and then switch your components back and forth and see if you can pick out your new component that stole your expectation bias heart.

This straw man immediately shows you don't know what you're talking about. So maybe you shouldn't have bothered starting this thread, mep.

 

[amirm]

This straw man immediately shows you don't know what you're talking about. So maybe you shouldn't have bothered starting this thread, mep.

We show more respect to our fellow members on WBF. Please don't force me to use the delete button and stay focused on the topic and not the person.

 

[Johnny Vinyl]

This straw man immediately shows you don't know what you're talking about. So maybe you shouldn't have bothered starting this thread, mep.

If you're so confident that Mark doesn't know what he is talking about, perhaps the better reply would have been to offer up your expert opinion. "Troll" posts or "flaming" isn't welcomed here.

 

[microstrip]

Micro

I would agree with you that conducting a proper "scientific" Double blind test is a difficult endeavor .. A simple blind test is , for the lack of better words, an eye-opener and it is really easy to do. Now, the results may not be as you term them"proper" but such tests however casual that they may be, remove more biases than sighted, I-know-already-what-I-am-about-to-hear tests, that we , audiophiles, myself included regularly conduct ... The point is not so much to eliminate ALL biases , that is a pretty difficult to do, maybe impossible; but to diminish their impact. This is better accomplished with knowledge removed than with sighted evaluation .. And that is a fact, not an opinion.

Frantz,

Any one with good sense will agree that expectations bias is avoided if the test is blind. However I have good reasons to question the practical aspects.

Consider testing amplifiers. Most probably they will have different gains. Every time you change the amplifier you have to calibrate the system for equal volume. 99% of the audiophiles do not know how how to do it. They will do their blind tests with volume settings differing by two or three dB. (80% of the people who want to compare amplifiers ask me to use the same volume setting in the preamplifier and will swear that when I change it I am cheating ... ) You can not imagine the number of people who claim to prefer the CD7 of Audio Research to the CD8 in blind or sighted tests just because the CD7 has 2.7V nominal output voltage and the (better) CD8 output is 1.9V. Or the people who claim balanced is better than single ended because of the 6 dB difference.

This comment applies to all electronic devices. People already discarded speakers as not practical to test blind. What else? Cables, that some who disagree with my views claim that make no change?

BTW, I have carried many proper blind tests. However they were private, and if I refer to any result that was not certified by an attorney, a priest and a doctor, I will be immediately remembered they were not scientific and referred to the Grand Inquisitor. So I will refer to them as funny experiments to entertain friends and children. One of funniest was comparing an all QUAD system (ESL63 + 44 + 405 + CD67) to another pair of ESL63 driven by all Audio Research units. No switches were used - the four speakers were side by side, but not symmetrically connected, the image shifted a little when I changed system, but no one knew which was the system. Every time similar copies of the CDs were used, all switching work was done by remotes. We also compared ESL63PRO with and without cloth and grille using four speakers - and the results were surprisingly consistent, people preferred the (withdrawn...) as it had better bass.

 

[Phelonious Ponk]

Consider testing amplifiers. Most probably they will have different gains. Every time you change the amplifier you have to calibrate the system for equal volume. 99% of the audiophiles do not know how how to do it. They will do their blind tests with volume settings differing by two or three dB.

Equalized volume is pretty important. You can easily buy a db meter for less than $50. Done. If that's too much, do multiple trials at different volume levels (do it with the db meter; that's even more interesting). If you're testing a $1,000 Emotiva against a $10,000 pair of monoblocks and 3db can make the listeners think the Emotiva is the better amp, you just learned something very valuable. If you learn that the mono blocks are preferred, even when down by 3db, you'll learn something equally valuable.

Tim

 

[microstrip]

Equalized volume is pretty important. You can easily buy a db meter for less than $50. Done. If that's too much, do multiple trials at different volume levels (do it with the db meter; that's even more interesting). If you're testing a $1,000 Emotiva against a $10,000 pair of monoblocks and 3db can make the listeners think the Emotiva is the better amp, you just learned something very valuable. If you learn that the mono blocks are preferred, even when down by 3db, you'll learn something equally valuable.

Tim

Tim,

The questions is not for the usd 50 expense - I can match sound levels to .1dB using proper calibration. But if you have to match levels with a sound meter every time you change something with an analogue potentiometer the efficiency of the switching is compromised. And a sound meter in the hands of inexperienced users ( a kind word for typical non technical audiophile) has a typical error of +/- 3dB. Replacing proper calibration by multiple trials is, IMHO, a controversy advice.

Non metered volume setting is one of the worst kind of manifestations of bias expectation in sighted comparisons - people always complain that their preferred equipment is playing softer and ask to increase the volume.

Thanks for the kind "learnings" about the Emotiva, but I am not in the audio kindergarten anymore, I now consider myself in the primary education classes.

 

[NorthStar]

Tim,

The questions is not for the usd 50 expense - I can match sound levels to .1dB using proper calibration. But if you have to match levels with a sound meter every time you change something with an analogue potentiometer the efficiency of the switching is compromised. And a sound meter in the hands of inexperienced users ( a kind word for typical non technical audiophile) has a typical error of +/- 3dB. Replacing proper calibration by multiple trials is, IMHO, a controversy advice.

Non metered volume setting is one of the worst kind of manifestations of bias expectation in sighted comparisons - people always complain that their preferred equipment is playing softer and ask to increase the volume.

Thanks for the kind "learnings" about the Emotiva, but I am not in the audio kindergarten anymore, I now consider myself in the primary education classes.

What can we say? :b ...Wish I had more money?

"Kindergarten", I read it right here, at WBF, and from this humoristic quote just above; I like! :b

 

[Phelonious Ponk]

And the joke of it is that if the rest of the system is working correctly, and the two speakers have their worst weaknesses tweaked away, then you won't! Tim's expectation bias is that different styles of speakers will sound different, obviously different; mine is that if you reduce certain types of distortion to a certain level that the "obvious" tonal differences between different components will evaporate in the face of the acoustic signature of the recording dominating the auditory experience. Horses for courses ...

Frank

This one should have a sticky on it. Through teaking away the distortions that cannot be measured, you can make Magicos indistinguishable from Klipsch. It reminds me of this one, which I saved for posterity...

My point about a high performing system is that it "shifts" the sound of irrelevant elements in the playback to another plane, or area of sound, that makes it easy for the mind to disregard it.

...in which you attribute to your tweaks a level of artificial intelligence sufficient enough differentiate signal from noise, shifting the bad stuff into the background.

I don't doubt that you hear these things, frank. And I don't doubt that the hearing of them is brought on by your tweaks. I just think that these, and many of your posts, belong in the expectation bias thread, as very extreme examples of the phenomenon.

Tim

 

[fas42]

I don't doubt that you hear these things, frank. And I don't doubt that the hearing of them is brought on by your tweaks. I just think that these, and many of your posts, belong in the expectation bias thread, as very extreme examples of the phenomenon.

Tim

Well, it's certainly part of my expectations these days. To repeat myself, hearing the "good stuff" was purely accidental in the first instance, it was never something I worked towards because I believed it possible, or heard it on someone else's system.

The very unfortunate part, for me, because I do now "expect" a system to get to this point, is that this whole audio business is very frustrating a lot of the time, because for one reason or another the setup is not firing sufficiently well for this quality to be realised. Just because I may have expectations, don' make it 'appen out of thin air ...

The bit about speakers makes sense: tonal qualities of such are just another form of distortion, so reducing distortions in an overall sense brings the character of widely varying speaker types into alignment. I've experienced this over and over again in my fiddling, the better the performance the less the speaker idiosyncrasies are apparent. Of course, going into a typical high end audio store this means nothing: every speaker there is screamingly, obviously different from the next, all the different distortions are painfully apparent. Having encouraged the friend to get into heavy duty optimising and tweaking it's fascinating to see the process yet again: very different electronics and speakers from mine, but the tonal signatures are getting closer and closer. In fact, he's ahead now in many areas, purely because the speakers particularly, and much of the electronics are better quality than mine ...

Frank

 

[Gregadd]

Any one with good sense will agree that expectations bias is avoided if the test is blind. However I have good reasons to question the practical aspects.

would that it were that simple.