The fallacy of Expectation Bias

[mep]

My point in starting this thread was to state that I clearly have expectation biases, but the truth will win out in the end. And in the case of my OP, the end didn’t take very long to get to. Now that was a simplified expectation bias that I only had $69 plus shipping invested in. My other examples were based on negative expectation biases of cheaper SS gear vice more expensive tube gear. I had over $5K invested in my tube preamp and when I inserted the Yamaha C2a preamp to replace it while the tube preamp was off in upgrade land, my expectation bias told me the cheap Yamaha preamp was going to be a dirt road in comparison to my beloved tube preamp. That bias blew up too even though I was emotionally committed to tubes and financially committed to my tube preamp. What I felt was the truth conquered the expectation biases I had.

I don’t think anyone is immune from having expectation biases, but I would like to think that truth will win out for many people in cases where their bias was completely wrong. I missed something in this thread where people are going to great lengths to discredit the “scientific method” (whatever that means) and/or claim they are immune to any form of expectation bias.

If the “scientific method” means designing and building gear in accordance with sound electrical engineering principles and utilizing top notch test gear to ensure you achieved the design goals, I don’t know who could or would want to dispute that. If the “scientific method” is a code word for using double blind testing to validate products, that’s a different turn off of main street.

 

[microstrip]

Science points that stereo is a flawed system, that can not generate per se the spatial envelopment that many people consider that is needed to have more listening pleasure in sound reproduction. As F. Toole points The solution to this is more channels - nothing else. However, using different loudspeaker directivities (to excite more listening room reflections), critical room treatments, tuned electronics and other gimmicks, audiophiles have been able to get more of the “something that was missing” from the stereo reproduction and increase the quality of their listening experiences.

As far as I can see this type of perception, that relies on our past listening experiences, is highly individual and not immediate – it is why the notion of better is also most times highly subjective, and prone to debates in the audiophile community. It is also somewhat marginal, recording dependent, and most of the time, depending on small technical aspects that must of us can not master, making it almost not fully predictable. However it is highly gratifying in terms of emotional response to sound reproduction when properly achieved.

Scientific approaches need repeatable phenomena under very controllable situations and good statistics – some thing that audiophiles can not provide. A few manufacturers, using their knowledge of sound reproduction and models, have established some targets to satisfy our particular requirements, either because we evolutionary adapted to their type of sound, or because they got it right for our desires.

If it was not for the huge amount of existing great stereo recordings and the almost non existence of quality multichannel recordings I can enjoy, I would follow the easy way – build a high quality multichannel system, where reliable science could help me taking the correct decisions. Unhappily in stereo I can not find this support, but existing speakers, tube amplifiers and fancy cables can calm my search for the non existing optimum with great music, without giving up the spatial envelopment. Also, reading again our administrator Steve answer to my question about his experience listening to stereo analogue 1” tape in excellent conditions some years ago, I know I am not alone.

Paul Stubblebine took me into their listening room which had a huge pair of Alon Wolf's Magico speakers ( a prototype which he never made again).He used a large floor standing Studer with 1" heads and took out his "running master" tape of The Arnold Overture (recorded at 15 ips)and played it for me, him and Doc. There were none of Dan's Bottleneck mods on this player but in my 38 years involved in this hobby I have just never heard anything as good as this ...NEVER. It was tantamount to being at the symphony and hearing it live.

All the usual disclaimers concerning Dukas The Sorcerer's Apprentice apply

 

[Phelonious Ponk]

I think I have been reminded repeatedly that blind testing is only required for small differences or differences that seem to deny known science.

It is good for determining whether or not subtle variances are actually audible. But it's also really good for eliminating expectation bias and revealing preferences without the benefit of sighted knowledge.

My other examples were based on negative expectation biases of cheaper SS gear vice more expensive tube gear. I had over $5K invested in my tube preamp and when I inserted the Yamaha C2a preamp to replace it while the tube preamp was off in upgrade land, my expectation bias told me the cheap Yamaha preamp was going to be a dirt road in comparison to my beloved tube preamp. That bias blew up too even though I was emotionally committed to tubes and financially committed to my tube preamp. What I felt was the truth conquered the expectation biases I had.

I think it can go either way - time can eliminate bias and it can reinforce it. I think it's going to reinforce the most when the bias, or opinion, goes untested. You had an "aha moment," I believe. I've had them through blind listening, yours came through unplanned, unexpected system change. Either way is good, if the end result is you end up with a better listening experience and a more objective approach going forward, IMO.

Tim

 

[fas42]

Science points that stereo is a flawed system, that can not generate per se the spatial envelopment that many people consider that is needed to have more listening pleasure in sound reproduction. As F. Toole points The solution to this is more channels - nothing else. However, using different loudspeaker directivities (to excite more listening room reflections), critical room treatments, tuned electronics and other gimmicks, audiophiles have been able to get more of the “something that was missing” from the stereo reproduction and increase the quality of their listening experiences.

Another beautifully apposite post, Micro. Luckily for music lovers the science that claims stereo is "flawed", is itself flawed, because it does not take the capabilities of the ear/brain sufficiently into account, and constantly fails to appreciate how crucial the quality, rather than the configuration, of the playback is to generating the "spatial experience". And I'm not going to throw in some mealy mouthed IMO's to keep people happy: the sort of experience Steve had that you mentioned at the end of the post is something I'm familiar enough with, over many years listening to be certain this is nothing out of the ordinary, just that most people don't go far enough in this hobby to encounter it regularly. Why that situation "worked" is that there were enough positives in the playback setup to offset the remaining negatives, and the illusion succeeded ...

The "something is missing" IS telling you something, not that you need multichannel, but the standard of sound is not yet high enough. The optimum DOES exist but the listener must be prepared, certainly at the moment at least, to go beyond the usual set of procedures advocated by the "experts", and not take short cuts to ameliorate negative elements in the sound on the way: the latter almost certainly will lead to failure in achieving that "optimum" sound experience.

Frank

 

[Ron Party]

The physics of sound has just been casually dismissed. Ph.D.s who have decades of experience in acoustics and psychoacoustics all have got it wrong. Mark, you ask if anyone here has discredited the scientific method. Well as we have now just read, it has been cast out the window along with the physics of sound waves.

 

[Phelonious Ponk]

And I'm not going to throw in some mealy mouthed IMO's to keep people happy

You almost never do, Frank. Why would you suddenly acknowledge the possibility of a reality beyond the one you've created for yourself now?

Tim

 

[fas42]

Every now and again people, even Steve, experience systems working at a very high level, of the order that I mention. So the fact that those individuals experience such must be discarded, consigned to the dustbin of insignificance or irrelevance or some strange anomaly. It can't possibly actually mean something, because that runs counter to the published results of the experts in the field; and as we all know if we study the history of science, that the recognised understanding and opinions of the leading lights of the day always wins out against any Johhny-come-lately.

Seriously though, the "good", or more informative sound can be achieved a number of ways, all being aids for the ear/brain mechanism to get the message of what is in the sound picture of a particular recording. To enumerate:

1) Pull the recording apart using DSP, emphasise different aspects of the spatial clues and pump it through different speakers. This is what Amir is talking about with the new Lexicon device.

2) Record the event in the first place with multiple tracks which encode plenty more "spatial" information. This is Micro's multichannel.

3) Fake the multichannel by doing some half decent fiddling with what's in the 2 stereo tracks, pump various bits out through extra speakers. This is what a lot of cheaper, Dolby Logic like, setups do, a bit of a mish mash of the ideas of 1) and 2), which sort of works to some degree: a little bit of a help

4) Get very expensive, very dynamically capable, well sorted out gear and just hook it up. Really sharp PA systems fall into this category, interesting comment by a TAS reviewer of listening to a CD over just such a system, went home to his Wilson MAXX speaker system with same CD, with quite disappointing outcome

5) Finally, do "heroic" tweaking of a conventional audio system, to get it to give of its best during playback. Not sure if I know any of this category ...

All these ways are not mangling the rules of physics, and acoustics: they are just ways of helping a listener's brain to get the message of what's in the recording, so that a decent illusion of a musical event forms, certainly one beyond what most people experience with a conventionally hooked up stereo system ...

Frank

 

[mep]

The physics of sound has just been casually dismissed. Ph.D.s who have decades of experience in acoustics and psychoacoustics all have got it wrong. Mark, you ask if anyone here has discredited the scientific method. Well as we have now just read, it has been cast out the window along with the physics of sound waves.

If you are talking about Frank, I have him on ignore so I don't see/read what he has to say unless someone quotes him.

 

[FrantzM]

If you are talking about Frank, I have him on ignore so I don't see/read what he has to say unless someone quotes him.

You're not alone

 

[fas42]

Actually, out of curiosity, I wonder if some one can point to the studies which must have been done, which support the thesis that the quality of the playback has no bearing on the subjective impression of the audio experience ...

Frank

 

[treitz3]

If "everything is based on sound" not on any information you've heard about the company, component, design, it's reported compatibility with X, etc, etc. If you bring no knowledge of the product to the first listening and are utterly uninfluenced by appearance, then you are not subject to expectation bias, because you have no expectations. If you have any of the knowledge of any of the above, and are still completely unaffected by expectations, then you, yourself, should be the subject of a scientific study because you are, indeed, an anomaly.

Tim

Dang, rough crowd.

Even if I do have knowledge of a widely regarded superior product's sound and the eye candy is off the charts to me, the reality hits when it hits my ears. Some things may be a perceived better, some things may have degraded. It all depends on what it is I actually hear. The end result is all I care about.

Ron, I am in no way trying to discredit the scientific method in order to attempt to validate my opinion. I completely respect science and I apologize if I have offended or discredited science. I'm only stating that there may be exceptions.

I'll be honest, you folks make me feel like I want to back out of what to me, is an extremely interesting discussion. I may be different to the norm. I may not go with the flow. I know what I hear, I trust my ears and the blind testing has already been done. If I am an anomaly that must be studied, I welcome it. I trust my ears.

 

[fas42]

Dang, rough crowd.

....

Ron, I am in no way trying to discredit the scientific method in order to attempt to validate my opinion. I completely respect science and I apologize if I have offended or discredited science. I'm only stating that there may be exceptions.

I'll be honest, you folks make me feel like I want to back out of what to me, is an extremely interesting discussion. I may be different to the norm. I may not go with the flow. I know what I hear, I trust my ears and the blind testing has already been done. If I am an anomaly that must be studied, I welcome it. I trust my ears.

No need to worry about the rough stuff, treitz3, it's just me that they would like to get in the gutter and give a solid working over to. I've been consistent here in going against the general grain of thinking, and it's got a few people's backs up, I'm sorry to say.

There are a few "anomalies" here in the forum but they generally keep their head down, play it safe most of the time. And don't worry about Tim, his bark's worse than his bite ...

Hopefully you can find some benefit in sticking around, all the best,
Frank

 

[Ron Party]

Actually, out of curiosity, I wonder if some one can point to the studies which must have been done, which support the thesis that the quality of the playback has no bearing on the subjective impression of the audio experience ...

Frank

The ultimate straw man argument of all time! Mark and Frantz, as you can see, your ignore list is well served if it is this kind of post which caused it in the first place.

 

[Ron Party]

Even if I do have knowledge of a widely regarded superior product's sound and the eye candy is off the charts to me, the reality hits when it hits my ears. Some things may be a perceived better, some things may have degraded. It all depends on what it is I actually hear. The end result is all I care about.

Hi Tom. Virtually all of us care about the end result. Attempts to eliminate bias serve to further that goal.

Ron, I am in no way trying to discredit the scientific method in order to attempt to validate my opinion. I completely respect science and I apologize if I have offended or discredited science. I'm only stating that there may be exceptions.

Since we are imperfect beings, and since we all are subject to numerous biases, including, without limitation, expectation bias, the best and perhaps only way to insure that one's opinion about sound or gear is based solely on sound is to form that opinion in a blind test.

 

[fas42]

I wonder if some one can point to the studies which must have been done, which support the thesis that the quality of the playback has no bearing on the subjective impression of the audio experience ...

The ultimate straw man argument of all time! Mark and Frantz, as you can see, your ignore list is well served if it is this kind of post which caused it in the first place.

I am actually reasonably serious about this: so, again, as an example, has there been a study where the clarity or intelligibility of sound, any sort of sound, is seen to be dependent on how much, in this case, deliberate distortion is added to the sound? Not noise, that's the easy one, but quite precisely known levels of non random, non-linear distortion. Anyone?

Frank

 

[fas42]

... as an example, has there been a study where the clarity or intelligibility of sound, any sort of sound, is seen to be dependent on how much, in this case, deliberate distortion is added to the sound? Not noise, that's the easy one, but quite precisely known levels of non random, non-linear distortion. Anyone?

Frank

Always interesting to really chase something down, and the first thing that becomes clear is that nothing has been done to look at this, except in the last few years. Our friend Geddes is mentioned often, and nonlinear distortion, the nasty stuff that I've mentioned many times only looks to be seriously investigated from about 2003 on.

A typical paper is along the lines of "Perception & Thresholds of Nonlinear Distortion using Complex Signals" by Eric Mario de Santis, etc in 2007, which has a nice introduction:

Characterizing the perceptual effects of
nonlinear distortion by means of conventional
metrics such as Total Harmonic Distortion
and Intermodulation Distortion has proven to
be rather ineffectual. Conventional metrics
have also proven unable to characterize the
perception of nonlinear distortion in complex
signals, and thresholds for the perception
of nonlinear distortion have been limited to
simple sinusoidal stimuli

and the conclusion finishes with:

The aim of the project was to obtain thresholds for the audibility of nonlinear distortion in terms of the subjectively correlated metrics. In doing so, a more revealing threshold may be obtained than previously described by research determining threshold using conventional metrics. Since conventional metrics show widely varying subjective rating of stimuli with the same metric values, determining a threshold using such a metric may be misleading. With highly correlated metrics such as the DS and Rnonlin metrics, a perceptually relevant value may be obtained.

A threshold experiment was conducted using the psychoacoustic metrics. The experiment investigated the same four distortion types used in the verification experiment, but with two different music types in order to assess the dependence of stimulus on the obtained thresholds. From the nonlinear threshold experiments, it was concluded that nonlinear distortion thresholds are dependent on the type of applied distortion and on the characteristics of the stimulus

So that's where we were in 2007: not too far along the road on understanding how nonlinear distortion gets in the way of a subjective experiencing of musical reproduction ...

Frank

 

[NorthStar]

When I scanned your post and saw "unwrapping" and "her two big.." I definitely wasn't expecting green eyes!

Exactly my point! ...Regarding 'Fallacy of Expectation Bias'. :b

 

[Phelonious Ponk]

Even if I do have knowledge of a widely regarded superior product's sound and the eye candy is off the charts to me, the reality hits when it hits my ears. Some things may be a perceived better, some things may have degraded. It all depends on what it is I actually hear. The end result is all I care about.

Everyone believes this sort of thing of themselves. I spent my career in advertising. No one is effected by it. We know. We ask them. And no one, especially men, are influenced by ads, especially emotional appeals, in the slightest. Yet the campaigns keep boosting sales. Even the ones targeted at educated males. Especially the ones based on emotional appeals.

Tim

 

[microstrip]

(...) Since we are imperfect beings, and since we all are subject to numerous biases, including, without limitation, expectation bias, the best and perhaps only way to insure that one's opinion about sound or gear is based solely on sound is to form that opinion in a blind test.

Since proper blind tests about most relevant aspects of sound are not within the reach of common audiophiles should we restrain ourselves from having opinions?

I always consider that any opinion about sound carries some bias. The analysis I carry on these opinions and the way I valuate them make my private bias expectation and maybe tailor the way I appreciate sound reproduction characteristics. Perfect. Our opinions are just that - no reasonable person should expect to build science on them.

To what degree our opinions and buying decisions are correctly or wrongly affected by expectation bias is a difficult subject. One can always consider that our bias is created by our long time perceptions and experiences, and reflects our individual preferences, so the choices will probably be more successful. Or consider that the expectation bias will just create false expectations resulting in delusion after a certain period of time. Humans are really complicated ...

 

[Phelonious Ponk]

Since proper blind tests about most relevant aspects of sound are not within the reach of common audiophiles should we restrain ourselves from having opinions?

No, but it should, perhaps, make us a little less sure of them. At least that's my opinion .

Maybe those of us on the "science" side of the argument might recognize that access to measurements comprehensive enough to be meaningful are actually pretty rare, and that the stuff that we expect to be neutral, based on incomplete measurements, brand, reviews, etc, might be just as compromised, or more, than some boutique gear we expect to be deliberately colored.

Maybe those on the "I trust my ears" side might recognize that their ears cannot be divorced from their perceptions, and all they can tell us is what they like. And, in recognizing no objective metric or standard beyond what they hear, it cannot ever be any more than that. They have chosen to make it personal, therefore it cannot be more natural, musical...fill in the pseudo-objective blank. It can only be what they like.

Tim