Why do some Objectivists fear Psychoacousitics?

[treitz3]

Good evening Amir. Key words being, " To what extreme". While I know some that wouldn't now a sonic signature between a MBL and a white van speaker, I do happen to know a few ears I trust that expectation bias would be at the complete opposite of the extreme. Unfortunately, those ears I can count one one hand. Hence my comment. If one knows how to listen, what to listen for and has enough experience? Expectation bias gets thrown out the window. The end result as to what hits my ears is the only thing that matters to me.

Tom

Not trying to be nasty. I know you believe what you say. I don't believe what you are supposing. Over, and over, and over and over again that claim is made only to turn out not to be true. Remove sight, and suddenly things are much different. Results would indicate it is an effect upon all humans. So until someone demonstrates differently I would assume otherwise.

Hello, esldude. I didn't pick up any nastiness in your post. I hear what you are saying, it does indicate that my observations may be skewed. I would love to demonstrate this for you as I *personally* don't believe in expectation bias along with a few other ears I know. Here's a post I made that touches on the same subject;

http://www.whatsbestforum.com/showt...io-junkie-when&p=300489&viewfull=1#post300489

Tom

 

[TBG]

Not that specific test of those speakers. The test i took was with a different set of speakers.


Again, I am not associated with Harman and what I experienced in taking that blind test 1000% sold me on value of blind testing speakers. Like you I kept thinking that the Harman testers would have some kind of preference for their speakers and hence would pollute the results. But once I took the test and understood the nature of their training, all of those doubts vanished.

The key about Harman's work is that they have managed to correlate good performance in their blind testing with objective data. The correlation is very high so there are more reasons to believe their outcomes.

What is the level of these correlations? You know the explained variance is the square of the correlation coefficient; don't you?

I might also note that Harman speakers have hardly been the rage among audiophiles.

Personally I don't think people in test conditions are isomorphic with listening to music. As a reviewer, I often suspect that listening as a reviewer is also invalid for readers' judgments about what components to buy.

 

[jkeny]

Why do you think Harmon employees liked Harmon speakers less when they didn't know if they were listening to them?

That's not the point - the point is, what's really the preference differential between sighted Vs blind listening? That test is not a good indicator as the Harmon employees used as test subjects are almost definitely more biased than the man in the street who wouldn't have that psychological baggage in the test.

Can you answer my question & it might throw some light on this topic for you "As I said if I selected 40 people who had the psychological bias "all DACS sound the same" & I ran a preference test which showed that there was no difference between blind Vs sighted, would you accept my results & my headline "Blind tests Vs sighted show no difference"?

Different set of biases in these test subjects if used in the test

 

[jkeny]

Not that specific test of those speakers. The test i took was with a different set of speakers.


Again, I am not associated with Harman and what I experienced in taking that blind test 1000% sold me on value of blind testing speakers. Like you I kept thinking that the Harman testers would have some kind of preference for their speakers and hence would pollute the results. But once I took the test and understood the nature of their training, all of those doubts vanished.

The key about Harman's work is that they have managed to correlate good performance in their blind testing with objective data. The correlation is very high so there are more reasons to believe their outcomes.

I guess what I'm asking Amir, if there is any other test results that shows a similar differential level between sighted & blind preferences? In your tests was the differential more like what is shown on the graph for speakers G, D & S (Harmon speakers) or more like speaker T ( non Harmon speaker) - little difference between sighted & blind listening?

 

[Whatmore]

That's not the point - the point is, what's really the preference differential between sighted Vs blind listening? That test is not a good indicator as the Harmon employees used as test subjects are almost definitely more biased than the man in the street who wouldn't have that psychological baggage in the test.

Can you answer my question & it might throw some light on this topic for you "As I said if I selected 40 people who had the psychological bias "all DACS sound the same" & I ran a preference test which showed that there was no difference between blind Vs sighted, would you accept my results & my headline "Blind tests Vs sighted show no difference"?

Different set of biases in these test subjects if used in the test

It's EXACTLY the point, John.
The Harmon test shows clearly the removal of bias.

Turning to your question, I think you are looking at it the wrong way. i think blind tests are useful for determining if there is a bias caused by knowledge. In your example, you are assuming that due to bias there will be no difference between DACs in the test with knowledge (sighted) so what are you testing for? Doesn't that immediately make the test pointless? So, no I wouldn't accept your test and headlines because I think you would be doing the wrong test in this case.

But I'll say two things here. First, you are using a very unlikely hypothetical scenario to throw out the results of a real experiment that has a perfectly reasonable explanation.
Second, I've answered your question, will you do me the courtesy of answering mine?

 

[jkeny]

It's EXACTLY the point, John.
The Harmon test shows clearly the removal of bias.

Yes but a bias which I suspect is unnaturally high & not the norm. Given normal people without any skin in the game would the differential between sighted & blind be as great as shown here - I suggest not & have given my reasons. You think otherwise but have not given any reason why

Turning to your question, I think you are looking at it the wrong way. i think blind tests are useful for determining if there is a bias caused by knowledge. In your example, you are assuming that due to bias there will be no difference between DACs in the test with knowledge (sighted) so what are you testing for? Doesn't that immediately make the test pointless? So, no I wouldn't accept your test and headlines because I think you would be doing the wrong test in this case.

OK, let's say I don't know that half of the test subjects are biased & think all DACs sound the same. Will my results be different than if I chose people wit no known biases? I suggest the results will be different. I'm suggesting the same applies for this Harmon test - the subjects have a known bias. Do you not get this point?

But I'll say two things here. First, you are using a very unlikely hypothetical scenario to throw out the results of a real experiment that has a perfectly reasonable explanation.
Second, I've answered your question, will you do me the courtesy of answering mine?

Sorry, I thought I gave the answer to this "Why do you think Harmon employees liked Harmon speakers less when they didn't know if they were listening to them?" - it's pretty obvious that their unnaturally high bias was removed but I'm saying that this may not be of much importance to the norm who don't have unnaturally high biases, one way or the other. I gave the opposite example where an unnaturally high bias ("all DACs sound the same") is not removed by blind testing & this skews the results. In the case of the Harmon employees, the sighted test gives unnaturally high preference towards Harmon speakers (except for speaker S but there may be other reasons for the low score of this speaker).

Again, I'll reiterate my thinking for the umpteenth time - speaker T (non Harmon speaker) may well be more indicative of the differential between sighted & blind listening - almost nothing i.e it doesn't change the preference by very much if at all. That is my experience with device other than speakers.

Do you accept this? And if not, why not?

 

[microstrip]

IMHO the shown results in this short article just proved that the results in blind condition are different from those in sighted condition, not that they are of better quality.
The key sentence for understanding the objective of the article is "The vast majority of audio equipment manufacturers and reviewers continue to rely on sighted listening to make important decisions about the products’ sound quality."

If we forget the reviewers (IMHO reviewers test systems in rooms, not individual components) and focus on the majority of the manufacturers we can can ask "How can we explain that some of this vast majority produce such good sounding speakers using such flawed methods? " As far as I know no other manufacturer has test conditions similar to those used by Harman to carry blind tests.

 

[jkeny]

What has blind testing to do with making a loudspeaker, measuring that might be useful !
Keith.

Listening would also be very useful or do you not count that aspect? And if "audio equipment manufacturers rely on sighted listening to make important decisions about the products sound quality", & the "dishonesty of sighted listening" is causing them to be fooled by their obvious biases - do they only produce good sounding products by pure luck?

 

[TBG]

IMHO the shown results in this short article just proved that the results in blind condition are different from those in sighted condition, not that they are of better quality.
The key sentence for understanding the objective of the article is "The vast majority of audio equipment manufacturers and reviewers continue to rely on sighted listening to make important decisions about the products’ sound quality."

If we forget the reviewers (IMHO reviewers test systems in rooms, not individual components) and focus on the majority of the manufacturers we can can ask "How can we explain that some of this vast majority produce such good sounding speakers using such flawed methods? " As far as I know no other manufacturer has test conditions similar to those used by Harman to carry blind tests.

I have known anyone with a Harman speaker. Most of those making speakers are small operations. I hate to review speakers as there are so many adjustments to make and so much breaking in to them. Personally, I have had speakers that have taken six months to get to their best sound.

I think the "flawed methods" are putting customers in a test situation rather than a listening to music situation. I have been in many sighted and blind situations as well as many of the absolutely stupid 30 sec of music to judge whether it is the same component or a different one. Never have I found any correspondence between the component that was best in the testing and one that I liked to listen to music with.

I much prefer to do quick A/B/A comparisons which obviously doesn't include speakers. I very much doubt if many people would subscribe to double blind publications nor would manufacturers advertise in such.

 

[esldude]

What has blind testing to do with making a loudspeaker, measuring that might be useful !
Keith.

Well, given that loudspeakers are the most imperfect part of the chain, one of the aims for Harman was to find out what kind of compromises result in the most preferred sound. So yes, you use measurements to make your speaker. Harman thinks if they know what kind of speaker measurements will be found most pleasing to the listener, they can use those as a target to make the speaker most will prefer for lowest possible price. Until you have that target, is it better to make the crossover great, make the speaker follow a certain on axis target, or off axis or some combination? They say they have enough data to predict from a particular suite of measurements which speaker will be most preferred among a group with an .86 correlation coefficient to what actually gets chosen in their tests.

 

[TBG]

Well that is explaining 74% of the variance. That really seems quite high, but if this is true it is worthwhile. But it doesn't seem to have gotten Harmon very far.

 

[esldude]

Well that is explaining 74% of the variance. That really seems quite high, but if this is true it is worthwhile. But it doesn't seem to have gotten Harmon very far.

I don't know. Not far in the high end world anyway. In the pro world they get more credit.

I can see why it doesn't get them far. Amir's description of his experience shows an example. Apparently one of the speakers he thought pretty respectable sounded quite a bit less good than others when he knew what he had been listening too after the test. It is what the bias thing is all about. A big, fancy, pretty, speaker with good marketing for why it is special can influenc you more than the the way Harman promotes some of the more pedsterian products. It gets street cred before you ever hear it.

 

[Robh3606]

But it doesn't seem to have gotten Harmon very far.

How have all the reviews on the Revel Performa 3 series been?? Reviews on the Array 1400 on Everest or the K2's??

Rob

 

[amirm]

What is the level of these correlations? You know the explained variance is the square of the correlation coefficient; don't you?

You ask me a question and before I answer it, you are warning me I don't know something? If you are interested in the answer, I will provide the data including the reference and quotes from AES paper. If you don't want to hear it, I won't bother.

I might also note that Harman speakers have hardly been the rage among audiophiles.

That may be more of a problem with audiophiles than Harman speakers.

Here is John Atkinson answering what the best speakers were that he ever heard in his room. Jump to 50:00 in this video:

Here is my transcription of it:
The best speakers I ever used in my room was the Revel Salon 2. They really produced the most neutral sound the best combination of bass definition and best bass control and best imaging. I wept when I had to send them back. I could not afford to buy them. They were really good. I mean speakers tend to rotate in and out of my room….I do miss those Revels.

And: http://www.soundstage.com/equipment/revel_ultima_salon2.htm

"Conclusion

No loudspeaker is perfect, but Revel’s Ultima Salon2 is the closest I’ve found. Its bass and high-frequency performance are beyond reproach, and its midrange deserves high praise for its neutrality, transparency, and speed. Some might wish for a little more of the midrange richness of, say, the Magico V2, but this would be more about desiring a certain type of sound rather than indicating any deficiency in the Salon2. From top to bottom, The Salon2 is an ultraprecise, remarkably refined, full-range transducer that delivers nothing short of state-of-the-art sound while making fewer compromises than any other speaker I’ve heard.

I foresee two main problems for the Salon2. First, it will be too big for some rooms -- though its footprint is modest, the speaker’s height makes it rather imposing, and its extreme-low-output bass could overload small rooms. Second is the price -- $22,000 can buy you a pretty good car, so it’s not exactly pocket change, particularly in these trying economic times. On the other hand, there are speakers costing much more that don’t offer the Salon2’s performance, technology, or build quality. It may not be cheap, but it’s worth the asking price.

If you have the space, the money, and the desire to own a "statement" loudspeaker that can take its place among the greats, Revel’s Ultima Salon2 is the one to look at. I’m sure someday I’ll review an even better speaker -- progress is defined by the continuing search for perfection -- but for now, at least, the Ultima Salon2 is the best passive loudspeaker I’ve ever reviewed."

There are reasons why Revel speakers are not all the rage with audiophiles but sonic performance is not it.

 

[aronjt]

While channel surfing a while back I was surprised to see a pair of Salon2 on the show 30 Rock. Wonder if they're even hooked up? Probably not.

Found a YouTube clip:

https://www.youtube.com/watch?v=O71664NdKmM

 

[Whatmore]

Y
<snip>
it's pretty obvious that their unnaturally high bias was removed

<snip>

John, if the closest we ever get to agreement is that blind tests can remove some biases then I'm a happy man
If you ever make it to Melbourne, the single malt is on me

 

[jkeny]

John, if the closest we ever get to agreement is that blind tests can remove some biases then I'm a happy man
If you ever make it to Melbourne, the single malt is on me

Thanks & I'd be keen to sit down with you for a pint of Guinness & chat if you ever get to Ireland

Just to correct what you seem to be taking from my piece you quoted - I don't believe knowledge (or sightedness) is necessarily that much of a bias in listening & that blind listening introduces far more factors that bias the results towards a null so my statement you quoted does not mean that blind testing is necessarily better

 

[microstrip]

You ask me a question and before I answer it, you are warning me I don't know something? If you are interested in the answer, I will provide the data including the reference and quotes from AES paper. If you don't want to hear it, I won't bother.


That may be more of a problem with audiophiles than Harman speakers.

Here is John Atkinson answering what the best speakers were that he ever heard in his room. Jump to 50:00 in this video:

Here is my transcription of it:
The best speakers I ever used in my room was the Revel Salon 2. They really produced the most neutral sound the best combination of bass definition and best bass control and best imaging. I wept when I had to send them back. I could not afford to buy them. They were really good. I mean speakers tend to rotate in and out of my room….I do miss those Revels.

And: http://www.soundstage.com/equipment/revel_ultima_salon2.htm

"Conclusion

No loudspeaker is perfect, but Revel’s Ultima Salon2 is the closest I’ve found. Its bass and high-frequency performance are beyond reproach, and its midrange deserves high praise for its neutrality, transparency, and speed. Some might wish for a little more of the midrange richness of, say, the Magico V2, but this would be more about desiring a certain type of sound rather than indicating any deficiency in the Salon2. From top to bottom, The Salon2 is an ultraprecise, remarkably refined, full-range transducer that delivers nothing short of state-of-the-art sound while making fewer compromises than any other speaker I’ve heard.

I foresee two main problems for the Salon2. First, it will be too big for some rooms -- though its footprint is modest, the speaker’s height makes it rather imposing, and its extreme-low-output bass could overload small rooms. Second is the price -- $22,000 can buy you a pretty good car, so it’s not exactly pocket change, particularly in these trying economic times. On the other hand, there are speakers costing much more that don’t offer the Salon2’s performance, technology, or build quality. It may not be cheap, but it’s worth the asking price.

If you have the space, the money, and the desire to own a "statement" loudspeaker that can take its place among the greats, Revel’s Ultima Salon2 is the one to look at. I’m sure someday I’ll review an even better speaker -- progress is defined by the continuing search for perfection -- but for now, at least, the Ultima Salon2 is the best passive loudspeaker I’ve ever reviewed."

There are reasons why Revel speakers are not all the rage with audiophiles but sonic performance is not it.

Amir,

You have a point, if Harman target are not the audiophile preferences, no hope they will please them. And the ridiculous story of JA weeping is not enough to move me. BTW, I think the same about his poetic story of retiring with the Alexia's. It is funny that the enemy (reviewers that carry sighted reviews) become good friends when they say something that supports our beloved ...

 

[esldude]

Amir,

You have a point, if Harman target are not the audiophile preferences, no hope they will please them. And the ridiculous story of JA weeping is not enough to move me. BTW, I think the same about his poetic story of retiring with the Alexia's. It is funny that the enemy (reviewers that carry sighted reviews) become good friends when they say something that supports our beloved ...

Harman's tests indicate a preference very widely and by a large margin agreed upon by listeners comparisons. Further the target their data pointed to isn't anything strange. A very even balanced response. One must pay particular attention to even small resonances as they are audible and detract from a speaker. A wide controlled, even directivity off axis. Is any of that non-audiophile?

They have tested employees, the public, pro audio people, and even groups from other countries with langauges other than English in case that altered a general preference. So far none of that seems to matter.

So if audiphile preferences are different what would they be? More coloration, more zing, more zip, what? Their proposed target isn't any oddball idea. I guess the audiophile oxen being gored is the idea some highly respected, revered high end designs have some measured performance issues, and when auditioned blind they score poorly. Other high end speakers that score well, don't do any better than less expensive speakers that more or less match their performance. Again, really not a strange idea.

 

[jkeny]

Leaving this Harmon diversion aside & getting back to the main point of the thread - one of the issues with A/B testing is that it ignores the important psychoacoustic aspect of audio scene analysis (ASA). ASA is how we all perceive what we hear & continually parse & organise it into audio objects & streams. Audio streams are not formed instantly, they require a certain number of seconds (don't know this number exactly). Therefore, they are not suitable to being differentiated by quick A/B changes (the favourite listening test for objectivists).

The funny thing is that auditory perception & ASA are the most important & crucial aspects of what we hear & how realistic the illusion of audio playback is perceived.