Why do some Objectivists fear Psychoacousitics?

[Whatmore]

The whole purpose of that test was what was discovered. The story is that when Sean Olive and Toole joined Harman, they suggested using blind testing for evaluating speakers. The speaker designers wanted to have nothing to do with it. They said they absolutely knew, due to their training and skill, which speaker sounded different in sighted evaluation. So Sean tested them and they obvious failed.

Having sat through the test, I can tell you that it is an eye opener. It instantly makes a believer out of you that bias exists in evaluating speakers even though they sound so different from each other. And being associated with harman or not does not impact the results.

Even with something as obviously different as speakers?
And yet there are those that will swear blind (pardon the pun) that these biases can simply be magicked away when it comes to components that have *much* smaller differences.

 

[amirm]

Even with something as obviously different as speakers?

Yes. That is the remarkable part. When I took the test I did not know what was behind the screen until the test was over. My jaw dropped when I saw what was behind the screen. Speakers can be so visual in the way they show off their design (e.g. horns, or B&W tweeters, electrostat), price (quality finish, fancy looking drivers), size, etc. All of that heavily influences your views of what you are hearing. But obviously has nothing to do with the sound.

One of the speakers in the blind testing sounded so bad/broken that I thought it was a defective speaker used as a control. When the curtains went up, it was a speaker that up to that point I thought it was a very good one! It just did not sound natural in blind testing. Objective measurements of that speaker show the serious faults so my blind evaluation was far more correct than sighted.

And yet there are those that will swear blind (pardon the pun) that these biases can simply be magicked away when it comes to components that have *much* smaller differences.

I have this metric I call placebo range. It is significantly large in my experience and hence, I can never be sure anything in that range of difference in sighted hearing can be reliable.

I think the current argument is over the reverse. That is, you can also erase fair bit of quality difference in blind testing if you second guess yourself in an attempt to take a test. I know I can erase differences that are proven objectively to be there and heard by others in blind testing in that manner.

It is a quandary that at the extreme may not have an answer and certainly not to everyone's satisfaction.

 

[amirm]

Just to add another data point, I was recently challenged to pass yet another test, this time MP3 at 320 kbps compared to the original. The source is the Nordic high res sampler from L2:

=====

foo_abx 2.0 beta 4 report
foobar2000 v1.3.5
2015-01-11 18:39:24

File A: 15 Haydn_ String Quartet In D, Op_ 76, No_ 5 - Finale - Presto.flac
SHA1: 702ee12a4f0c39f0d53c8ce521210febc51693b5
File B: 15 Haydn_ String Quartet In D, Op_ 76, No_ 5 - Finale - Presto.mp3
SHA1: d740409c775dfb7e9987c9e80c440ab8e72e1c69

Output:
DS : Primary Sound Driver

18:39:24 : Test started.
18:40:20 : 01/01
18:40:29 : 02/02
18:40:38 : 03/03
18:40:47 : 04/04
18:40:54 : 05/05
18:41:07 : 06/06
18:41:13 : 07/07
18:41:24 : 07/08
18:41:38 : 08/09
18:41:52 : 09/10
18:41:52 : Test finished.

----------
Total: 9/10
Probability that you were guessing: 1.1%

-- signature --
583f03f959b0354ccaafec70aa2861f080fc1f4c

Looking at the results, I am sure you can see that I really could hear the difference given the successive correct guesses. But then at #7, I stumbled once. I lost my concentration a bit, and forgot which way I was voting and got it wrong. One has to agree that the test generated the wrong outcome in that one instance. Had I lost my concentration across two more, my results would have been deemed as invalid and not of high confidence. Yet, we know that I could have passed it.

Subjectively the difference was both significant (channel separation) yet small enough to make the test hard.

 

[treitz3]

<snip> ...Speakers can be so visual in the way they show off their design (e.g. horns, or B&W tweeters, electrostat), price (quality finish, fancy looking drivers), size, etc. All of that heavily influences your views of what you are hearing. <snip>

To some, Amir. To some. Not all.

Tom

 

[amirm]

To some, Amir. To some. Not all.

Tom

I am confident I can show anyone that they are subject to this bias. To what extreme can be debated. But the factors absolutely remain. This is fresh in my mind as I went from room to room at CES. Speakers made an impression before I heard a thing. Tons of effort is being put toward the industrial design of speakers because it absolutely impacts purchase decisions.

 

[treitz3]

I am confident I can show anyone that they are subject to this bias. To what extreme can be debated. But the factors absolutely remain. This is fresh in my mind as I went from room to room at CES. Speakers made an impression before I heard a thing. Tons of effort is being put toward the industrial design of speakers because it absolutely impacts purchase decisions.

Good evening Amir. Key words being, " To what extreme". While I know some that wouldn't now a sonic signature between a MBL and a white van speaker, I do happen to know a few ears I trust that expectation bias would be at the complete opposite of the extreme. Unfortunately, those ears I can count one one hand. Hence my comment. If one knows how to listen, what to listen for and has enough experience? Expectation bias gets thrown out the window. The end result as to what hits my ears is the only thing that matters to me.

Tom

 

[amirm]

Good evening Amir. Key words being, " To what extreme". While I know some that wouldn't now a sonic signature between a MBL and a white van speaker, I do happen to know a few ears I trust that expectation bias would be at the complete opposite of the extreme. Unfortunately, those ears I can count one one hand. Hence my comment. If one knows how to listen, what to listen for and has enough experience? Expectation bias gets thrown out the window. The end result as to what hits my ears is the only thing that matters to me.

Tom

I have 40+ years of experience in listening, plus training, plus design experience and I can't set aside expectation bias. So more power to anyone who can do so otherwise.

 

[esldude]

Just to add another data point, I was recently challenged to pass yet another test, this time MP3 at 320 kbps compared to the original. The source is the Nordic high res sampler from L2:

=====

foo_abx 2.0 beta 4 report
foobar2000 v1.3.5
2015-01-11 18:39:24

File A: 15 Haydn_ String Quartet In D, Op_ 76, No_ 5 - Finale - Presto.flac
SHA1: 702ee12a4f0c39f0d53c8ce521210febc51693b5
File B: 15 Haydn_ String Quartet In D, Op_ 76, No_ 5 - Finale - Presto.mp3
SHA1: d740409c775dfb7e9987c9e80c440ab8e72e1c69

Output:
DS : Primary Sound Driver

18:39:24 : Test started.
18:40:20 : 01/01
18:40:29 : 02/02
18:40:38 : 03/03
18:40:47 : 04/04
18:40:54 : 05/05
18:41:07 : 06/06
18:41:13 : 07/07
18:41:24 : 07/08
18:41:38 : 08/09
18:41:52 : 09/10
18:41:52 : Test finished.

----------
Total: 9/10
Probability that you were guessing: 1.1%

-- signature --
583f03f959b0354ccaafec70aa2861f080fc1f4c

Looking at the results, I am sure you can see that I really could hear the difference given the successive correct guesses. But then at #7, I stumbled once. I lost my concentration a bit, and forgot which way I was voting and got it wrong. One has to agree that the test generated the wrong outcome in that one instance. Had I lost my concentration across two more, my results would have been deemed as invalid and not of high confidence. Yet, we know that I could have passed it.

Subjectively the difference was both significant (channel separation) yet small enough to make the test hard.

Like so many of these online format comparisons there is a level difference of around a quarter db with the MP3 being lower.

 

[esldude]

Good evening Amir. Key words being, " To what extreme". While I know some that wouldn't now a sonic signature between a MBL and a white van speaker, I do happen to know a few ears I trust that expectation bias would be at the complete opposite of the extreme. Unfortunately, those ears I can count one one hand. Hence my comment. If one knows how to listen, what to listen for and has enough experience? Expectation bias gets thrown out the window. The end result as to what hits my ears is the only thing that matters to me.

Tom

Not trying to be nasty. I know you believe what you say. I don't believe what you are supposing. Over, and over, and over and over again that claim is made only to turn out not to be true. Remove sight, and suddenly things are much different. Results would indicate it is an effect upon all humans. So until someone demonstrates differently I would assume otherwise.

 

[Luiz Felipe]

I can try help with some experience.

All time I'm in Hi End Market, I listen this kind of questions and I perfectly understand they will ever exist. It is more complex to understand then you think but I will try to help.

Day after day, working on it, I have the opportunity to listen my showroom system, with a huge different kind of costumers and to listen a lot of systems at their homes listening al kind of opinions. After that I noted three important questions what every one who purpose blind tests and mainly do not believe in some equipments and systems never consider. They are:

- People do not listen the same way ( any tests must to depart from a default parameter, but if people listen differently ways is impossible to have that)
- You must to consider the personal taste on listen
- You must to consider the personal reference listening systems

So, you can make any test with any system blind or not with 40 or with 40k persons. Do not matter, this experience just belong to each individual person. What I listen, and how I like to listen is based on the three factors Above. Every time we share any information on this or another one forum, we are talking about our experience, taste and capabilities on listen.

So, have something what I aways tell to every single person about audio. If you do not listen difference, do not purchase it, be glad to not listen a difference of another persons are saying to listen, because it will be cheeper for you, and do not believe on anyone opinion without listen your self. Just you can chose the better for you. On beginning it is hard, because you haven't experience enough, so listen the how many systems you can, visit shows, and exercise to take your own conclusions. If you listen a product and do not like it, believe in your self.

Best Regards
Luiz

Let me be clear - I'm talking about using Harmon employees in this particular sighted Vs blind test taken from Sean Olive's blog headed "The Dishonesty of Sighted Listening Tests"

View attachment 18517

I wonder why Olive heads his article ""The Dishonesty of Sighted Listening Tests" when in fact, the conditions of the test are skewed?

 

[amirm]

Like so many of these online format comparisons there is a level difference of around a quarter db with the MP3 being lower.

Fascinating. How did you manage to get your hands on the MP3 I created to assess that?

 

[esldude]

Fascinating. How did you manage to get your hands on the MP3 I created to assess that?

Oh well working from memory can fool you. I jumped the gun on that one. 3 weeks ago I was looking at and using the files on the 2L page. They had numerous formats. All except the highest one was lower in level. I thought you got the MP3 from there. So was thinking it would have been lower like all the others. I just checked and see there was no MP3. So in that case, if you used the right conversion, it won't be lower in level.

 

[jkeny]

The whole purpose of that test was what was discovered. The story is that when Sean Olive and Toole joined Harman, they suggested using blind testing for evaluating speakers. The speaker designers wanted to have nothing to do with it. They said they absolutely knew, due to their training and skill, which speaker sounded different in sighted evaluation. So Sean tested them and they obvious failed.

Having sat through the test, I can tell you that it is an eye opener. It instantly makes a believer out of you that bias exists in evaluating speakers even though they sound so different from each other. And being associated with harman or not does not impact the results.

I didn't know you were at that test, Amir?
I still feel that using Harmon employees as test subjects is not generally representative of the difference between sighted & blind testing - IMO it brings another bias to the table - it's a non-representative sample. I would like to see results using test subjects who had no skin in the game - I would imagine differences would be a lot closer.

 

[jkeny]

.... If you listen a product and do not like it, believe in your self.

Best Regards
Luiz

Agreed!

 

[jkeny]

Fascinating. How did you manage to get your hands on the MP3 I created to assess that?

I would imagine he thinks you are gaming the system, yet again, as he has surmised you were doing on ARnyK's high res, jangling keys test?

 

[Whatmore]

I didn't know you were at that test, Amir?
I still feel that using Harmon employees as test subjects is not generally representative of the difference between sighted & blind testing - IMO it brings another bias to the table - it's a non-representative sample. I would like to see results using test subjects who had no skin in the game - I would imagine differences would be a lot closer.

As I see it, two of the three Harmon speakers were preferred in sighted tests - precise the sort of bias you might expect from Harmon employees.
Yet when tested blind, the differences all but disappeared, indicating that the bias had been removed.

So, I'm not sure exactly what you are objecting to.

 

[jkeny]

As I see it, two of the three Harmon speakers were preferred in sighted tests - precise the sort of bias you might expect from Harmon employees.
Yet when tested blind, the differences all but disappeared, indicating that the bias had been removed.

So, I'm not sure exactly what you are objecting to.

I'm suggesting that the difference in preference between sighted & blind (their Harmon bias) is not representative of the norm. - it is a very specific, select group of test subjects & I query what the actual difference in preferences would be between sighted & blind when a "normal test group" were used. One hint we get from the results is in the non-harmon speaker T - negligble difference shown - meaning sighted or blind makes no difference in preference! Is this the norm? I don't know I would like to see further tests.

I raised this in a post & stated not to use this test as an indicator of the difference between sighted & blind for all the reasons I stated - that's my objection. I also object to the title of Olive's blog "The Dishonesty of Sighted Tests" - I'm of the opinion that this is a very skewed headline for what is presented.

As I said if I selected 40 people who had the psychological bias "all DACS sound the same" & I ran a preference test which showed that there was no difference between blind Vs sighted, would you accept my results & my headline "Blind tests Vs sighted show no difference"?

 

[Whatmore]

I'm suggesting that the difference in preference between sighted & blind (their Harmon bias) is not representative of the norm. - it is a very specific, select group of test subjects & I query what the actual difference in preferences would be between sighted & blind when a "normal test group" were used. One hint we get from the results is in the non-harmon speaker T - negligble difference shown - meaning sighted or blind makes no difference in preference! Is this the norm? I don't know I would like to see further tests.

I raised this in a post & stated not to use this test as an indicator of the difference between sighted & blind for all the reasons I stated - that's my objection. I also object to the title of Olive's blog "The Dishonesty of Sighted Tests" - I'm of the opinion that this is a very skewed headline for what is presented.

As I said if I selected 40 people who had the psychological bias "all DACS sound the same" & I ran a preference test which showed that there was no difference between blind Vs sighted, would you accept my results & my headline "Blind tests Vs sighted show no difference"?

Why do you think Harmon employees liked Harmon speakers less when they didn't know if they were listening to them?

 

[FrantzM]

Why do you think Harmon employees liked Harmon speakers less when they didn't know if they were listening to them?

.....

 

[amirm]

I didn't know you were at that test, Amir?

Not that specific test of those speakers. The test i took was with a different set of speakers.

I still feel that using Harmon employees as test subjects is not generally representative of the difference between sighted & blind testing - IMO it brings another bias to the table - it's a non-representative sample. I would like to see results using test subjects who had no skin in the game - I would imagine differences would be a lot closer.

Again, I am not associated with Harman and what I experienced in taking that blind test 1000% sold me on value of blind testing speakers. Like you I kept thinking that the Harman testers would have some kind of preference for their speakers and hence would pollute the results. But once I took the test and understood the nature of their training, all of those doubts vanished.

The key about Harman's work is that they have managed to correlate good performance in their blind testing with objective data. The correlation is very high so there are more reasons to believe their outcomes.