It’s All a Preference

[microstrip]

Let me first quote you the section from one of Sean Olive's AES Paper: Differences in Performance and Preference of Trained versus Untrained Listeners in Loudspeaker Tests: A Case Study

"In each listening test, listeners were required to rate
each loudspeaker on the interval preference scale defined
in the listener instructions (see Appendix 2). (...)

Amir ,
If you do not give us access to Appendix 2, the musical selections you used and the system that was used, all you refer is just "musical" to my years. And no one will be able to comment it adequately.

 

[amirm]

Amir ,
If you do not give us access to Appendix 2, the musical selections you used and the system that was used, all you refer is just "musical" to my years. And no one will be able to comment it adequately.

It is a table so a bit of a pain to cut and paste but here it is:

Artist, Track, and Album
James Taylor, “That’s Why I’m Here” from “That’s Why I’m Here,” Sony Records.
Little Feat, “Hangin’ on to the Good Times” from “Let It Roll,” Warner Brothers.
Tracy Chapman, “Fast Car” from “Tracy Chapman,” Elektra/Asylum Records.
Jennifer Warnes, “Bird on a Wire” from “Famous Blue Rain Coat,” Attic Records.

"The program signals were reproduced from the hard
disk on the control computer equipped with a digital sound
card (SEK’D ProDif 96). The AES-EBU signal was fed to
a digital switcher–distributor (Spirit 328 digital mixer) and
converted to four analog signals using an eight-channel
Studer D19 digital-to-analog converter. Precise level
matching between loudspeaker was done by adjusting the
trim controls on each analog output. Each loudspeaker was
amplified with a Proceed AMP3 amplifier."

 

[microstrip]

It is a table so a bit of a pain to cut and paste but here it is: (...) [/i]

Amir,
Thanks but your help - but I was addressing mostly the listener instructions in Appendix 2. Do you have them?

 

[mep]

Also, If read that paper correctly, they only used one speaker for the demo so you were hearing some form of mono?

 

[amirm]

Amir,
Thanks but your help - but I was addressing mostly the listener instructions in Appendix 2. Do you have them?

That is even more challenging to cut and paste with graphs and such . Here is what it says:

"In these tests you will be judging the sound quality of
different loudspeakers and rating them according to your
personal preference. You MUST enter a rating for each
loudspeaker in the appropriate box after the program
selection has ended. Please enter your ratings using the
following prefence scale: (10 = Really like, 0/1 = really dislike with others in between).
Your ratings can contain up to one decimal place (e.g., 7.3. 2.5).

DO NOT GIVE TIED SCORES IN ANY ROUND.
If you do, the computer will ask you to reenter your ratings.
You should seperate your preference ratings among different
speakers to reflect your relative preference between
two speakers. Use the following guidelines: [graph showing how different scores need to be for Slight Preference, Moderate Preference, and Strong Preference ranging from 0.5 to 2).

Finally, we encourage you to write comments about
what you like and dislike about the sound of the speakers
you are comparing: what aspects is it about the speaker
that makes you prefer it (or not prefer it) over the other
speaker(s)?"

 

[amirm]

Also, If read that paper correctly, they only used one speaker for the demo so you were hearing some form of mono?

Yes, it is a proper summed mono signal. In another research paper, they vary the number of speakers from one to many. This was a test of EQ systems. When testing in mono, the EQ system performed much better than no-EQ. When they increased the number of speakers, the score for the EQ system did not change much but the score for no-EQ went up! Put another way, we seem to be less discriminating of room/speaker flaws as the number of channels goes up.

 

[microstrip]

That is even more challenging to cut and paste with graphs and such . Here is what it says: (...)

Amir,

I just entered part of the text you entered in google and found the full 20 pages paper in pdf version - I will go through it and will come back after reading it.

I was however disappointed reading that: All tests were performed double blind using monophonic (single-loudspeaker) comparisons.

Does this mean that the four speakers were tested in mono? And all we are debating is a preference for a single speaker in mono?

 

[treitz3]

Lord, I hope not.

 

[amirm]

Amir,

I just entered part of the text you entered in google and found the full 20 pages paper in pdf version - I will go through it and will come back after reading it.

I was however disappointed reading that: All tests were performed double blind using monophonic (single-loudspeaker) comparisons.

Does this mean that the four speakers were tested in mono? And all we are debating is a preference for a single speaker in mono?

I just addressed that above. While I can think of some specialized situations where stereo testing is needed, a speaker is a single device with no dependency on another unit to do its job. To the extent a speaker doesn't sound right, adding another to it doesn't make it do its job better all of a sudden.

 

[MylesBAstor]

I just addressed that above. While I can think of some specialized situations where stereo testing is needed, a speaker is a single device with no dependency on another unit to do its job. To the extent a speaker doesn't sound right, adding another to it doesn't make it do its job better all of a sudden.

What about imaging and spatial resolution among other qualities?

 

[amirm]

What about imaging and spatial resolution among other qualities?

The former is the exception that I made. We must keep in mind however that frequency response errors are far easier for us to detect than those artifacts. And it needs to be proven that a speaker that does poorly in mono, it becomes a hero in stereo.

 

[microstrip]

The former is the exception that I made. We must keep in mind however that frequency response errors are far easier for us to detect than those artifacts. And it needs to be proven that a speaker that does poorly in mono, it becomes a hero in stereo.

Amir,

I disagree with your humorous (I hope!) statement. The responsibility of proving that mono preferences can be extended to stereo preferences belongs to the authors of the study (if they want to prove it), or anyone using it to claim preference in normal conditions.

I know marketing has very free rules, but from what I have read before in the net I was always persuaded these preference tests reflected a typical application - that must be stereo. My fault - I should have read the small letter lines. Thanks to you and this debate I will not loose much more of my time with it.

Perhaps some people find that stereo imaging and spaciousness are too strongly weighted by audiophiles. But if this is part of the definition of an audiophile, manufacturers should respect it.

BTW, my words are not a critic to the excellent work of Harman scientists (the long paper is really interesting), but to its use in marketing and to denigrate rival companies based in mono tests.

BTW2, I start to have a feeling of the existence of some small bitterness from some Harman scientists towards the limitations of stereo, that can not deliver us all the best they wanted us to have in sound reproduction, and their endorsement to the much superior multichannel.

BTW3 reading about these tests conditions I found that they also challenge the myth of the 90% room - 10% equipment. More on the later in another thread.

 

[rbbert]

It's hard to believe that a poor mono performer would become a good stereo performer. It does seem believable that a good mono performer might not do so well in a stereo pair.

 

[microstrip]

It's hard to believe that a poor mono performer would become a good stereo performer. It does seem believable that a good mono performer might not do so well in a stereo pair.

Rbbert,
It is hard to believe that stereo can do most of what some of us claim it does. So much than a few of our members do not believe it and claim it is all imagination.

 

[amirm]

Amir,

I disagree with your humorous (I hope!) statement. The responsibility of proving that mono preferences can be extended to stereo preferences belongs to the authors of the study (if they want to prove it), or anyone using it to claim preference in normal conditions.

And they have proven it with research. Here is an example from AES paper, Comparison of Loudspeaker-Room Equalization Preference for Multichannel, Stereo, and Mono Reproductions: Are Listeners More Discriminating in Mono?

"In this experiment, a panel of trained listeners gave comparative preference ratings for three different loudspeaker equalizations based on anechoic and in situ measurements evaluated in a semi-reflective room, using three multichannel music recordings reproduced in surround, stereo, and mono. These equalizations were compared to the unequalized loudspeaker. The results are summarized as follows: all three equalizations were equally preferred over the unequalized system. The differences in preference ratings increased as the number of playback channels was reduced from 5 channels (surround) to 1 (mono). "

From Dr. Tool's book of acoustics:

"In monophonic tests, listeners reported large differences in both sound quality and
spatial quality, and, if anything, there were stronger differentiations in the
spatial quality ratings. This was defi nitely not anticipated, but these listeners
had little doubt that there were substantial differences in both rating categories.
However, in stereo listening, most of the differences disappeared. The two highly
rated loudspeakers (AA and E) kept their high ratings, almost identically in fact,
but the loudspeaker (BB) that had a low rating in mono became competitive in
stereo. In fact, looking at the stereophonic data, the scatter diagrams of judgments
indicate a lot of indecision about the relative merits of these loudspeakers
in both categories: sound quality and spatial quality."

In video world, we routinely turn off the color and look at the black and white image for distortion. With color, the image becomes "pretty" and we become far less critical of artifacts that are still in there.

I know marketing has very free rules, but from what I have read before in the net I was always persuaded these preference tests reflected a typical application - that must be stereo. My fault - I should have read the small letter lines. Thanks to you and this debate I will not loose much more of my time with it.

That's fine.

Perhaps some people find that stereo imaging and spaciousness are too strongly weighted by audiophiles. But if this is part of the definition of an audiophile, manufacturers should respect it.

Manufacturers do I am sure. The issue is that it needs to be proven that there is an advantage to a speaker doing poorly in mono, when it comes to stereo. The fact that we become less critical in stereo helps mask poor performance but doesn't advantage it over a speaker that does better in mono.

BTW, my words are not a critic to the excellent work of Harman scientists (the long paper is really interesting), but to its use in marketing and to denigrate rival companies based in mono tests.

They present data. Other companies are free to present theirs. Where would I read about their published listening tests? They seem to be missing, no? Surely they can repeat Harman's tests but in stereo and come and show that the outcome was the other way around. Such data does not exist.

When someone publishes a paper in journals of AES and ASA, there is a much higher bar than "marketing" material. Dr. Tool has had some 30-40 papers published in combination in both with this type of theme where our assumptions are put to test using listening tests. He has done this now for something like 30 years. You would think it were all a marketing thing by now someone would have called him on it. Instead, multi-billion dollar companies have transformed how they do business using that research.

BTW2, I start to have a feeling of the existence of some small bitterness from some Harman scientists towards the limitations of stereo, that can not deliver us all the best they wanted us to have in sound reproduction, and their endorsement to the much superior multichannel.

It is not bitterness but proper search that says in a typical home listening room, the dimensions are too small as to create a sense of envelopment. There is extensive research that they have performed on this. Therefore they conclude that the only way to get there is to have it be in the recording and presented in multi-channel. Dr. Toole also goes into extreme detail of how stereo is defective due to interference between the two speakers in creating phantom center imaging for example.

Really, you can disagree with their work but these guys are anything but biased for the sake of being biased. They work for a company that produces $70,000 speakers yet if you listen to Dr. Toole, he will try to convince you that a $300 speaker and a sub may do the job as well! Our research arm at Microsoft was the same way, often presenting papers which was damaging to our commercial goals. But you have to let researchers do that or they will not work for you and instead, stay at universities. The hope is that you integrate enough of their work to make it worth the negative damage you may get this way.

BTW3 reading about these tests conditions I found that they also challenge the myth of the 90% room - 10% equipment. More on the later in another thread.

Very much so. I have spent days arguing these myths on AVS about treating rooms. If you really want to read Dr. Toole, that is the topic you want to read about. He has the one and only integrated story of how to approach your room+speaker. What I wrote on low frequency optimization is based on that. Next write up will cover this topic.

 

[microstrip]

And they have proven it with research. Here is an example from AES paper, Comparison of Loudspeaker-Room Equalization Preference for Multichannel, Stereo, and Mono Reproductions: Are Listeners More Discriminating in Mono?
(…)

Again, an argument that increases my conviction that these works were carried mainly to be used as research and development tools, not evidence of quality in stereo. The two variables – spectral accuracy and stereo imaging properties are not independent, and just considering that suppressing one of them makes a better judge does not prove anything about the second one.

In video world, we routinely turn off the color and look at the black and white image for distortion. With color, the image becomes "pretty" and we become far less critical of artifacts that are still in there.

Bad for research, but good for end users – the less the users see the artifacts in the final product the better. Remember the thread we had on whether in a better system we are more or less aware of the recording limitations and defects?

Manufacturers do I am sure. The issue is that it needs to be proven that there is an advantage to a speaker doing poorly in mono, when it comes to stereo. The fact that we become less critical in stereo helps mask poor performance but doesn't advantage it over a speaker that does better in mono.

You are reversing the facts. From my perspective the issue is that these works do not prove that the characteristics outlined in FR are the main ones for excellent stereo imaging properties and how they influence it.

They present data. Other companies are free to present theirs. Where would I read about their published listening tests? They seem to be missing, no? Surely they can repeat Harman's tests but in stereo and come and show that the outcome was the other way around. Such data does not exist.

The companies choose to present or not to present their data and scientific models. Most of them are in the business to supply excellent speakers, not to do scientific research. This happens in many areas of instrumentation.

When someone publishes a paper in journals of AES and ASA, there is a much higher bar than "marketing" material. Dr. Tool has had some 30-40 papers published in combination in both with this type of theme where our assumptions are put to test using listening tests. He has done this now for something like 30 years. You would think it were all a marketing thing by now someone would have called him on it. Instead, multi-billion dollar companies have transformed how they do business using that research.

I stress it again. I am not questioning the papers and science of F. Toole and many others. But I question the use and interpretation of others of their excellent work. And there are marketing issues involved when you choose a Martin Logan to represent the dipoles family.

It is not bitterness but proper search that says in a typical home listening room, the dimensions are too small as to create a sense of envelopment. There is extensive research that they have performed on this. Therefore they conclude that the only way to get there is to have it be in the recording and presented in multi-channel. Dr. Toole also goes into extreme detail of how stereo is defective due to interference between the two speakers in creating phantom center imaging for example.
Really, you can disagree with their work but these guys are anything but biased for the sake of being biased. They work for a company that produces $70,000 speakers yet if you listen to Dr. Toole, he will try to convince you that a $300 speaker and a sub may do the job as well! Our research arm at Microsoft was the same way, often presenting papers which was damaging to our commercial goals. But you have to let researchers do that or they will not work for you and instead, stay at universities. The hope is that you integrate enough of their work to make it worth the negative damage you may get this way.

Again, I do not disagree with their work. The only think I do not see in their papers is evidence that the preference ranking they established in the works that were referred in this thread is an universal ranking or that it can be used by typical audiophiles and that loudspeakers that fail in the FR characteristics they outline are inferior. Also I feel they (or sometimes others for them ) are openly misrepresenting the competition.

Very much so. I have spent days arguing these myths on AVS about treating rooms. If you really want to read Dr. Toole, that is the topic you want to read about. He has the one and only integrated story of how to approach your room+speaker. What I wrote on low frequency optimization is based on that. Next write up will cover this topic.

I have to say it was one of the parts I read from the “Sound Reproduction” – just after the masterly written three fist chapters. Slowly I want to read it all!

 

[Robh3606]

I stress it again. I am not questioning the papers and science of F. Toole and many others. But I question the use and interpretation of others of their excellent work. And there are marketing issues involved when you choose a Martin Logan to represent the dipoles family.

You have me curious. What would have been a better speaker to use?? Is there anything radically different from other dipoles??

Again, an argument that increases my conviction that these works were carried mainly to be used as research and development tools, not evidence of quality in stereo. The two variables – spectral accuracy and stereo imaging properties are not independent, and just considering that suppressing one of them makes a better judge does not prove anything about the second one.

To bad you are not into horns. If you take a clasic horn with an exponential flair it has DI charaterisitics similar to a dynamic driver. It beams as you go up in frequency. If you compare that to a CD/Constant Directivity type horn it is very close to what Toole outlines as the ideal DI and Power Response. When you compare the older systemx to the newer CD systems the newer systems image much better. There is a corellation between the smooth more consistant FR off axis and better imaging. With horns anyway JMHO


Rob

 

[Gregadd]

You have me curious. What would have been a better speaker to use?? Is there anything radically different from other dipoles??

I would have chosen a different front end. IME digital combined with ss is not the best match for ML. But that's another debate

 

[KeithR]

this all reads like one big Harman commercial. sorry guys. Revels require big SS for the most part, so I've never been into their sound.

A dealer i used to know sold both Revel and Wilson and the Wilson was selected overwhelmingly by his customers- but that was the W/P 7 and Salon 1 iterations. Could easily be reversed now.

I do miss the Salon 1 look with the blue/grey panels

 

[mep]

this all reads like one big Harman commercial. sorry guys.

See my post under the "Do we hear differently" thread:

http://www.whatsbestforum.com/showthread.php?6482-Do-we-hear-differently&p=107319#post107319