Objectivists, Harman Testing, Reviewers, and Reality

[esldude]

Then there is the subsequent frequency response of the Logan posted again a number of times, and it's up and down all over the panel's operating range. That DOES NOT RAISE any eyebrows at Harman or anywhere else? Are we really serious? And is that part of the "research" to be taken seriously? I put "research" in quotes because to me it's just investigation, and a poor one at that, in a poor "room". No wonder it's been suggested we won't be able to tell our favorite speaker sitting in a corner opposite another, with the listener in the middle - who evaluates speakers like that, and what value does it add anyway. But then we all agree that the ROOM is of utmost importance Need some ice cream.

I don't know why this raises eyebrows. Measurements by Stereophile of various panel speakers show similar results in anechoic response especially in the off axis results. Some manage pretty good results on axis, and smoothed, but off axis and in the waterfall plots panels tend to be very 'unruly' in their response. It is a good question why it can look so bad and sound good. On the other hand, maybe when you don't see the panel it really doesn't sound that good vs the competition. Or perhaps that delayed ringing in the waterfall plots is some sort of euphonic coloration preferred over accurate response.

 

[rockitman]

The factors that make great pleasing sound cannot be measured. Measurement waxing poetic is a waste of effort unless you are trying to sell gear pontificating that the great measurements translates to the best sound. A great marketing ploy by Harman and I am sure many others.

 

[ack]

I,already asked the same question! Must be an echo in here!

Sorry, I stopped reading that other thread; just saw your latest post. Ice cream?

 

[ack]

I don't know why this raises eyebrows. Measurements by Stereophile of various panel speakers show similar results in anechoic response especially in the off axis results.

It should raise eyebrows because no panel set up correctly sound like that graph - and I have owned both MLs and Magnepans, dating back to the 80s.

 

[Fitzcaraldo215]

If you want to read a magazine that puts measurements first, buy Stereo Review. Ooopsss. It's gone. Guess everything sounded the same so who needed the magazine? Certainly not for the music reviews.

Otherwise it's simple. The other magazines have subjective reviews and it's up to you read or not. We are all adults. It's like watching TV. You don't like the show, change the channel.

Hey, Myles, I subscribed to quite a number of high end startup, subjective-only, thank you, publications in the 70's and '80's. They all disappeared, too. One of them was yours, as I recall. Was it Ultimate Audio? I do not remember. But, whatever it was, you never completely fulfilled my subscription. You probably owe me a few bucks, you, SOB.

I am indeed impressed with how staunchly the reviewers and their fans defend the totally loosey-goosey word of subjective reviews and reviewers against any and all alternatives and possible improvements to the dialogue in trying to reach a better understanding of the "truth". Subjective reviews and reviewers are, frankly, just all over the place with no consistency whatsoever in the quality or accuracy of the information they convey.

It is not one, easily characterized, monolithic "thing" at all. There are some truly bad actors out there, along with the good. But, like all human endeavors, they have a wide range of usefulness, credibility, technical sophistication, honesty, experience, etc., etc. There are precious few, very few, who I like and trust, not that I would ever buy just on their recommendation, either. But, I pay attention closely to the "good guys" while completely ignoring the "bad guys", though some of them are very skilled and engaging writers of total, egocentric BS, IMHO. Oh, yeah, there are a lot of famous names, past and present, on my "bad guy" list, more bad than good by far.

My tiny list of most trustworthy reviewers are willing to give subjective opinions. But, at the same time, they understand and appreciate the value of audio science and greater objectivity in trying to reach a more unbiased presentation of what they see as the essential truth. Any reviewer who thinks needs look no further than his own ears and prior listening experience, whatever that might actually be, Lord knows, is not on my list.

We might have specs on equipment, but we have nothing like that on reviewers, their rooms, their listening experience, etc., etc. Oh, wait, the great HP was going to unravel that for us for awhile by publishing reviewers' astrological signs, which he did as I cancelled my TAS subscription while barfing into the toilet. That was one of the most abject low points in the history of subjective reviewing and audiophilia, IMHO. But, subjective reviewers are the self-appointed heroes of their own world, the gurus, the "experts", the wheat, not the chaff, the non-wankers. Oops, sorry, wrong thread.

Sorry to appear so totally cynical. I know a number of reviewers personally, and I have visited their rooms and heard the equipment being reviewed. The good ones are totally honest, fair minded and desirous of searching for and trying to convey the truth of how a component really sounds to their fellow audiophiles. They do not get paid a lot, and it is not at all an ego thing for them at all. It is about the challenge of learning and sharing the knowledge. They have sufficient humility to know what they do not know and they welcome any additional useful information, even if that information might, God help us, come from the objectivist camp. They are open minded and they see the many pitfalls their colleagues have fallen into, and they admit it, honestly. Yet they persist and they do a good job and try to do an even better job. So, my belief in a better side of human nature is happily confirmed by these guys. But, then there are the others....

So, how do we tell one kind of reviewer from the other?

I suggest we send all the reviewers to Harman's spinarama room, put them on the pedestal and subject them to some rigorous testing and scoring of their "fidelity".

This us all totally subjective, of course, so YMMV.

 

[esldude]

Peter, I explained all of this repeatedly in the other thread. How can you turn around and ask this kind of question, seemingly ignoring everything I wrote???? This is the picture I posted for where the measurements are performed:



It is an anechoic chamber. How on earth do you measure a loudspeaker for radiation at one specific angle in a real room where reflections are bouncing all over the place?

Note that magazine reviewers are not measuring this. They don't have access to anechoic chamber so they approximate by keeping the microphone close to the loudspeaker. That technique has limitations such as lack of integration of drivers at too close or a distance, or inclusion of too many reflections at farther back. So while informative, and decent approximation, they are not the same thing.


We don't perform listening tests in anechoic chambers. We perform them in well, a room. Harman has multiple such rooms. I have shown you a picture of one with a speaker shuffler that I sat in. There are others at Harman. Here is one where the loudspeaker is rotated in place. That room is built to an ITU reference standard for acoustics testing and one is supposed to be a model of "domestic listening room." Here is a picture of it I took while Dr. Olive was running the speaker training test on us:



The turntable for the loudspeakers unfortunately is behind the video projection screen. But essentially three loudspeakers are mounted to it and rotated. This is used for in-wall and car audio speakers. This room was also used for Room EQ blind testing. All the results in this room mirror the one used in the other room. There is a whole conference paper on design of this room.

Here is one where they can shuffle stereo speakers and in a much larger room:


And the other end of it:



As with everything else Harman does, there is extensive research paper and testing published for this room. Here is the end of that paper:

"4.0 CONCLUSIONS
In summary, we have described a new facility designed to test multichannel components
efficiently and as bias-free as possible. The facility includes acoustically transparent
listening screens that hide the identities of all multichannel loudspeakers and equipment
within the audio path. Particular attention has been taken to address the two of the most
problematic variables in listening tests: the listening room and the position(s) of the
loudspeaker. Through the use of a computer automated speaker shuffler, we have greatly
reduced the amount of time and effort required to set up and test multiple comparisons
between loudspeakers by reducing the factor position to a one-dimension or level
variable. Typical loudspeaker evaluations should be reduced in length by a factor of 24:1.

The listening room itself is capable of testing up to three different 5.1 or 7.1 channel
systems and accommodate 1-6 listeners at a time. The measurements we have shown in
this paper indicate its performance in its current form meets the very highest standards set
out by the ITU and EBU recommendations, in terms of volume, geometry, reverberation
time, and the control of early reflections. The acoustics of the room can be easily altered
from hemi-anechoic to more typical domestic room conditions by adding reflective
panels to the room’s boundaries.

Finally, the experimental design, set up and control are computer-automated so that
experiments can be easily repeated, and are less prone to human error. The more time consuming
and mundane tasks such as collection and analysis of data have also been
computer-automated, so that experiment report writing becomes a simple cut-and-paste
operation."

Really guys. You have to resist the urge to keep thinking this is a bunch of corrupt agenda meant to confuse people. For that, you need to look at the marketing material you are reading about your favorite loudspeaker. These are true researchers who happen to work at Harman with very deep pockets, able to conduct such research at grand scale. They are not sitting there fooling themselves or other people because you all happen to think of things they have not.


You honestly think they put 70 loudspeakers in one room and tested them with 250 people??? No, they did not. These are composites of many independent studies. Yes there are a few loudspeakers in the room. The only effect I have heard of here is the woofer acting as a tuned absorber but that happens when it is unloaded.


What were the comments above then?


The Salon 2 came years and years after Dr. Olive and Toole performed their research there. From what I recall, the testing there relied on turntables rotating loudspeakers and the technique I mentioned: symmetrical placement in a room behind a curtain (i.e. two at a time).


Loudspeaker is put on a lazy susan in an anechoic chamber. A series of frequency response measurements are then taken with that microphone array giving you a slice in 3D Space from horizontal to vertical. The turntable is then moved precise number of degrees and another array snapshot is taken. This is repeated until you have completed 360 degrees. Once there, you have 72 measurements of a 3-d space around the loudspeaker. These are independent data points. They are then combined into three different curves, when in turn are combined into weighted composite for prediction of loudspeaker preference/performance.

Come on Amir. In one picture they show a room with near purple carpet, with red (bright red) bases on blue chairs. They may think we aren't on to them, but such a visually dissonant combination of colors would seriously hamper the aural perception of the more sensitive listeners. Leaving them clear to hide the intentionally diminished sensitivity of the most sensitive listeners among the manipulatable results of common swine listeners. Now I don't know how this works, but I just know the results are no good, and don't apply to really experienced audiophiles. We aren't like other people, and don't like it when a big company tries to pretend they are just looking for the truth about good speaker design. Especially when the results keep showing the emperor has no clothes, and even common folk have a preference for the same sound qualities as audiophiles. Somehow I tell you it is just wrong. Even when it results in superb sounding speakers it is still wrong I tell you. Wrong.

Haha! Haha!

 

[MylesBAstor]

My reaction would be like JA's when that happens. He shows puzzlement why the reviewer said X, when the measurements show the opposite. If there is a dip in mid-frequencies for example, the reviewer can say all he wants about how great the vocals are. I won't believe it because I know there is no model of fidelity that calls for sucking out the vocals that way. If I were hearing the person in the flesh, there would not have been that suck out. And I know loudspeakers can be designed that sound great without that dip.

So the measurements could never be wrong? Only the reviewer?

 

[microstrip]

Everyone can learn to extract insight from these measurements. Again, I encourage you to watch John's video, and read his articles. So there is a fix to lack of education. There is no fix for lack of data that is not provided in a review. If you have the measurements, you can learn to utilize it. If you don't , that is the end of the road.

Should we carry a poll in WBF? What percentage of our readers know how to compute max current from the phase and amplitude information? Or do we want graphs just to decorate our rooms?

 

[microstrip]

(...) Note that magazine reviewers are not measuring this. They don't have access to anechoic chamber so they approximate by keeping the microphone close to the loudspeaker. That technique has limitations such as lack of integration of drivers at too close or a distance, or inclusion of too many reflections at farther back. So while informative, and decent approximation, they are not the same thing.
(...)

Fortunately magazine reviewers are not measuring a lot - most of it would be a source of misinformation. Compare the results of German Audio, Hifi-critic , Stereophile and/or Soundstage for the same speaker. They have significant differences and are grossly incomplete. Surely I assume that Harman professionals or NRC do it correctly.

 

[mep]

Should we carry a poll in WBF? What percentage of our readers know how to compute max current from the phase and amplitude information? Or do we want graphs just to decorate our rooms?

Better than that, let's take a poll and see how many people (beyond the usual suspects that hang on and second everything Amir says) think every audio reviewer should be taking measurements of everything we review. Let's take another poll and see how many people (again beyond the usual suspects) think reviewers should be leading the charge to demand OEMs provide a full set of measurements for all of their products. And if we could get a modicum of honesty, how many people even care if there are measurements? And if you don't know the difference between specifications and measurements, your vote doesn't even count.

 

[rockitman]

Better than that, let's take a poll and see how many people (beyond the usual suspects that hang on and second everything Amir says) think every audio reviewer should be taking measurements of everything we review. Let's take another poll and see how many people (again beyond the usual suspects) think reviewers should be leading the charge to demand OEMs provide a full set of measurements for all of their products. And if we could get a modicum of honesty, how many people even care if there are measurements? And if you don't know the difference between specifications and measurements, your vote doesn't even count.

I'm for the who cares measurement poll. I graphed enough during calc classes in college.

 

[still-one]

Better than that, let's take a poll and see how many people (beyond the usual suspects that hang on and second everything Amir says) think every audio reviewer should be taking measurements of everything we review. Let's take another poll and see how many people (again beyond the usual suspects) think reviewers should be leading the charge to demand OEMs provide a full set of measurements for all of their products. And if we could get a modicum of honesty, how many people even care if there are measurements? And if you don't know the difference between specifications and measurements, your vote doesn't even count.

I don't care whether I ever see a measurement for a piece of gear I am considering purchasing. I will look at "specs" to see whether speaker A and amp B are in theory compatible but in the end it only matter what they sound like in my room. Other than my speakers everything I own was A/B'd (and the occasional C) to see if it sounds better than what I was currently using. Sometimes they did and others not so much.

 

[MylesBAstor]

Hey, Myles, I subscribed to quite a number of high end startup, subjective-only, thank you, publications in the 70's and '80's. They all disappeared, too. One of them was yours, as I recall. Was it Ultimate Audio? I do not remember. But, whatever it was, you never completely fulfilled my subscription. You probably owe me a few bucks, you, SOB.

I am indeed impressed with how staunchly the reviewers and their fans defend the totally loosey-goosey word of subjective reviews and reviewers against any and all alternatives and possible improvements to the dialogue in trying to reach a better understanding of the "truth". Subjective reviews and reviewers are, frankly, just all over the place with no consistency whatsoever in the quality or accuracy of the information they convey.

It is not one, easily characterized, monolithic "thing" at all. There are some truly bad actors out there, along with the good. But, like all human endeavors, they have a wide range of usefulness, credibility, technical sophistication, honesty, experience, etc., etc. There are precious few, very few, who I like and trust, not that I would ever buy just on their recommendation, either. But, I pay attention closely to the "good guys" while completely ignoring the "bad guys", though some of them are very skilled and engaging writers of total, egocentric BS, IMHO. Oh, yeah, there are a lot of famous names, past and present, on my "bad guy" list, more bad than good by far.

My tiny list of most trustworthy reviewers are willing to give subjective opinions. But, at the same time, they understand and appreciate the value of audio science and greater objectivity in trying to reach a more unbiased presentation of what they see as the essential truth. Any reviewer who thinks needs look no further than his own ears and prior listening experience, whatever that might actually be, Lord knows, is not on my list.

We might have specs on equipment, but we have nothing like that on reviewers, their rooms, their listening experience, etc., etc. Oh, wait, the great HP was going to unravel that for us for awhile by publishing reviewers' astrological signs, which he did as I cancelled my TAS subscription while barfing into the toilet. That was one of the most abject low points in the history of subjective reviewing and audiophilia, IMHO. But, subjective reviewers are the self-appointed heroes of their own world, the gurus, the "experts", the wheat, not the chaff, the non-wankers. Oops, sorry, wrong thread.

Sorry to appear so totally cynical. I know a number of reviewers personally, and I have visited their rooms and heard the equipment being reviewed. The good ones are totally honest, fair minded and desirous of searching for and trying to convey the truth of how a component really sounds to their fellow audiophiles. They do not get paid a lot, and it is not at all an ego thing for them at all. It is about the challenge of learning and sharing the knowledge. They have sufficient humility to know what they do not know and they welcome any additional useful information, even if that information might, God help us, come from the objectivist camp. They are open minded and they see the many pitfalls their colleagues have fallen into, and they admit it, honestly. Yet they persist and they do a good job and try to do an even better job. So, my belief in a better side of human nature is happily confirmed by these guys. But, then there are the others....

So, how do we tell one kind of reviewer from the other?

I suggest we send all the reviewers to Harman's spinarama room, put them on the pedestal and subject them to some rigorous testing and scoring of their "fidelity".

This us all totally subjective, of course, so YMMV.

I think that was a very fair assessment.

 

[esldude]

It should raise eyebrows because no panel set up correctly sound like that graph - and I have owned both MLs and Magnepans, dating back to the 80s.

Do you have any measurements of them? I have measured a number of panel speakers. About half show measurements not all that different from the one Harman shows for the M-L they tested. Also not different than several panel measurements from Stereophile. So no it didn't raise my eyebrows. The people who own those speakers look at the results and say much the same. They don't sound like those measurements. However, when I get them to move the speaker and make other changes which improve those measurements some they agree it is an improved listening experience. I still can't make a flat even response from uneven panel responses. Only ameliorate the damage. The amelioration seems to work though.

So it would be interesting, if you heard your panel speakers sighted and were happy with them, would you then in a blind preference test find them superior or inferior to a better measuring speaker. Harman data suggests your panels might not please you as much in sightless comparisons as you would expect. I haven't been in a position to try that sort of comparison myself directly. The measurements of the M-L they show aren't really surprising.

BTW, in my limited sampling of measuring panels, the hashy response seems more prevalent in curved panels than flat panels. Though both show some of it.

 

[microstrip]

Better than that, let's take a poll and see how many people (beyond the usual suspects that hang on and second everything Amir says) think every audio reviewer should be taking measurements of everything we review. Let's take another poll and see how many people (again beyond the usual suspects) think reviewers should be leading the charge to demand OEMs provide a full set of measurements for all of their products. And if we could get a modicum of honesty, how many people even care if there are measurements? And if you don't know the difference between specifications and measurements, your vote doesn't even count.

IMHO the big question is what magnitudes do we want the reviewers to measure. Just the Harman ones that have been openly publicized?

 

[BlueFox]

Better than that, let's take a poll and see how many people (beyond the usual suspects that hang on and second everything Amir says) think every audio reviewer should be taking measurements of everything we review. Let's take another poll and see how many people (again beyond the usual suspects) think reviewers should be leading the charge to demand OEMs provide a full set of measurements for all of their products. And if we could get a modicum of honesty, how many people even care if there are measurements? And if you don't know the difference between specifications and measurements, your vote doesn't even count.

Personally, I do not really care if measurements are published. As an engineer, I do read them, but at the same time, at least for electronics, expect them to be close to perfect for just about any piece of gear, from a $300 AVR, and up. How it sounds is what I am interested in, and the more reviewers, and forum posters, who agree give me a much better idea of the sound ability of gear than do measurements.

Obviously, from a design perspective, measurements are a necessity, but not from a listening perspective. However, they are useful for endless Internet discusions.

However, measurements for cables are probably useless since we cannot yet measure what is occuring at the atomic level, and I suspect that is where the audible differences between cables are originating.

At the end of the day, these discussions are moot. I buy gear to listen to and enjoy music, and that is the bottom line.

 

[Phelonious Ponk]

Can you imagine the hue and cry from OEMs if every reviewer started taking full sets of measurements and publishing them like they were facts?

Here lies the ultimate problem with your approach. Full sets of measurements are facts. It's the subjective, poetic nonsense that is fiction.

Tim

 

[amirm]

First of all thanks for clarifying that they measure in an anechoic chamber - but the ML response graph looks to me quite wrong.

What's wrong about it? Here is stereophile measurements of it [models may not match]:

Here is Dr. Olive's:

Both show clear resonances in the panel at high frequencies (the chewed up response). JA comments on the same:

"The panel's output is less easy to interpret. The top octave is shelved down, while the mid- and low-treble are broken up by peaks and notches which may be due to resonances, or to interference effects specific to the particular microphone position chosen. " He also shows the same problems off-axis:

By the way, Harman listening test comparison was performed in the larger multi-channel room ("MLL") whose picture I post.

And no, no one thinks there is some sort of corrupt agenda here, it's just that I personally cannot take those "researchers" seriously, who perform listening tests: a) in that room with other speakers present (whose drivers do and will contribute to the sound of others playing); b) using Proceed amps.

How come neither one of those had the deleterious effect on the better performing loudspeakers?

So I ask, what does anyone think about Proceed amps driving all kinds of loads, including those dropping to 1ohm or less in the treble?

I have listened to this demo and did not hear remotely any amplifier straining. How can you tell that between that chewed up frequency response and the amplifier, the latter is responsible for the preferences?

Would Proceed be able to drive the Q5, an extremely difficult load, if not only by its phase anomalies (as measured by magazines).

At these levels, no problem at all.

And more than that, what about the Proceed sound?

Here are the reviews of the Proceed: "The BPA 3's frequency response was flat from 20Hz to 20kHz, and nearly as good from 10Hz to 50kHz (–0.07dB at 10Hz, –0.23dB at 50kHz at 2W into 4 ohms). The frequency response in the balanced mode was slightly less flat: down by 0.41dB at 50kHz! "

0.23 db variation across the audible band. Now compare that to the differences in loudspeakers. Do you really think that the amplifier timbre dominates when the other thing is orders of magnitude more?

Come to think of it, I did listen to Proceed well over a decade ago, and frankly, I thought they were a bit of a joke.

Until such time that you can bring data that shows loudspeaker preferences change with amplifiers, that is neither there, nor here. This amplifier puts out 230 watts at 4 ohms which is more than capable of playing music at a few watts that was used in testing.

If these "researchers" work for those with deep pockets, why not use better electronics?

Because they are rooted in audio engineering, not beliefs audiophiles have. When testing for extreme colorations of loudspeakers, any difference an amplifier may (and I am being generous in allowing the possibility there), is in the noise.

No need to answer, I remain unmoved by Harman's approach and "research", which to me is, again, investigation more than anything else. It may be time for Harman to repeat those old tests with modern speakers and far worthier electronics, in far better rooms.

Regardless of the acoustic "research" Harman may have done, there is so much more to making a great speaker, and it all starts with the quality of the dynamic drivers. If Harman thinks they can build drivers the way Magico does or better, I'd like to hear about it.

If you are in Seattle area ever, I would be happy to show you what they can do. But again, I don't want to confuse Harman and its products with this research. This work dates back to NRC and predates Harman involvement. The message has remained consistent and ton of loudspeaker designers reference this work. You all are new to it but that is not the case in the industry.

 

[MylesBAstor]

What's wrong about it? Here is stereophile measurements of it [models may not match]:

Here is Dr. Olive's:

Both show clear resonances in the panel at high frequencies (the chewed up response). JA comments on the same:

"The panel's output is less easy to interpret. The top octave is shelved down, while the mid- and low-treble are broken up by peaks and notches which may be due to resonances, or to interference effects specific to the particular microphone position chosen. " He also shows the same problems off-axis:

By the way, Harman listening test comparison was performed in the larger multi-channel room ("MLL") whose picture I post.


How come neither one of those had the deleterious effect on the better performing loudspeakers?


I have listened to this demo and did not hear remotely any amplifier straining. How can you tell that between that chewed up frequency response and the amplifier, the latter is responsible for the preferences?


At these levels, no problem at all.


Here are the reviews of the Proceed: "The BPA 3's frequency response was flat from 20Hz to 20kHz, and nearly as good from 10Hz to 50kHz (–0.07dB at 10Hz, –0.23dB at 50kHz at 2W into 4 ohms). The frequency response in the balanced mode was slightly less flat: down by 0.41dB at 50kHz! "

0.23 db variation across the audible band. Now compare that to the differences in loudspeakers. Do you really think that the amplifier timbre dominates when the other thing is orders of magnitude more?


Until such time that you can bring data that shows loudspeaker preferences change with amplifiers, that is neither there, nor here. This amplifier puts out 230 watts at 4 ohms which is more than capable of playing music at a few watts that was used in testing.


Because they are rooted in audio engineering, not beliefs audiophiles have. When testing for extreme colorations of loudspeakers, any difference an amplifier may (and I am being generous in allowing the possibility there), is in the noise.


If you are in Seattle area ever, I would be happy to show you what they can do. But again, I don't want to confuse Harman and its products with this research. This work dates back to NRC and predates Harman involvement. The message has remained consistent and ton of loudspeaker designers reference this work. You all are new to it but that is not the case in the industry.

You do realize there is no consensus on the ways to properly measure dipoles-even down to heir efficiency?

BTW what does preference mean? Tonality, frequency response, imaging, dynamics, etc? The same to everyone?

 

[esldude]

You do realize there is no consensus on the ways to properly measure dipoles-even down to heir efficiency?

BTW what does preference mean? Tonality, frequency response, imaging, dynamics, etc? The same to everyone?

So let us ignore what Amir has noted before, and start round the circle again? (the Bill Clinton tactic)