Why Some Audiophiles Fear Measurements

[Old Listener]

But, as incomplete as they may be, to go from 'measurements do not tell us all' (meaning, I hope, the current incomplete measurements) to 'the measurements we have tell us nothing' is to throw the baby out with the bathwater.

If I had a dollar for every time that I've read this faulty train of thought from ''measurements do not tell us all' ... to 'the measurements we have tell us nothing', I'd be able to afford some really expensive gear.

The striking thing is that the people who most stridently proclaim that current measurements are inadequate have no interest in establishing new and improved measurements.

Bill

 

[Gregadd]

Using your ears IS fine and needed, but hidden within that phrase is the (sneakily unspoken) exhortation to 'ignore the measurements'. That is what makes it deceitful, not using your ears to find your preference. Nothing wrong with that.


Surely we pervert the hobby when we require that the end user " measure or engage in some scientifc method." "using your ears is fne" is like saying to the eater at a fine restaurant that using his taste buds, sense of smell and vision is "ok." the ingredients in the chefs recipe is but a curiosity for the eater. All that matters is that the meal tastes great. How it became so is the sole province of the chef. For me no one will ever convince me that the meal was bad by showing me the recipe.

Listening is the reason d'etre. Not a an alternative metric.

The only measurement rquired of the end user, should he so desire, is to meausre his room. Mike has done that to the nth degree.

 

[Ron Party]

I keep reading in subjectivist Audiophile posts that they feel some are trying to force them to change the way they go about their audio hobby. Some are joy killers. Sob, sob, sob. Utter nonsense. Nobody is requiring anyone to do anything.

Subjectivists are so worried their golden ears might be proven unrealiable that they feel persecuted by skeptics and scientists. Sounds like the audio equivalent of an inferiority complex.

Face it. Golden ears, heck everyone, have seriously less than perfect hearing, less than perfect memory, less than ..., well, we're human and were less than perfect. Subjectivists would seriously advance the cause if they could, maybe, possibly, even one time, just show an ounce of humility.

So for the love of ..., stop erecting straw men arguments. And, of course, no one will convince you of anything because you don't understand, or worse flat out reject, the scientific method. Instead of even trying to learn about where rationalism leads one in matters audio you invoke a completely inapposite argument.

Why do Audiophiles fear measurements? Heck, the question seemingly could be restated as why don't Audiophiles understand preference? Or why don't Audiophiles acknowledge their is a limit to their being? Obviously these are rhetorical questions, and obviously they don't apply universally to all audiophiles, but when some people categorically reject science and wear their Audiophile *credentials* on their sleeves well, at least in an on line blog, you better believe you will be exposed.

See, some of us care deeply about the sound we're getting and we are humble enough to know that we have boundaries. We admit that we can be and often are fooled. We admit that we can be and almost always are biased. We admit that what one of us might like or think is the best another might think the opposite, and we are not offended by that - IOW, we understand the concept of preference. We admit that, in some cases, measurements are more reliable than our less than otherworldly, godly, golden ears hearing abilities. For us, understanding and applying the scientific method enables us to get even better sound so we can enjoy the music even more. So when we read this elitist "Trust Your Ears" mantra, when what you really mean is trust your superhuman ears and eyes, well we call B.S.

 

[terryj]

Hi greg (oh, and also Mike)..I have been meaning to say that mikes method of auditioning is completely fine, there can be no arguments with it.

No-one expects people to buy without audition, or buy according to specs only (me? I'd always audition loudspeakers, after that I don't care for how anything sounds, so an amp only needs to have sufficient power yada yada....I realise I would certainly be in a minority).

But, we are not so much talking about any particular individuals method of seeing if a piece of gear appeals to them. We are talking about an industry wide manifestation, we are talking about the industry (now that is, I guess that is off topic!! haha) not the individual.

So greg, we all take different pleasures from any given hobby surely, so for some to enjoy 'how amps work' (even tho that is not for me) is ok no? Some like the technical and scientific/engineering aspect, others don't. I find it a big stretch to say enjoying that is a perversion of the hobby.

BUT, it does give us an insight into the 'rabid' subjectivist viewpoint.......maybe not quite what you intended but nevertheless. So, especially when placed against the particular part of the post you were addressing rather ironic and rather freudian eh?

It has one from 'ignore the measurements' to 'it is a perversion to consider audio with any sort of scientific view'.

Hmm, think you proved my point.

Listening is the reason d'etre. Not a an alternative metric. For sure it is, think we covered that quite recently. From the industry standpoint, what drives improvements? A succession of different ears with different tastes trying every possible dish willy nilly...in other words no possible consensus...or theory first, design second followed by engineering validated by measurements??

Now, the OP mentioned why audiophiles fear measurements, but what I find interesting is that the BEST available speaker measurements do not come from the industry, but from the OPoster! Anyone else find that 'amusing'?

Why does it bug me a little that we, the customer, loses out with this state of affairs? Because it is a deception, the pretence is that it serves the customer, it does not. If *they* were serious about it there would be a push for these accurate measurements, a desire to find out what/which ones correlate with out perceptions...then we would have all of the industry posting measurements rather than Jeff being an oddity, the one off.

Sean's results would be looked at more carefully, where is any sort of validation (or challenging..by experiment) of his results by the mags? Heck, if they (the mags) were halfway serious then all of the staff writers would be on a bus for a personal tour/experience of harmons facilities, maybe all do the listening course! That may prevent a future report of prodigious bass when the opposite is true.

We can live in hope.

In their own field they set the rules, determine who plays in their sandpit and who cannot. I use as exhibit A that unbelieveable pile of trash written by (lavorna??) called 'who are the fragile souls' or whatever. A view very much in your line greg, I'm sure you would approve.

How long do you think it might be till a 'right of reply' is awarded?? yeah, that's what I thought too. Still, as I said, it is their sandpit.

BUT, what happens is this. Whilst not allowing any alternative viewpoint in their own publications, annoying but ultimately fair enough I suppose, they then get on and populate forums and continue to push that viewpoint, all the while protected by the forums themselves. Because we are not allowed (usually) to mention dbt or any such thing.

So they get to push that viewpoint to their adoring audience and are simultaneously protected outside as well.

Given the high end tenor of this forum, that there ARE so many who no longer buy the company line is telling, and personally a bit of a relief.

 

[Phelonious Ponk]

What's not being said here, or being said too gently, is that we're not really talking about preference. Preference is great and knowing your own mind so you can find your joy is greater. There is no need to justify preference. On the other hand...

- Claiming that your ears are more accurate and hear something magical beyond the capabilities of calibrated scientific instruments

- Arguing that believing in measurements is equal to "listening to charts and graphs"

- Insisting, when what is measured doesn't sync with what is heard, when what is heard under controlled conditions does not reflect what is believed, that the methodology is problem, not the much more likely answer that human error, even prejudice, is at play

...this is not having, and embracing, preference. It is making excuses for why the data does not support your desire to believe that your subjective choice is objectively superior. It is rational judgement overwhelmed by the desire to be right.

The same people who can easily spend thousands of dollars for a watch that they know is less accurate than the clock on their cell phone cannot admit that a vinyl record playing on a Rube Goldberg of a deck is a less accurate reproducer of the recording's signal than a lossless file playing through a studio monitoring system. They cannot simply say "It is beautiful and it sounds good to me," they would rather create new words and believe in undiscovered mysteries to rationalize their subjective preference into the objectively superior choice. It has little to do with subjectivity vs objectivity, and nothing to do with music. I don't understand it, but there it is.

P

 

[Gregadd]

Measuring is for manufacturers and engineers. Listening is for audiophiles. ..and thank you for proving my point. That is the benign invitation to measure and engage in the "scientific method" is not a mere invitation to engage in intellectual discourse, but remains a thinly veiled accusation that those of us who "listen" are delusional and just plain wrong.

As far as not being allowed to mention DBT it has been mentioned ad nauseum. The problem with DBT/ABX is that its proponents would treat it as a conclusion in itself and not just a testing methodolgy.
The term subjectivist has been cast upon me by the arrogant who assume I have no science backgorund and ignore meausrements. I have no desire to become an audio engineer just to win an argument on an audio forum. I refer you to John Curls article on audio design. It is complex. If a genuis like him who has deddicated his life to audio design and is still searching for answers. how am I the layman going to break grown?

 

[Phelonious Ponk]

Measuring is for manufacturers and engineers. Listening is for audiophiles.

While that is probably a variation on #2 in my post above, I would agree whole-heartedly, that if you're not in the process of treating your room, measuring is not something audiophiles should waste good listening time upon. I would add that in too many cases, however, the following amendment is appropriate:

Measuring is for manufacturers and engineers. Denying the results is for audiophiles.

P

 

[Gregadd]

What's not being said here, or being said too gently, is that we're not really talking about preference. Preference is great and knowing your own mind so you can find your joy is greater. There is no need to justify preference. On the other hand...

- Claiming that your ears are more accurate and hear something magical beyond the capabilities of calibrated scientific instruments

- Arguing that believing in measurements is equal to "listening to charts and graphs"

- Insisting, when what is measured doesn't sync with what is heard, when what is heard under controlled conditions does not reflect what is believed, that the methodology is problem, not the much more likely answer that human error, even prejudice, is at play

...this is not having, and embracing, preference. It is making excuses for why the data does not support your desire to believe that your subjective choice is objectively superior. It is rational judgement overwhelmed by the desire to be right.

The same people who can easily spend thousands of dollars for a watch that they know is less accurate than the clock on their cell phone cannot admit that a vinyl record playing on a Rube Goldberg of a deck is a less accurate reproducer of the recording's signal than a lossless file playing through a studio monitoring system. They cannot simply say "It is beautiful and it sounds good to me," they would rather create new words and believe in undiscovered mysteries to rationalize their subjective preference into the objectively superior choice. It has little to do with subjectivity vs objectivity, and nothing to do with music. I don't understand it, but there it is.

P

Our ears and brain are more accurate than science for the tasks we perform on a daily basis. Have you trued to call your telephone company and encountered thier voice recognition software?

To some silence is a concession. It is unlikely I will convert anyone to my side with words. I speak to make sure everyone knows there is another side. For us 'subjectivist" I quote George Clinton of Parliment Funkadelic regarding arguments over who has the best band, "let's take it to the stage." When the music is turned on, we dominate the high end.

Back to my vacation. I gaze at the horizon across the ocean and am reminded how inadequate man is at trying to replicate Gods' work.

 

[Gregadd]

Denying the results is for audiophiles.
Laugh out loud.

 

[Jeff Fritz]

Cerainly no one would suggest that the measurements should replace the listening. That is a simple straw man.

Measurements should supplement the listening when evaluating a product's high-fidelity credibility. If you think the listening impressions are all that matter, then my question would be whether you would value a system of checks and balances to confirm them. For instance, what if a reviewer stated the following:

"I have to say that the . . . loudspeaker system is the best that there is, the best that there ever was, and I suspect the best that there will ever be."

Would you expect there to be some type of objective element to supplement that listening impression? Would it be helpful? Or is that statement all you need?

 

[Johnny Vinyl]

Measuring is for manufacturers and engineers. Listening is for audiophiles. ..and thank you for proving my point. That is the benign invitation to measure and engage in the "scientific method" is not a mere invitation to engage in intellectual discourse, but remains a thinly veiled accusation that those of us who "listen" are delusional and just plain wrong.

As far as not being allowed to mention DBT it has been mentioned ad nauseum. The problem with DBT/ABX is that its proponents would treat it as a conclusion in itself and not just a testing methodolgy.
The term subjectivist has been cast upon me by the arrogant who assume I have no science backgorund and ignore meausrements. I have no desire to become an audio engineer just to win an argument on an audio forum. I refer you to John Curls article on audio design. It is complex. If a genuis like him who has deddicated his life to audio design and is still searching for answers. how am I the layman going to break grown?

I was going to say something quite similar to your statement above Greg, but perhaps in a slightly different context.

As someone who doesn't understand the vast majority of measurements posted by manufacturers, I have a tendency to only pay attention to those that I do...and let owner opinion and auditioning play more of a primary role when considering a component. That does not mean I don't value measurements I don't understand, because I sure do. I just feel...and here is where I agree with your statement Greg, that measurements are indeed more for manufacturers and engineers.

I would think that all of the manufacturers are paying close attention to the published results of their competition. I would also like to think, that after analyzing those results, there will be significant discussion with in-house engineers as to why some measurements measure as they do. Are we lacking in one area, and why? Or...why do we measure better here and not there? Hopefully everyone will be trying to improve and produce components that are continually bettered.

As a consumer I have to trust that published manufacturer data is 100% accurate, so that when I consider a component I can have faith in the honesty of the report, but that is where it ends for me. The measurements only tell part of a story, so the next step for me is to find user reviews of in-home experiences and then (if satisfied) proceed with investigating dealers for a possible audition.

In the end I would say they are all of equal importance. Using measurement stats alone is a foolish endeavor, as is trusting your golden ears exclusively.

John

 

[FrantzM]

I keep reading in subjectivist Audiophile posts that they feel some are trying to force them to change the way they go about their audio hobby. Some are joy killers. Sob, sob, sob. Utter nonsense. Nobody is requiring anyone to do anything.

Subjectivists are so worried their golden ears might be proven unrealiable that they feel persecuted by skeptics and scientists. Sounds like the audio equivalent of an inferiority complex.

Face it. Golden ears, heck everyone, have seriously less than perfect hearing, less than perfect memory, less than ..., well, we're human and were less than perfect. Subjectivists would seriously advance the cause if they could, maybe, possibly, even one time, just show an ounce of humility.

So for the love of ..., stop erecting straw men arguments. And, of course, no one will convince you of anything because you don't understand, or worse flat out reject, the scientific method. Instead of even trying to learn about where rationalism leads one in matters audio you invoke a completely inapposite argument.

Why do Audiophiles fear measurements? Heck, the question seemingly could be restated as why don't Audiophiles understand preference? Or why don't Audiophiles acknowledge their is a limit to their being? Obviously these are rhetorical questions, and obviously they don't apply universally to all audiophiles, but when some people categorically reject science and wear their Audiophile *credentials* on their sleeves well, at least in an on line blog, you better believe you will be exposed.

See, some of us care deeply about the sound we're getting and we are humble enough to know that we have boundaries. We admit that we can be and often are fooled. We admit that we can be and almost always are biased. We admit that what one of us might like or think is the best another might think the opposite, and we are not offended by that - IOW, we understand the concept of preference. We admit that, in some cases, measurements are more reliable than our less than otherworldly, godly, golden ears hearing abilities. For us, understanding and applying the scientific method enables us to get even better sound so we can enjoy the music even more. So when we read this elitist "Trust Your Ears" mantra, when what you really mean is trust your superhuman ears and eyes, well we call B.S.

Great Post and one which expresses much better what I tried in my last one.. That the specter of objective measurements is humbling. for some it can be almost humiliating.

 

[Alan Sircom]

I'm assuming what you eventually got to above was that the measurements missed something that the listening caught. That is definitely one of the things that could be wrong. It could be that the wrong measurements were taken, or the instruments were not properly calibrated...lots of possibilities. The blind listening test methodology could also be "wrong." It's not hard to get it wrong and, in fact, if research professionals were not in charge, the odds are pretty good.

P

Well, you see... that's the problem. If you hold that audio fidelity can be neatly summed up by assessment of frequency response, distortion, signal to noise ratio and time-based errors (as Ethan Winer proposes), all the products I have mentioned are functionally identical.

It seems unlikely that the instruments were not properly calibrated because:

1. They were measured side by side, so that if there was a mis-calibration, the product results would be equally mis-calibrated
2. The fact that the measurement and listening tests were initially so divergent sent up warning signs, so we had the product tested at a second lab. With identical results

Similarly, because the results were so divergent with the measurement, we established a more stringent layer of tests than usual. Our usual blind, level-matched AB tests were basically qualitative and we needed to test quantitively for identicality, so a double-blind, level-matched ABX series was put forward. We used the same listeners (who already knew the environment, the system and the music content) and were to conduct the tests over two day sessions, with different sets of listeners across both days, which would give 12 separate ABX tests. The test was effectively halved, because in both sessions the listeners refused to return for the afternoon session because they claimed they 'felt like lab rats', even though they were used to spending two or more at a time on blind AB testing.

This meant I ended up with eight distinct ABX tests, of which the subjects correctly identified the products in all eight cases. Unfortunately, even eight out of eight is too low to eliminate sheer chance with better than 95% confidence but mindful of the first day revolt, I did ask the subjects on day two to qualify their ability to identify one product over the other. Of the four tests run in the second session, three scored a four and one a five in a zero-to-five scale of ease of identification. I am not sure how relevant the qualifier is, and if it smacks of desperation in the face of a collapsing test, you got it.

Despite the lack of statistical robustness due to the 'match abandoned' nature of this test, I still maintain this is relevant. It highlights a potential for objective elements to manifest themselves in a controlled listening environment that do not register on paper.

 

[Jeff Fritz]

As a consumer I have to trust that published manufacturer data is 100% accurate, so that when I consider a component I can have faith in the honesty of the report, but that is where it ends for me.

Coincident Speaker Technology Total Eclipse Loudspeakers: "Claimed sensitivity is 94dB/W/m . . ."

http://www.soundstage.com/revequip/coincident_total_eclipse.htm

Measured sensitivity is 86dB: http://www.soundstagemagazine.com/measurements/coincident_total_eclipse/

See a problem? Oh yeah, it sounds 94dB. Gotcha.

 

[Johnny Vinyl]

Coincident Speaker Technology Total Eclipse Loudspeakers: "Claimed sensitivity is 94dB/W/m . . ."

http://www.soundstage.com/revequip/coincident_total_eclipse.htm

Measured sensitivity is 86dB: http://www.soundstagemagazine.com/measurements/coincident_total_eclipse/

See a problem? Oh yeah, it sounds 94dB. Gotcha.

Kinda proves my point.......don't just go by measurements. On the other hand.....who's right? You or them?

John

 

[MylesBAstor]

Coincident Speaker Technology Total Eclipse Loudspeakers: "Claimed sensitivity is 94dB/W/m . . ."

http://www.soundstage.com/revequip/coincident_total_eclipse.htm

Measured sensitivity is 86dB: http://www.soundstagemagazine.com/measurements/coincident_total_eclipse/

See a problem? Oh yeah, it sounds 94dB. Gotcha.

OK that's interesting and what Hi-Fidelity did before JGH left the magazine. But the disconnect one must ask is then how does 86 dB sound like 94 dB? Something in the measurements isn't telling the whole story.

 

[steve williams]

I have a question for Jeff, Alan or Myles or anyone for that matter

When a speaker is sent for review or when a speaker is heard under ideal room conditions, has there ever been an instance where you "liked" the speakers only to now have a look at the measurements, which if "bad" creates some bias on your parts only to now "not like" the speakers?

IOW do you listen to the speakers before or after reading the measurements? If you read the measurements before listening and the measurements are not so good, does this introduce bias to your reviews?

By the same token if you see measurements that are really good, has there been an instance when after listening to such speakers you didn't like their sound?

I guess what I am really trying to ask is "if you read the measurements before listening, does this not add some degree of bias either way on your part?"

Not trying to stir up the pot but just a question that I feel is worth asking

 

[Alan Sircom]

I have a question for Jeff, Alan or Myles or anyone for that matter

When a speaker is sent for review or when a speaker is heard under ideal room conditions, has there ever been an instance where you "liked" the speakers only to now have a look at the measurements, which if "bad" creates some bias on your parts only to now "not like" the speakers?

IOW do you listen to the speakers before or after reading the measurements? If you read the measurements before listening and the measurements are not so good, does this introduce bias to your reviews?

By the same token if you see measurements that are really good, has there been an instance when after listening to such speakers you didn't like their sound?

I guess what I am really trying to ask is "if you read the measurements before listening, does this not add some degree of bias either way on your part?"

Not trying to stir up the pot but just a question that I feel is worth asking

No, it's not stirring up the pot, it's a very good question. And, no... I make it a point to know as little about the product during the listening test and ideally even until write-up stage. That includes price, if possible. That way I get a feel for the product's performance and its relative position in the pantheon of similar products.

That means if I listened to product X and liked it and it transpired it measured like a POS, I'd go into print liking something that measured like a POS. That happens sometimes, but not as much as people expect. Usually ears and labs correlate relatively well.

 

[steve williams]

But please answer my question.....


Has there been an instance when after reading the measurements that your opinion was swayed based on what you heard?

 

[Johnny Vinyl]

No, it's not stirring up the pot, it's a very good question. And, no... I make it a point to know as little about the product during the listening test and ideally even until write-up stage. That includes price, if possible. That way I get a feel for the product's performance and its relative position in the pantheon of similar products.

A sincere question for you Alan!

When you've done the review and are ready to compose it into words for publication, do you make a recommendation or provide an opinion on it's sound or on its measurement data?

John

PS: Steve....that was a wonderful question.