Why Some Audiophiles Fear Measurements

[Phelonious Ponk]

in the last month or so i've been lucky to have been involved in the development process of a tonearm. it's already an established product but the designer has been improving it. he has used my room and system to confirm the various changes he has been making. first we listen to his current tonearm; then the last version of the improvements, and then the newest changes.

it's been a facinating process to witness first hand. the designer is a musician and composer by profession, and his tonearm design is supported by the department of engineering of a major university, so it's a serious product.

but the only way to really know whether the changes are an improvement is to listen.

I mean no offense, Mike, but in any other scientific or engineering endeavor, this would be considered unprofessional at best. Imagine BMW developing a new front end suspension system, testing it not in a laboratory and on a test track on multiple vehicles under controlled conditions, not using measurements and metrics, but on an individual BMW owner's car, in his garage, and by driving around his neighborhood, relying on the subjective impressions of the designer and his buddy, the BMW lover. No chance of bias there, huh?

A couple of guys listening in somebody's living room is not only not the only way to know if the changes are an improvement, it's not even a reliable way. It's not a repeatable way. It's not a professional, scientific or credible way. There are almost no areas of design and engineering outside of "high-end" audio, where this sort of design and testing methodology, and the resulting product, would be taken seriously. Of course all of that is not to say that a superior product can't result from such a process. There is always luck.

P

 

[Ron Party]

but the only way to really know whether the changes are an improvement is to listen.

We all have our own methodologies, but this sort of a blanket absolute statement is at best, well.., and at worst, just flat out wrong, unless you are talking strictly about improvement = preference. In fact, this absolutism is a complete rejection of science.

 

[Mike Lavigne]

-P- and Ron,

no problem. i think you guys are missing my point. i never represented that the listening done in my system was the whole process; i wrote "he used my room and system to confirm the various changes he had been making". i really do not know the steps he took to get to the point of listening in my room. as i mentioned; he has his own process of design and manufacturing. my room simply had a level of resolution which allowed him to confirm choices prior to manufacturing. a final proof of performance to some degree i suppose.

i repsect the contributions science makes to our music reproduction enjoyment. but it still comes down to 'enjoyment' ultimately......and we gotta listen for that part.....or at least this designer thought that was helpful.

if you attend RMAF you can listen to this arm yourself and decide whether this designer is thinking wrong.

mike

 

[Phelonious Ponk]

-P- and Ron,

no problem. i think you guys are missing my point. i never represented that the listening done in my system was the whole process; i wrote "he used my room and system to confirm the various changes he had been making". i really do not know the steps he took to get to the point of listening in my room. as i mentioned; he has his own process of design and manufacturing. my room simply had a level of resolution which allowed him to confirm choices prior to manufacturing. a final proof of performance to some degree i suppose.

i repsect the contributions science makes to our music reproduction enjoyment. but it still comes down to 'enjoyment' ultimately......and we gotta listen for that part.....or at least this designer thought that was helpful.

if you attend RMAF you can listen to this arm yourself and decide whether this designer is thinking wrong.

mike

Mike, I misunderstood you. There is absolutely nothing wrong, of course, with finally testing a product in a real world situation. Sounds like that is all your friend is doing. It also sounds like it is a great compliment to your system and your room.

P

 

[Mike Lavigne]

In fact, this absolutism is a complete rejection of science.

i don't see it that way at all. i did not say that the only way to discover if it measured better was to listen. measuring better and someone thinking it sounds better are two different things. related mostly but not the same.

it's debatable sometimes what measurement results might be interpreted as better or worse.

have you ever spent time comparing tonearms? and cartridges? and phono stages? all at the same time. it's a very dynamic situation. lots of synergy, lots of science, lots of subjectivity. my system has 3 tt's, 4 arms, multiple cartridges, three phono stages. lots of context for consideration. it's the reality for analog products.

i can tell you this. if the arm did not sound as good subjectively the changes would have been rejected.

 

[silviajulieta]

as i type this; i'm listening to the preamp/phono stage that Mr. Fremer reviewed. boy; the bass is pretty good. i have another respected phono stage in my system and this one is subjectively better in the bass. i actually have 2 separate phono stages in the same model preamp that Mr Fremer reviewed, the other one is set-up and connected to a Mono cartridge and also has very nice bass.

i'll add that it did take a few tries before i was able to correctly match my Lyra Olympos SL cartridge and the Miyajima Premium be Mono cartridge to the preamp properly to get the proper bass performance. but that is typical of most phono stages.

i'm going with Mr. Fremer's subjective comments. of course; my perceptions are subjective too.

Dear Mike : That IMHO is the " easy way to go ": subjective, when this word appear everything added is useless: right?

Well, totally wrong, especially coming from a reviewer. Which kind of " scientific " evidence we need to believe in " some " measurements especially when we are talking on measurements made at the most critical " area/stage " in a phono stage unit named: RIAA eq, frequency deviation!!!! where that frequency deviation is not a " discrete frequency " that affect only that frequency but almost three music octaves!!!!

With all respect to you and MF: only a deaf can't hear it: from 500hz and down in the Vitus and in the Dartzeel not only was/is affected three octaves but almost six octaves!!! of ten/10 audible music octaves: almost 60% of the performance of that unit is out of target !. So what are you hearing? as Phelonious Ponk said: something is wrong really and deep wrong, for say the least. I know that both of you ( MF ) are not deaf, so what happen? do you or MF have a non-biased answers? could be that in those reviews MF made mistakes? and you?

We learn on our self errors, I never defend my mistakes: I learn to accept it, enmmended and in this manner I grow up and attain several rewards .

I heard that Daretzeel deviation ( way before STP measurements time and review. ) in three differents systems with four different cartridges. Why defend that unit? ( I'm not talking if you o MF likes its performance I just talking on evidence: hearing evidence and measured evidence. ) the evidence is there, deny it does not help Dartzeel, MF or you but only leaves/left " space to speculations ". It is obvious that even real scientific evidence/facts are not enough against " subjective ", nothing is.

" Subjective " is the way to say: I'm right end of controversy, period. As I said the " easy road ".

I don't care about Vitus or Dartzeel I care and worry the way/road you take in the whole subject.

I understand that you are biased through the Dartzeel because you own it but what about the Vitus or that famous 350K amplifier review with a 10+db frequency deviation?: subjective too?

Mike, when things happen one time I can accept your position on the subject but when the same " thing " happen one and again one and again many times only a stupid person or a real mad person can accept in the way you said it and I can tell you that as me several people that are reading this thread are not either: stupid or mad.

Regards and enjoy the music,
Raul.

 

[Ron Party]

no problem. i think you guys are missing my point. i never represented that the listening done in my system was the whole process; i wrote "he used my room and system to confirm the various changes he had been making". i really do not know the steps he took to get to the point of listening in my room. as i mentioned; he has his own process of design and manufacturing. my room simply had a level of resolution which allowed him to confirm choices prior to manufacturing. a final proof of performance to some degree i suppose.

i repsect the contributions science makes to our music reproduction enjoyment. but it still comes down to 'enjoyment' ultimately......and we gotta listen for that part.....or at least this designer thought that was helpful.

if you attend RMAF you can listen to this arm yourself and decide whether this designer is thinking wrong.

mike

Mike, based on your reply, I believe I was correct in interpreting what you previously wrote. I did not miss your point at all. If the sole determinant of whether there was an improvement is, as you wrote, "enjoyment", then improvement=preference.

It's this distinction those of us who give more than lip service to the scientific method try never to forget.

 

[silviajulieta]

That's why we don't grow up or grow up so slowly: everything in HE audio by CONVENIENCE is subjective!, end of debate: period.

How could we in this thread can find a " solution "/answers ( true/real answers. ) when suddenly any one of us take the " subjective flag " and that's it: end of discussion.

Why we take the threads in audio forums always like a contest where some one has to win ( better if me. ) or where we defend our opinion even if it is totally wrong? Why can't we have and " open and non-biased " debates? just a friendly debate!!! where no one lose and everyone win!!!, think about: all reside in each one attitude.

Regards and enjoy the music,
raul.

Btw, Ron I'm with you on your posts.

 

[Mike Lavigne]

Mike, based on your reply, I believe I was correct in interpreting what you previously wrote. I did not miss your point at all. If the sole determinant of whether there was an improvement is, as you wrote, "enjoyment", then improvement=preference.

It's this distinction those of us who give more than lip service to the scientific method try never to forget.

speaking only for myself, and not in any way for the arm designer, i agree 100% with your perspective about my viewpoint. i do not pretend to worry about scientific method. if i did, with my woeful lack of technical knowledge, i would be subjected to objectivist's interpretation of numbers for my system/room/software decisions. when i see where that gets many (not all certainly) objectivists subjectivly....i want no part of that. i'll carry on by trusting my ears. it's not perfect; but i'm enjoying my music and moving along the path in the hobby just fine....learning as i go by listening.

 

[The Smokester]

in the last month or so i've been lucky to have been involved in the development process of a tonearm...but the only way to really know whether the changes are an improvement is to listen.

This statement might be slightly exagerated but it HAS to be true.

Science and engineering are the means to fulfill the design requirements, but the design requirements are totally from ones' head (including the ears).

 

[Ron Party]

i don't see it that way at all. i did not say that the only way to discover if it measured better was to listen. measuring better and someone thinking it sounds better are two different things. related mostly but not the same.

I know what you wrote, Mike. I'm sorry to harp on this. Your words speak for themselves and, really, the only reason why I'm getting involved in this thread at all right now is, as I wrote in my last post, to emphasize the roles of subjective preference and measurements (the latter being the OT).

What you said was the only way to determine if there was an improvement is to listen. This is true if, and only if, improvement = preference. If improvement is determined by, for example, being truer to the source, then listening certainly is not the only way. It may not be a way at all.

Indeed, as you wrote, the arm might measure better but someone might think it sounded worse than the previous iteration. But how do we know this is true 100 percent of the time for all people, including those that made this initial assessment?

if the arm did not sound as good subjectively the changes would have been rejected.

I would expect no less. But I would expect all iterations of the arm sounded *good*. So what makes one version of the arm better, at all times, for all people, or stated another way, what constitutes an improvement and, equally if not more significant, how does one determine if there is an *improvement*?

These Qs bring the discussion back full circle to the OT: why are audiophiles afraid of measurements? Designers certainly aren't.

 

[Mike Lavigne]

Ron,

ok, how about the only way to confirm an improvement is to listen?

my point being that a measurment is just that, and only that. but the darn thing has got to sound good too. (yes, 'sound good' is subjective). in other words, the measurement must result in a preference. if not; what's the value of it?

Mike

 

[Mike Lavigne]

regarding the OT. it's not that i fear measurements, it's more that i'm not interested in them as any proof of anything. they might get my attention in particular situations and cause my to investigate a piece of gear. but once the music starts any measurement has zero value in the decision making. it either sounds right or not.

 

[Ron Party]

Determine = confirm. No difference.

The term "only" is an absolute. If we're going to be dealing in absolutes, then your statement is just a reiteration of that which we've been discussing. Correctly restated, the only way to determine/confirm preference is to listen. I know, everyone who's reading this thread knows I'm just repeating myself. This is my last post on this aspect of the discussion.

 

[Ron Party]

speaking only for myself, and not in any way for the arm designer, i agree 100% with your perspective about my viewpoint. i do not pretend to worry about scientific method. if i did, with my woeful lack of technical knowledge, i would be subjected to objectivist's interpretation of numbers for my system/room/software decisions. when i see where that gets many (not all certainly) objectivists subjectivly....i want no part of that. i'll carry on by trusting my ears. it's not perfect; but i'm enjoying my music and moving along the path in the hobby just fine....learning as i go by listening.

Hey Mike, I know you enjoy inviting people over to your (incredible) room to share some scotch or fine wine and listen to music. How 'bout the next time you are down here in the Bay Area, maybe for another listen at Steve's house, the scotch will be on me? Then after about 4 or 6 drinks, maybe I can convince you to move just one click more in the direction of embracing the scientific method. ;-)

 

[Mike Lavigne]

Hey Mike, I know you enjoy inviting people over to your (incredible) room to share some scotch or fine wine and listen to music. How 'bout the next time you are down here in the Bay Area, maybe for another listen at Steve's house, the scotch will be on me? Then after about 4 or 6 drinks, maybe I can convince you to move just one click more in the direction of embracing the scientific method. ;-)

fair enough; only i've never had 4 shots of Scotch before at the same time.....so i'd likely agree to anything at that point.

 

[silviajulieta]

Ron,

ok, how about the only way to confirm an improvement is to listen?

my point being that a measurment is just that, and only that. but the darn thing has got to sound good too. (yes, 'sound good' is subjective). in other words, the measurement must result in a preference. if not; what's the value of it?

Mike

Dear Mike: So to understand your point ( or maybe not. ). what about a real improvement in a specific measure or measures in an audio item that after your listening decide was not only an improvement but a degradation?

Could you give me an explanation about? your position? thank you in advance.

Regards and enjoy the music,
raul.

 

[terryj]

have I ever mentioned what interests me in audio is the human aspect?? haha. Not the gear (but certainly the music) but our collective human reactions.

I go back to an old police investigation truism, I think, not a policeman! 'Who benefits from the crime?'

crime may be an over the top term, but hey, on deeper pondering it could very well be the most appropriate term after all.

One thing I think we do all agree upon (???) is that measurements, as we usually encounter, are insufficient to tell the whole story. Either things like measuring the wrong things, smoothed and 'edited' measurements, simplified or trivialised measurements, that sort of stuff.

There are some who go further than that of course, who seem to claim that we are unable to do the correct measurements. That may or may not be the case, I think we could only answer that question after seeing how the 'proper' measurements fare.

But, as incomplete as they may be, to go from 'measurements do not tell us all' (meaning, I hope, the current incomplete measurements) to 'the measurements we have tell us nothing' is to throw the baby out with the bathwater.

Ok, so here is where I am going....who benefits from the 'pathetic' crop of measurements the industry presents us with??? SURELY we know more than what we are provided with.

Another way to view the same situation, who would benefit if we DID get more comprehensive and useful measurements.

The customer. The manufacturers that do give well engineered products.......................................................

Hmm, pretty short list. Perhaps some of you can think of ones I missed.

that the customer loses out by not having the correct data is a pretty damning state of affairs. Those manufacturers that can be 'relied' upon to provide good solid engineering are, paradoxically, the boring mainstream non exotic ones.

Where do we find the abysmal engineering at outrageous prices (often, don't want to paint with too broad a brush)?? the boutique 'forge our own path' guys.

So, we have an industry that encourages and promotes 'just listen to it', who (often) actively denigrate measurements as next to useless (which they are...if we go by the ones they themselves use) and the acolytes who take their gurus word as gospel.

Seems to me that we are lumbered with this appalling situation by the actions of the industry itself, in cahoots with the very expensive hi end guys with all the bling and fling. THEY are quite happy too that most resort to 'just use your ears'.

Using your ears IS fine and needed, but hidden within that phrase is the (sneakily unspoken) exhortation to 'ignore the measurements'. That is what makes it deceitful, not using your ears to find your preference. Nothing wrong with that.

So who benefits from the crime?? The mags that continually find the latest and greatest (which produces turnover, which in turn produces advertising revenue), the manufacturers that, if pushed to stand on equal grounds of engineering competence would fail dismally, so no wonder they are happy to grease the wheels with revenue.

I often wonder, when again I am reading another lament about the death of the hi end audio industry or it's lack of relevance to the mainstream guy, whether or not any of the writers can recognise some of the birds coming home to roost.

 

[Mike Lavigne]

Dear Mike: So to understand your point ( or maybe not. ). what about a real improvement in a specific measure or measures in an audio item that after your listening decide was not only an improvement but a degradation?

Could you give me an explanation about? your position? thank you in advance.

Regards and enjoy the music,
raul.

i assume what you meant to write was "not only not an improvement but a degradation?" instead of what you did write "not only an improvement but a degradation?" which does not make sense to me. is that correct?

if that interpretation is correct of your question, you are looking for an instance where some piece of gear measured better (whatever that might mean) but sounded worse to me.

considering your question, i suppose it might refer to some upgrade to a current piece of gear i owned, or maybe some gear which might be an upgrade compared to something i had already owned.

the problem for me to confront your question that way is that i simply don't much consider gear changes in terms of measurements. my approach would be to listen to the gear someplace and want to try it, or maybe someone will offer to send something for me to listen to, or i'll read about it and pursue it. i'll check the measurements in terms of compatability with my system certainly, but i mostly ignore distortion and noise floor specs as they are not reliable indicators of actual in system performance.

i auditioned my speakers at the designers home during the final design stage then got the second production set. i had the prototype of my preamp in home 6 months prior to production, and then one of the first production units. i had the first retail unit of my digital player. 2 of my tt's are one-off vintage redo's. i had prototypes of three of my four tonearms in my system as prototypes before i recieved the production versions. i had three versions of my RTR tape repro unit in home before i was satisfied with my unit.

so much of my gear is not exactly main stream. i have had a sonic target in mind and the gear had to measure up in listening performance.

so no; i cannot think of a situation where i was concerned enough with a measurement to consider it seriously in my decision. OTOH; my feedback on some of my gear likely has influenced it's design.

 

[Robh3606]

But, as incomplete as they may be, to go from 'measurements do not tell us all' (meaning, I hope, the current incomplete measurements) to 'the measurements we have tell us nothing' is to throw the baby out with the bathwater

Hello Terry

I couldn't agree more. The problem is that many times you don't see the measurements that really can differentiate a really good system from the norm. An example of this is damped titanium compression drivers and Beryllium diagrammed drivers. Both have a very audible effects. That said you would not be able to see it in the normal frequency response graphs that are normally published. You will see it in an ETC graph looking at the decay over time. Graphs like that are never published by manufacturers. So here you have a situation where you can clearly hear a difference but the published specs give no clue of why the damped or Be diaphragms sound better.

Rob