Measurements and the Correlation of What We Hear

[mep]

I don’t want to start another food fight, so I ask that anyone that posts a response to this thread do so with respect to our fellow members. I would like to discuss the relationship between science/measurements to the correlation of what we hear.

I believe there are some solid measurements that will correlate with what we hear. For example, if measurements indicate that a preamp, power amp, or speaker has no output below 50 Hz, we would all hear that (or at least I’m real sure we would). Ditto if the measurements showed no high frequency response above 8 kHz. We could maybe agree on a distortion threshold that would be audible to all and possibly a signal to noise ratio as well. Keep these thoughts in your back pocket…

I don’t know what percentage of our members believe that audio electronics are improving every year (or pick your own periodicity), but for arguments sake, let’s say that many do believe electronics are steadily improving. I think that many would agree that outstanding measurements have been achieved for many years by differing pieces of electronic components (preamps, power amps, digital components of all stripes, etc.) with reference to FR, distortion, and signal to noise ratio to name but a few. If we believe that components are steadily getting better with regards to sound quality (more accurate, more neutral for example), do we have any measurements that graphically show us how/why components today are better sounding than components of say five years ago? Aside from jitter in digital components, I’m not sure we do.

We have had components that measured ruler flat for years. We have had components with ultra-low distortion and with an outstanding signal to noise ratio for years. What I’m struggling with is that if we believe today’s components (which with few exceptions won’t really measure any better than many components of yesteryear) actually sound better, what science/measurements do we have to explain it? Do we really have any measurements that will show us why something sounds better?

I think the answer is no, we don’t. It doesn’t mean it couldn’t be measured if we knew what to measure and had the instrumentation to do so, but I don’t think we have measurements to point to in order to show why something sounds as good as it does. We can certainly use measurements to show when something is poorly designed or something is designed and built very well, but I don’t think those measurements will tell us why one high-end component sounds better than another even if everyone agreed it did (which of course would never happen).

In summary, my question is if today’s components are better sounding than last year’s or five years ago, what measurements tell us that is true?

 

[Bruce B]

I think one of the parameters that can't be summarily measured is the tonality/timbre of an instrument. We can certainly measure freq/amplitude and even soundstaging by the timing between both outputs, but not the difference between a Yamaha C7 and Bosendorfer or Fazioli.

 

[microstrip]

You can do a little exercise - go to the Paul Miller site:

http://www.milleraudioresearch.com/avtech/index.html

There you have the most comprehensive set of measurements of electronics available on the net - tens of measurements for every parameter, for many devices - even the vintage Krell KSA50. Try to figure out if you can find how these devices sound from all these figures. I can't.

 

[mep]

There you have the most comprehensive set of measurements of electronics available on the net - tens of measurements for every parameter, for many devices - even the vintage Krell KSA50. Try to figure out if you can find how these devices sound from all these figures. I can't.

And I guess that's my point.

 

[FrantzM]

I think one of the parameters that can't be summarily measured is the tonality/timbre of an instrument. We can certainly measure freq/amplitude and even soundstaging by the timing between both outputs, but not the difference between a Yamaha C7 and Bosendorfer or Fazioli.

If it can be heard it can be measured.

 

[steve williams]

Mark, in situations such as this when I am evaluating new gear in my system and find that I like it, as I have said here many times before, I ask myself "am I hearing something different or something better and if better, then how so"

I would also suggest that all too often what we are hearing is something different but not necessarily better.

How many times have we read comments from members about "how the sound stage is deeper, wider, less distortion, more nuance, better grip, more linear etc, etc, etc. Is this expectation bias or what?

If something sounds better or different shouldn't it really be measurable when compared to the original component.

 

[steve williams]

You beat me to it Frantz. I completely agree.

 

[MylesBAstor]

I think what might be more productive (as I tried to do in one piece) is have some designers weigh in on the topic. Designers certainly do a lot of measurments before releasing their electronics and speakers and would be interesting to see what they find useful.

 

[fas42]

Just had a quick look at the Miller Audio site (for the first time!) and there is almost nothing there that is useful in the report for assessing sound quality, for the Mark Levinson No53 Reference mono power amplifier that I looked at. As far as I'm concerned the only parameters of SQ value were the CCIR IM Distortion and Output Impedance, especially the former. The testing there should be enormously expanded, to be done at multiple power levels from very low to near maximum, with pairings of many different frequencies, with the two frequencies at very different magnitudes from each other, and with combinations of 3, 4, etc frequencies.

That alone should tell you vastly more about the significant differences between amplifiers, without even thinking about other sorts of tests ..

Frank

 

[mep]

If something sounds better or different shouldn't it really be measurable when compared to the original component.

Steve-That is my question. Are you saying that newer gear doesn't sound better than older gear, just different? And Frantz, ditto for you.

 

[microstrip]

Just had a quick look at the Miller Audio site (for the first time!) and there is almost nothing there that is useful in the report for assessing sound quality, for the Mark Levinson No53 Reference mono power amplifier that I looked at. As far as I'm concerned the only parameters of SQ value were the CCIR IM Distortion and Output Impedance, especially the former. The testing there should be enormously expanded, to be done at multiple power levels from very low to near maximum, with pairings of many different frequencies, with the two frequencies at very different magnitudes from each other, and with combinations of 3, 4, etc frequencies.

That alone should tell you vastly more about the significant differencies between amplifiers, without even thinking about other sorts of tests ..

Frank

Frank,

I will return to a point I have previously referred - you can take many measurements but you have to combine their results to return a set having only a finite few numbers that correlate with sound quality and have a causal connection with it. IMHO, suggestions of techniques that can not converge in such a result would not be useful estimators.

 

[mep]

I want to give a realistic scenario for the objectivists who believe in the power of science and measurements (and there is nothing wrong with that by the way): You are a happy owner of the ARC REF-3 and you trade it in for the ARC REF-5. You break in the ARC REF-5 and you have no doubt it is a much better preamp than the ARC REF-3 (and don’t get hung up on the ARC example, we could just as easily pick a SS component). The bass is better, the soundstage is bigger in all dimensions, the voices sound more holographic, yada-yada. As a subjectivist, what measurements can you point to that clearly indicate why the REF-5 sounds better than the REF-3?

Please don’t fall back on the argument that if we can hear it, we can measure it, because that is what I’m trying to address with this thread. If one component is truly better sounding than another, we should be able to point to the measurements and explain why it is so. We can’t have it both ways. If you believe in measurements and yet you can’t point to any that explain why one component sounds better than another, than I don’t see how you can turn around and tell objectivists that they aren’t hearing what they think they are hearing.

 

[mep]

I would also add that re-capping an amp will make it sound different than with the 20 year old caps, but some of those old caps "held on and re-released the signal" and thus can be considered more musical.........just something to chew on Mark.
Tom

Tom-Old caps with "memory" would not pass muster in today's high end. I don't know who would consider those to be more "musical." Those are the types of distortion that are easy to banish and bring added clarity to the music signal by using higher quality caps.

 

[fas42]

Frank,

I will return to a point I have previously referred - you can take many measurements but you have to combine their results to return a set having only a finite few numbers that correlate with sound quality and have a causal connection with it. IMHO, suggestions of techniques that can not converge in such a result would not be useful estimators.

Yes, it is reasonable to end up with only a fixed or limited number of figures specifying some parameters or qualities, but first you have to do the tests! In one sense the Miller material is a perfect example of doing things which in my book are a bit silly, over and over again there are tests of pure sinewave power into pure loads, not a single test into a load mimicking a real speaker of some standard type, hence almost useless for asessing behaviour into real loads.

Audio systems are complicated, our ear/brains are very sophisticated: simple testing will not yield results which are meaningful. A methodology for doing IM distortion tests would be to try a battery of preliminary variations, look closely at those results, and if there are any evident trends then do more vigorous testing in a very specific way tailored to the particular unit. Now you have evidence of real, measurable characteristics that distinguish that component, and hopefully correlate with listening data, from others.

Audio Note is a brand that historically has had quite bizarre test results, and it sounds different, so it can be done ...

Frank

 

[fas42]

Tom-Old caps with "memory" would not pass muster in today's high end. I don't know who would consider those to be more "musical." Those are the types of distortion that are easy to banish and bring added clarity to the music signal by using higher quality caps.

But the point would be, what conventional measurement technique would clearly show the effect of changing those caps when comparing the numbers before, and the numbers after?

Frank

 

[MylesBAstor]

There is, IMO, one incontrivertable fact, that is:

A hearing test, with sinewaves, or square waves, or triangle waves, or music at a lot of discrete volumes, can be used to record your ability to hear it or not and at what volume and frequency and thus, yes, measurements can correlate with hearing...........but, what happens inside your head as far as how accurate that sound is, well, there ain't no correlation. As I have said also before, the stereo image depth, stage, etc., is what happens in your brain. Yes, you can design speakers or record tricks to get exaggerated soundstage etc, but each persons brain will react or reconstruct it a bit differently.

I am saying that there is nothing in audio electronics signals that can not be measured, and its just a matter of how complex (how many tones, waveforms, nulling of signals, blah blah) that you want to do to slice down the differences in how any single amplifying device passes the signal through.

And maybe, since our hearing is not as good as it once was, perhaps we just don't hear the things that irrated us as much (thinking of excess treble and irration for example in early cd recordings/players...if you can't hear past 8 khz those frequencies might as well be light waves since your apparatus can not hear it, and that can be proven by measurments.

My opinion, when folks argue that an amp with 0.00003 percent TOTAL harmonic distortion across the frequency band sounds sterile, well, they are used to a lot of coloration. No problem with that. I am the first to pronounce manipulating sound to ones satisfaction by whatever means pleases you.

Hearing as a physical measurable thing, we can do.......perception or preference are in the minds of the beholders...

I would also add that re-capping an amp will make it sound different than with the 20 year old caps, but some of those old caps "held on and re-released the signal" and thus can be considered more musical.........just something to chew on Mark.

Tom

So you're saying whatever means are used to achieve those essentially "unmeasureable" levels of distortion have no effect? I think you'd have quite a few designers and published papers disagreeing.

http://www.parasound.com/pdfs/JCinterview.pdf

http://jockohomo.net/data/7470.pdf

http://www.q-audio.com/johncurl.pdf

http://www.reliablecapacitors.com/pickcap.htm

http://www.stereophile.com/phonopreamps/640

 

[amirm]

I had a brief conversation with Floyd Toole recently which in my mind crystallized part of the answer. Let's see if we agree on that here.

Distortion can come in three types:

1. Linear. Mark gave examples of this that are reflected in frequency response variations.

2. Constant non-linear distortion. This is distortion that is not source dependent.

3. Non constant, non-linear distortion. This is distortion that is content dependent.

Let's see if we agree that if the system is making linear transformation of the system, that can easily be measured. After all, a linear transformation changes the levels either absolutely or at different frequencies both of which we know how to measure. The effect on the listener is a bit less predictable but there is good bit of data on desirability of such things as flat frequency response and such, and correlation studies that Harman has done on speaker performance (wrt to linear performance) that says variations in measurements show up as variations in listener preference.

As we go to the other two categories, the difficulty of measurements and audibility correlation becomes harder. Let's take a compressed AAC file at 256 kbps. This file will have linear measurements using test tones that will be identical to the source file. Its level will also be (near) identical. Now feed the the encoder difficult material with sharp high frequency transients and the system generates non-linear distortion that is challenging to quantify as it is strictly data dependent. There is no one number that indicates what distortion there is or there isn't.

My discussion with Floyd was around his statement that humans easily hear linear distortions and that there is no difference between people in their ability to hear the same. I asked him then why it is that training does help with hearing compression artifacts. His answer was that non-linear distortion is indeed impacted by training and audibility does vary therefore between people. He went on to say that audio equipment has too little non-linear distortion for that factor to matter.

Putting the last bit aside for a moment, it seems that we have a good realization here that this problem has different facets in it. We know that when it comes to linear performance, equipment has achieved that quite well for some time. Therefore in that area, we have to agree that advances (outside of speakers) are not there.

Question therefore becomes the non-linear distortions. How much of them we have. And how well we measure them. I will punt the difficult first question . On the second, I know that when it comes to non-linear distortion, we rely on psychoacoustic modelling that gets us somewhat close to the truth but cannot achieve perfection. Certainly no simple measurement number comes even remotely close to guiding us there. Things like IMD and THD are useless in those regards as they are not based on what distortion we hear more of and what not.

I will stop here to get some reactions .

 

[DonH50]

I think one of the parameters that can't be summarily measured is the tonality/timbre of an instrument. We can certainly measure freq/amplitude and even soundstaging by the timing between both outputs, but not the difference between a Yamaha C7 and Bosendorfer or Fazioli.

Having measured years ago a Baldwin, Bosendorfer, Steinway, and Yamaha at college I can disagree with this. The changes in frequency and time domains were pretty apparent among the three pianos. Making the measurements and procssing them back then was non-trivial (no DSOs and digital test systems unless we built it), but the results were interesting. This was a Master's project for a friend of mine (I was working a couple of projects at the time as an undergrad, one to make a multiphonic synth when such were rare, and the second to build a servo sub, ditto).

 

[steve williams]

Steve-That is my question. Are you saying that newer gear doesn't sound better than older gear, just different? And Frantz, ditto for you.

That's a good question as newer gear should sound better than older and assuming the listener has made a wise choice. In my case Mark, coming up this audio ladder every speaker type I have owned in the past 17 years have been Wilson ( WP 5.1, WP 6, MAXX series l, X-1 Series lll Grand Slamm, X-2 Series l and now the X-2 series ll....notwithstanding I also owned the XS, and still use in my HT WATCH Center, surround and rear). Needless to say it is a sound that my ears like and moves me. Similarly in the same 17 years my amps have been, Krell KSA 250 KSA 300, 650m, 750Mcx, ARC Ref 600 Mk lll and Lamm ML 2.1 and now the present ML3. So I don't
change brands any too often but rather move up in the production chain. I have also said that for my ears and my system Krell amps were as good as it gets. I know you will like them Mark. I still use a Krell TAS in my HT.

So for me Mark sounding better might refer merely to increased power and/or larger speakers.

So similarly, to "my" ears there is just something simply magic to my ears using Wilson/Lamm gear. Now, just as JackD from Manila has stated so shall I ...I am a dealer of sorts for Lamm gear and this is also in my statement about myself in the Introduce Yourself Forum" .

I do believe as well that many times hobbyists who are searching for the Holy Grail are buying gear based on something that often sounds different. As a result we find some good deals on second hand gear on Audiogon.

The short story however Mark is that if you can hear a difference it can be measured. I'm not an engineer or a physicist etc but I like to think myself educated. I did take physics for several years including sound as did you. Science is science. Listening does not trump science.

 

[microstrip]

Amir,
Can you explain this key sentence "His answer was that non-linear distortion is indeed impacted by training and audibility does vary therefore between people. "? Thanks.