Is Testing the Real Snake Oil? Roger Skoff Debunks the Debunkers Roger Skoff Debunks the Debunkers - Positive Feedback
Is Testing the Real Snake Oil? Roger Skoff Debunks the Debunkers
positive-feedback.com
Roger Skoff on Testing from PFO - FYI
- Thread starter Cellcbern
- Start date
positive-feedback.com
It’s a slow news day, and the Olympic events being broadcast today are not yet interesting enough to keep me glued to the screen. So I read the Roger Skoff piece. I can’t say I felt particularly enlightened. Of course measurements are not the whole story when it comes to evaluating audio gear. Didn’t we learn a long time ago that tube gear generally measures inferior to SS gear (i.e. THD, freq response, etc) but this has little or no relevance for sonic preference? Measurements might in fact be important variables that correlate to sonic preferences, but as many have pointed out over the years, we may not yet know exactly what to measure in that regard, so the entire measurement argument may lack validation until we understand the correlation of yet unknown data measurements to sonic perception.
I also took exception to his point on A/B comparisons. Of course music is a complex signal. But that hardly makes A/B comparisons irrelevant. In fact, the single variable that is being studied in A/B experiments is not the material used or the gear used in playback, but rather a sonic percept. Either you like something more, less or the same. One’s sonic preference is a perfectly valid variable to assess. The problems with A/B comparisons, as legions of thoughtful pundits have pointed out previously, is that audio memory lasts about 7 seconds, which is why many reviewers value long term listening more than short-term A/B comparisons using music as the source material.
Bottom line- IMHO, Roger’s essay was hardly a treatise of great insight as there was little there that hasn’t been said better or more wisely by others over the past several decades.
I also took exception to his point on A/B comparisons. Of course music is a complex signal. But that hardly makes A/B comparisons irrelevant. In fact, the single variable that is being studied in A/B experiments is not the material used or the gear used in playback, but rather a sonic percept. Either you like something more, less or the same. One’s sonic preference is a perfectly valid variable to assess. The problems with A/B comparisons, as legions of thoughtful pundits have pointed out previously, is that audio memory lasts about 7 seconds, which is why many reviewers value long term listening more than short-term A/B comparisons using music as the source material.
Bottom line- IMHO, Roger’s essay was hardly a treatise of great insight as there was little there that hasn’t been said better or more wisely by others over the past several decades.
If a speaker sounds good and measures poorly that is excellent information! Learning what doesn't matter is just as important as learning what does matter. There's no chance of learning if we don't try to measure and compare the measurements to our perceptions of audio quality. Short term vs. long term impressions make it more difficult. I remember my dad talking about supermarket testing where the sweetened fruit juices and applesauce would always be preferred in a quick direct comparison blind taste test. Over time though I find the experience is more enjoyable with the unsweetened varieties.
Tim
After playing with speakers for a long time I believe that if you work on that speaker to get it measure well it will invariably sound better
There are so many decisions to make in a design the possibilities are virtually endless there seems to me there is always a path to good measurements and great sound ... but there are many forks in the road !
Is it not often the case that we buy on short term impressions (and reviews/hype) and later sell on long-term impressions?
As the consumer, all that matters to me is what the result sounds like to me in my room. The designer, on the other hand, understands the usefulness of measurements.
I think Vladimir Lamm did a series of subjective listening studies Involving many people and then tried to correlate the results to measurements. These studies form the basis of the designs of his equipment. Others may know more details about this.
As the consumer, all that matters to me is what the result sounds like to me in my room. The designer, on the other hand, understands the usefulness of measurements.
I think Vladimir Lamm did a series of subjective listening studies Involving many people and then tried to correlate the results to measurements. These studies form the basis of the designs of his equipment. Others may know more details about this.
Last edited:
I totally agree. By measuring and correcting speakers I can make them sound better to my ears - assuming they need some adjustment. When I build my own speakers they always definitely need adjustment.
It would be interesting to know more about the metrics he developed, although I could see why he might not be too eager to share it all. I agree that end users should not be expected to make a bunch of measurements as that is primarily the job of the designers. I find something like REW to be fairly useful for seeing what's happening in a room, but could also totally understand why someone might not want to mess with it. If you're thoroughly enjoying the sound you have to be either very curious or very worried that you're missing out on some potential to start taking a bunch of measurements.
"As Chief Design Engineer of Research and Development at the Lvov Radio & Electronics factory (in the Soviet Union) Lamm had both the resources and large pools of test subjects for conducting hundreds of blind and double-blind listening experiments. From these he accumulated massive amounts of data about what happens when people hear certain sounds, including a complex sound like music. With data in hand he used differential equations to develop scientific models that described mathematically what he calls "the human hearing mechanism." He converted those equations into electro-mechanical models and implemented them in specific circuit topologies.
Lamm tested his circuit designs with hundreds of human listening subjects to demonstrate that, given human physiology, only a few combinations of audio circuitry will work for us as listeners. We cannot change how we perceive sound or music, even in the face of what passes for good specs. "As humans," Lamm observes, "we are created in a certain way. We perceive sound on various levels: conscious as well as subconscious or intuitive. We perceive sound not just with our ears, but with the whole body." From his research he developed a set of theoretical ideals against which he evaluates any amplifier. He called these constructs the Absolute Linearity of a System -- a sort of unified field theory of amplifier design that explains how an amplifier should measure if it is to reproduce sound congruent with the way people naturally perceive it. Without going into detail about the specific measurements Lamm uses, the basic high-level idea is this: as gain is applied the amplifier should preserve the harmonic structure and spectral balance of the musical source signal. Lamm’s evaluation criteria also places specific emphasis on the types and values of feedback utilized in an amplifier." Ref
I found on another forum some further description of his idea, which is that harmonic distortion should be the same across the entire audible bandwidth, and rising steadily with gain to a maximum of about 3%. I've seen recent measurements that demonstrate the 2nd, 3rd, 5th etc. order distortions compared to frequency at different gain levels and into different loads, with the presumption that they should all ideally stay proportionally similar in all circumstances. Typically class D amplifiers aren't as good at it but the distortion is all super low on the good ones. I find that if distortion is reasonably low it just doesn't matter to me. It doesn't bother me as much as I expect it to at something like 1 or 2% but it doesn't help anything either. My brain interprets it as noise. The irritating colorations which I imagine to be caused by even or uneven order distortion usually turn out to be uneven frequency response, diffraction effects, room reflection effects - really major issues that are readily measured.
Yes, that is my understanding as well. I had not seen the 3% number and I'd like to read more - would you please offer a link. Your second sentence observation is line with Lamm's approach.
His theory of the Absolute Linearity of a System (ALS) looks at Total Harmonic Distortion +Noise versus Power and versus Frequency. When looking at THD vs Power graphed for various frequencies, the curve is the same for each frequency. Likewise when he looks at THD vs Frequency at different Power the graph is flat for each level of power. This (in part) describes his notion of the 'ideal amplifier' against which any amplifier can be assessed. What is fascinating about his empirical research is finding that only certain circuit topologies can meet this ideal. What is genius is recognizing that audio equipment should be designed for the way humans hear - that doesn't change.
There are some some points worth considering but I can not take this article seriously as a whole. The author does not seem to have a good understanding of physics of sound. There is wrong information and fallacies in this article.
For example and I quote from the article "Even despite measured proof that a speaker is capable of response down to 20 Hz or even lower, if a room is smaller than one half the wave length of the frequency to be reproduced (for a 20Hz tone, that would be about 27 feet [8.23 meters]), that frequency simply cannot be propagated in it, and to buy speakers that go that deep for that room would simply be a waste of money."
This is a common misconception and I get why it happens. Sound waves are mostly represented 2D and as we know they have certain cycle lengths we make an association. However sound waves are positive and negative pressure pulses. If this was not the case, you would not be able to hear bass frequencies on a headphone. You don't need a 8.6meters* (344m/s divided by 20hz) or as claimed at least half wavelength of 20hz, 4.3meters deep headphones do you? Because unlike a large room, your ear canals or headphone cups have very little space and you need to move a much smaller volume of air. Therefore even a very small transducer is able to move the amount of air needed to be moved and create the desired frequency response.
The problem with bass in a room is there but the problem is of resonance. The smaller the room the more the resonances there will be as there will be more influence of the room and it will cause many dips and peaks. This is also the reason that if you are planning on applying acoustic treatments, it is more logical to start with low frequency treatments.
There are more problems and arguments that are misguided but I will stop here. I see someone who "believed" in measurements which was a wrong thing to do to start with. These are figures for you to get a more balanced and objective understanding. If you do not know how they derived these figures and what they represent, what other figures you need to get a more complete and meaningful picture, you will just end up like kids playing trumps.
*the correct figures
For example and I quote from the article "Even despite measured proof that a speaker is capable of response down to 20 Hz or even lower, if a room is smaller than one half the wave length of the frequency to be reproduced (for a 20Hz tone, that would be about 27 feet [8.23 meters]), that frequency simply cannot be propagated in it, and to buy speakers that go that deep for that room would simply be a waste of money."
This is a common misconception and I get why it happens. Sound waves are mostly represented 2D and as we know they have certain cycle lengths we make an association. However sound waves are positive and negative pressure pulses. If this was not the case, you would not be able to hear bass frequencies on a headphone. You don't need a 8.6meters* (344m/s divided by 20hz) or as claimed at least half wavelength of 20hz, 4.3meters deep headphones do you? Because unlike a large room, your ear canals or headphone cups have very little space and you need to move a much smaller volume of air. Therefore even a very small transducer is able to move the amount of air needed to be moved and create the desired frequency response.
The problem with bass in a room is there but the problem is of resonance. The smaller the room the more the resonances there will be as there will be more influence of the room and it will cause many dips and peaks. This is also the reason that if you are planning on applying acoustic treatments, it is more logical to start with low frequency treatments.
There are more problems and arguments that are misguided but I will stop here. I see someone who "believed" in measurements which was a wrong thing to do to start with. These are figures for you to get a more balanced and objective understanding. If you do not know how they derived these figures and what they represent, what other figures you need to get a more complete and meaningful picture, you will just end up like kids playing trumps.
*the correct figures
I found some of this stuff on ASR, searching for Lamm. As expected there are some highly skeptical perspectives on there. Also on ASR there are charts showing distortion at different frequencies vs power level for a variety of amps. They just show total distortion, not breaking it up into types of distortion as I at first thought. One thing I learned on there is that diverging distortion with frequency is a problem that class D chip amps tend to have more than other amps, although it can remain very low across the board. I'm not sure I can really hear it at all down that low based on some tests that demonstrate distortion at various levels. I was surprised how high it had to be before it became readily apparent to me. And what it sounded like was not nearly as unpleasant as I had expected, even the odd ordered stuff. It just sounds like noise and mild break up to me. More disappointing than painful. What bothers my ear most is peaky resonances. What I don't know about the tests I listened to is if the distortion was artificially applied in a way that spread it evenly among frequencies or if it was more frequency dependent. That would be an interesting comparison to try. If there was a way to apply a small amount of appropriate distortion that made just about everything more enjoyable to listen to... maybe that's what tubes do, at least for some people?
Similar threads
 | Steve Williams Site Founder | Site Owner | Administrator |  | Ron Resnick Site Co-Owner | Administrator |  | Julian (The Fixer) Website Build | Marketing Managersing |