The Absolute Sound’s Review Methodology: First Principles

[Holmz]

... I had the same result with the Mark Levinson No. 35 which received a rave review. It was nicely built but sounded dark and terribly sterile.

That “dark and sterile”, is sometimes not a bad thing.
(It would certain help to quell a sibilant and bright speaker.)

Unless one is using tube gear, then it is hard to get the frequency response too far out, as amps with feedback are renowned for having low distortion… and flat response from 20 to 20kHz.

Secondly the distortion products are responsible for a lot when amp is sounding sharp and loud, versus when they sound quiet and maybe some people call that dark??

A Jadis set of electronics running into those O/96 speakers look nice and sound good, but I would not refer to the ensemble as objectively low distortion.

 

[the sound of Tao]

Reading through this thread I conclude that the use of 'objective' and 'subjective' bring little insight. It is easy to say "it's just his opinion" or as at least one regular refers "it's just his subjective opinion". I wish someone would lay out the meaning of the phrase "objective audio review" as that pertains to sonic description.

They could be conflating objectively subjective with unbiased. It is a bit like aiming for an unbiased observation or opinion. Though I like the framing of the notion of observation rather than opinion as it leans into the aims of being a more objective subjective view. The weight in focus in terms of how much of what a reviewer expresses as observation and the amount they give over to their opinions in a piece is also telling.

Interpretation of observations and extractions of wisdom from the listening experience is also a potential value here. The more absolute they are in their opinions can reflect less awareness about some diversity in perception or preference in particular elements or experiences of sound.

The strong clues to the assessment approach are also in the format… the essay style report.

So if we look to what TAS publish it is a fairly typical audio review approach… a bimodal mix of subjective and objective testing. So predominantly subjective based assessments (the human interpretation of listening tests) with some supporting objective assessment criteria (typically a small panel of measurements and specs) for supporting technical analysis all built into an essay style report structure.

If it were a YouTube video report the 100% presenter grabs to camera would often typically be dominated by subjective info and any supers on screen as text or any cutaways to graphics or tight shots of the gear are likely to be supporting objective info such as measurements and specs or supporting visuals. The bimodal mix approach rather than unimodal one assessment style approach tends to work in the context of audio gear reviewing. The objective info or data should ideally correlate and support the subjective listening interpretation and vice versa.

 

[tima]

I was royally screwed 30 years ago when I bought all this Jadis equipment after reading a stereophile review. I had the same result with the Mark Levinson No. 35 which received a rave review. It was nicely built but sounded dark and terribly sterile.

Who is accountable for your purchases? Sounds like you screwed yourself.

 

[tima]

They could be conflating objectively subjective with unbiased. It is a bit like aiming for an unbiased observation or opinion. Though I like the framing of the notion of observation rather than opinion as it leans into the aims of being a more objective subjective view. The weight in focus in terms of how much of what a reviewer expresses as observation and the amount they give over to their opinions in a piece is also telling.

Mea culpa I don't understand. "Objective subjective"? "objectively subjective" "They"?

So predominantly subjective based assessments (the human interpretation of listening tests) with some supporting objective assessment criteria (typically a small panel of measurements and specs) for supporting technical analysis all built into an essay style report structure.

What are listening tests if not listening to sound and music by a person? Again I have a problem with 'subjective'. Am I correct understanding you to say that when sound/music is described by a person it is subjective and when sound/music is described by an ammeter it is objective?

 

[the sound of Tao]

Mea culpa I don't understand. "Objective subjective"? "objectively subjective" "They"?



What are listening tests if not listening to sound and music by a person? Again I have a problem with 'subjective'. Am I correct understanding you to say that when sound/music is described by a person it is subjective and when sound/music is described by an ammeter it is objective?

Perhaps this really comes back to whether human perception is seen as interpretation of experience or as exact measure and a recording of the facts. Some things are relatively concrete and easy to ascertain and distinguish even using human perception as the primary tool but some things are more abstract and complex and less absolute. If you say “what I observed” you are technically relating through subjective observation… if you are interpreting experiences you are subjectively doing so, and if you are expressing opinions in a qualitative framework you are engaging in even more subjectivity. If a reviewer chooses to see all these as things as purely objective that’s up to them.

To know beyond this for sure we’d need have to have a detailed working model of perception that could be approached as being framed as a law of perception. There would need to be an exact map and formula for human perception… so consciousness and perception would need to be singularly defined and confirmed through neuroscience. It isn’t. These things are beyond the remit of current science. Maybe consciousness is numinous and beyond exact specification.

As I said earlier I’m just working with the assessment theory we use in education generally for qualification frameworks for assessments.

I’ve given as many examples and explained it as best as I can. As I’ve also said if it’s not an issue of compliance it’s moot if there are no standards used in audio review.

In education we have qualification compliance so our assessment theory tends to have had a lot of work done on it and its all constantly peer reviewed… but even old humble scholars know we’re only approximating in making a call on the process and exactness in perceptual modes.

 

[Holmz]

…

What are listening tests if not listening to sound and music by a person? Again I have a problem with 'subjective'.

Yes to listen implies ears, and a brain, and everything else.

Am I correct understanding you to say that when sound/music is described by a person it is subjective and when sound/music is described by an ammeter it is objective?

An ammeter is possibly a bit of a straw man argument?
(But it would objectively quantify the current flowing in some DC speaker, like maybe a fan.)

However if we say the ammeter becomes a microphone, or a microphone on a Klippel machine.
Then with that we might objectively quantify the SPL, and the resulting frequency spectrum… and in the case of the Klippel we might objectively quantify the radiation pattern, as well as impulse response.

If we say that the bass sounds “tight” or “fast” those words are describing some subjective listening experience.
The objective corollary of “tight bass” would be the group delay, transcient response and impulse response.
With those measurements we can describe it in terms that are quantifiable, and not open to interpretation and clever wordsmithing.

 

[tima]

Yes to listen implies ears, and a brain, and everything else.


An ammeter is possibly a bit of a straw man argument?
(But it would objectively quantify the current flowing in some DC speaker, like maybe a fan.)

However if we say the ammeter becomes a microphone, or a microphone on a Klippel machine.
Then with that we might objectively quantify the SPL, and the resulting frequency spectrum… and in the case of the Klippel we might objectively quantify the radiation pattern, as well as impulse response.

If we say that the bass sounds “tight” or “fast” those words are describing some subjective listening experience.
The objective corollary of “tight bass” would be the group delay, transcient response and impulse response.
With those measurements we can describe it in terms that are quantifiable, and not open to interpretation and clever wordsmithing.

How is the meaning of these two sentences different?

"Then with that we might objectively quantify the SPL, and the resulting frequency spectrum… and in the case of the Klippel we might objectively quantify the radiation pattern, as well as impulse response."

"Then with that we might quantify the SPL, and the resulting frequency spectrum… and in the case of the Klippel we might quantify the radiation pattern, as well as impulse response."

 

[Holmz]

How is the meaning of these two sentences different?

"Then with that we might objectively quantify the SPL, and the resulting frequency spectrum… and in the case of the Klippel we might objectively quantify the radiation pattern, as well as impulse response."

"Then with that we might quantify the SPL, and the resulting frequency spectrum… and in the case of the Klippel we might quantify the radiation pattern, as well as impulse response."

You tell me.

Are you trying to just remove objective?
or
Are you trying to work towards sliding in subjective?
or
Something else?

If you are looking at some frequency response graph, that is more on the objective end.
And hearing something, that makes you reach for the frequency response graph would be the subjective side.

It is not an overly nuanced concept.

 

[Robh3606]

I'm really sorry but I'm afraid I don't understand the question here.

Hello Ron

Should have been more clear I was talking about a Stereophile review with listener impressions and measurement's. If there are issues with either it would call for increased scrutiny of the equipment under review.

Are you saying here that you are not agreeing with the TAS view that live music as a reference means you can make an objective observational assessment?

I don't have an issue with music as a reference just the whole "objective review" part of it. They have how many reviewers all using separate systems in different rooms all using their owns systems as the reference? Depending on who gets the to review X, there can be different outcomes.

The strength of measurements as an objective tool is the lack of bias and standardization. Using a system you picked based on your own bias as a reference defeats any claim of a bias free evaluation. IMHO

I personally don't care about measurements, but I often am curious to see them.

Ok I look at both give them about the same weight.


Rob

 

[Amir]

The TAS Staff published today a very interesting and thoughtful essay, styled as answers to questions about, among other things, objective versus subjective reviews, objective observations versus subjective feelings, the value of objective measurements of audio components and issues with double-blind testing.

The part distinguishing subjective reviews from "observational, objective reviews" interested me the most:

Question: Why do you do subjective reviews?

Answer: We don’t. Or for the most part we try not to make that the core of our reviewing. We aim to do observational, objective reviews. Now, there is some confusion about terminology in which “quantification” is “objective” whereas human “observation” is “subjective”. But this is wrong. That notion incorrectly glosses over a critical distinction. “Subjective” in the dictionary means human reactions that primarily involve feelings. But humans are also capable of observing objectively.

. . .

A simple example may help make some sense of this important distinction. If your car is parked next to your house and we ask “which is taller?” you will observe that your house is taller than your car. It isn’t that you feel your house is taller, it is that you are fully capable of objectively observing the height differences.

Question: You talk about objective observation as a superpower, but how is your opinion objective?

Answer: To use observation as a meaningful measurement technique, you must have a reference standard. This is the case with quantified measurements too, just as it is the case with objective observation. Comparing what we (or you) hear to a reference gets us out of the realm of opinion (subjective feelings). As we said above, it isn’t your opinion that your house is taller than your car. It is an observable fact. It is an observable fact whether a guitar sounds like a guitar, and if it doesn’t, to what degree and in what way.

In music audio, we use the sound of real music and real musical instruments (the absolute sound) as the reference standard. An audio system that can reproduce a guitar or a singer or a jazz band or a symphony so that it sounds believably real, will tend to be more satisfying for most listeners most of the time. This latter point is our experience from over 50 years of listening to live music and audio reproduction of music across hundreds of reviewers.

The Absolute Sound’s Review Methodology: First Principles — the absolute sound

Q:I’m looking for a reviewer that reviews audio components like an engineer would. While I’m not an engineer, I’m fascinated by technical details and measurements. Is there anyone out there that reviews audio components who not only describes what they hear but breaks down the components i.e.

apple.news

---------------------------------------------------------------------------------

We always -- certainly I always -- talk about subjective reviews. But is this an incorrect characterization?

Are TAS reviewers stating observational, objective sonic impressions rather than subjective sonic impressions? Are these observational, objective impressions "facts"?

Is the example of visually observing objectively that a building is taller than a car a valid analogy to reporting on the sound of audio components?

Is the sound of an acoustic guitar an objective reference? Does a reference of the absolute sound of an acoustic guitar allow an observation to rise above a mere subjective impression and become an observable, objective fact?

What do you think?

We should back to definitions, what I know is the definition of subjective experience is about human perception and it does not mean our perception is wrong. If you think your jean is blue then it is blue and the objective measurements prove it but your idea about the jean color is in category of subjective experience. The objective definition is about external world (out of human brain) so it is free from any human perception so in science we use objective method to be free from human perception and also have an independent reference.
I think we should not mix it.

Subjectivity and objectivity (philosophy) - Wikipedia

en.m.wikipedia.org

 

[andromedaaudio]

We should back to definitions, what I know is the definition of subjective experience is about human perception and it does not mean our perception is wrong. If you think your jean is blue then it is blue and the objective measurements prove it but your idea about the jean color is in category of subjective experience. The objective definition is about external world (out of human brain) so it is free from any human perception so in science we use objective method to be free from human perception and also have an independent reference.
I think we should not mix it.

Subjectivity and objectivity (philosophy) - Wikipedia

en.m.wikipedia.org

Fully agree , if there was an audiomagazine that called itself OP id probably start reading it
OP stands for our perception

 

[Eldar]

For some reason the sense of hearing seems to be downgraded to "subjective". The human sense of hearing can be objective just like our other senses. For example, I can evaluate how something looks. If I state something is blue is that subjective? I could even make finer observations about the color and say that it is light blue or Turquoise. I could say that it is luminescent or pearlescent. When I say these words most can immediately visualize what I mean by these observations.

The same thing can be applied to sound. One of the issues is lack of a common vocabulary. We all grew up with a box of crayons so we all generally agree what each color name represents. Unless someone was a musician, concepts of sound were not part of their education. IMO, we also lack what I consider a meaningful vocabulary. For example, what does "warm sounding" mean? Is this just a euphemism for time smearing? I believe HP tried to establish a vocabulary that people could relate to very early on. Words like imaging and soundstage. However, both of these elements are mostly a function of how well the speaker are matched and placed. One could say "It was possible to achieve very precise imaging with speaker A when I placed it in my room." This would tell me that the speaker manufacture is putting in effort to closely match the left and right speaker.

It doesn't seem that the body of reviewers are organized in any real way. They are simply employed by the media publisher to write about products. Like I said in the other post about reviewers, the first criteria is that they are good writers and not necessarily good listeners. We all assume they are, but... I read far fewer reviews these days but one trend I have noticed is that much more of the review is spent on describing the physical attributes of the product (size, shape, buttons, look and feel, etc.) and much less space dedicated to how it sounds. How it goes about making music.

I believe the color analogy is much more relevant than the house car height analogy. I am an audiofile, photographer and wine lover. All three areas suffer from the challenges related to subjectivity vs. objectivity. If you put an inexperienced person in front of multiple hifi systems or multiple pictures and serve a series of wines, you will get nothing but very subjective and uneducated assessments. Then you can go to an experienced person in any of the areas, who has developed preferences, standards, language etc. and, due to having seen, listened to or tasted many alternatives, and you will get a much more qualified view. However, it is far from being objective. In all three areas we are talking subtleties, details, balance etc. etc. The house car analogy is more like sound; mono vs. stereo, colours; green vs. blue or wine; red vs. white.

The color analogy; You observe two photographers photograph the same scenes and subjects. They use different camera systems, but (almost) the same angles and the same light. A week later you are in your favourite chair, with fixed lighting and look at series of prints from this shoot. One photographer on Monday and one on Thursday. You assess composition, exposure, focus, depth of field, focal length, colours, tones, saturation, contrast, shadow detail, balance, texture etc. on the images as they are and also up against what it looked like during the shoot (the absolute vision). In my view a very relevant challenge to being in front of a hifi system. An experienced person, who does this for a living, will be more qualified and closer to (his/her) objectivity. However, having done this with both audio, photographs and wine over the last 40-50 years, and having observed myriads of others doing the same, it is not truly objective.

In all three areas, on top of personal preferences, body battery, blood sugar, stress level, mental balance and all the rest of it, will influence your view. Blindfolded you would describe the same system differently, if you review them a month apart. What is the absolute sound? Carnegie Hall, Sydney Opera House, Royal Festival Hall, Shoreline Amphitheather or Ronnie Scotts? The same instrument will sound different in any of these.

I have a selected few reviewers I read, because I over time have learned to understand them. I understand their language and I understand their preferences, because I have read many and I have listened to many of the components they review. I do the same with wine. I do not consider them objective though, but I consider them consistent and thorough.

 

[Lycurgus]

For some reason the sense of hearing seems to be downgraded to "subjective". The human sense of hearing can be objective just like our other senses. For example, I can evaluate how something looks. If I state something is blue is that subjective? I could even make finer observations about the color and say that it is light blue or Turquoise. I could say that it is luminescent or pearlescent. When I say these words most can immediately visualize what I mean by these observations.

Is the dress blue/black or white/gold...

Even the strawman argument about the height of the car/house, which is an obvious over simplification, could be negated by changing the viewer's perspective until the car "appeared" higher than the house.

Human perceptions are subjective because our senses are not calibrated to a standard reference. You can say one speaker is louder than the other, but you would not be able to say by how much with any accuracy. Senses are also subject to a wide variety of outside influences (mood, fatigue, alcohol consumption, state of mind) that can render their "objectivity" as useless.

If someone says "I can hear a difference between these cables" there is no way I can dispute that statement, even if I cannot hear a difference. That listener's personal experience is unavailable to me. The statement however is subjective and not objective.

Measurements using calibrated equipment, circumvents most of these issues. It doesn't mean measurements are better than listening tests, it just means they are more useful for making objective comparisons that are quantifiable. Listening tests will always be subjective except in extremely obvious cases.

 

[Stevet]

Can't we all agree that these reviews are just a part of the purchase/decision making equation and that the weight we give the review depends on many factors, but that the most important part of the purchase/decision making equation is to educate one's self and take the time to carefully listen to those pieces you are considering for purchase in an environment as close to the one you have as possible. Anyone who is purchasing solely off of a review, without more, is really taking a chance.