A New Way to Evaluate Audio Systems

[Ron Resnick]

Paul James published today an interesting article proposing a new way to evaluate audio systems:

A New Way to Evaluate Audio Systems

Dedicated music lovers would like to identify and use the best quality audio equipment. That way they could, potentially, have a richer musical experience. Yet, as Norwitz and Qvortrup identify in Are You On The Road To... Audio Hell?,” none of the methods frequently used to evaluate systems...

www.resistormag.com

He proposes using an original vocals track and a cover of that track by a different artist, and evaluating on audio systems based on the extent to which they maximize the perceived sonic differences between the two recordings.

What do you think about this idea?

 

[Mike Lavigne]

misguided. too simplistic and one dimensional. on it's own as a part of the picture, sure. but only a small part. not that it's wrong, just incomplete. for me. and maybe my listening skills are not good enough. i need more time and more examples to have a better feel.

when i make a change and am judging whether it's the right thing to do for a system presentation change, i will listen to 75-100 vocal cuts a few times over a couple of weeks. and many other types of music and formats and in various states of mind.

OTOH if you have a history of using a particular tool and it works and are satisfied with your system development progress, knock yourself out. just not my way. and my system expectations include high energy and large scale music, which this method seems to ignore.

i do respect that for particular set-up questions, that sometimes how a particular recording sounds can be helpful. such as Romy's DPoLS for speaker positioning.

judging individual components, or cartridge set-up, are slightly different cases.

 

[PeterA]

In my opinion, vocals are not enough to evaluate the quality of an audio system. I like listening to my system and then going to hear a symphony or string quartet or jazz performance, and then listening to that same type of music on my system again. It’s pretty easy to judge whether or not it sounds natural.

For me, the guide and reference is live music not another recording, especially an original and then a cover. I have a bunch of different performances of “nature boy“. Different instruments and different singers. They all have their qualities.

 

[wil]

This is presuming that vocal covers only have subtle differences from the original or one another. What does the author think, that cover artists are just trying to produce a copy of the original? If that were so, why do a cover in the first place?

 

[PeterA]

I just read the article. Differences are inadequate to assess the quality of an audio system. Information retrieval does not tell you how realistically a performance is being presented. For that, one must have a reference in his mind against which to judge.

Ron, I think the proposed method is flawed.

 

[tima]

Paul James published today an interesting article proposing a new way to evaluate audio systems:

A New Way to Evaluate Audio Systems

Dedicated music lovers would like to identify and use the best quality audio equipment. That way they could, potentially, have a richer musical experience. Yet, as Norwitz and Qvortrup identify in Are You On The Road To... Audio Hell?,” none of the methods frequently used to evaluate systems...

www.resistormag.com

He proposes using an original vocals track and a cover of that track by a different artist, and evaluating on audio systems based on the extent to which they maximize the perceived sonic differences between the two recordings.

What do you think about this idea?

I think its silly. Not enough range in the human voice.

 

[LL21]

Akin to evaluating a tennis player by watching him just serve and volley. Nothing like watching a player in a live, complete, competition game.

 

[Kingrex]

We need someone to do a rewrite of the artice, then we compare the two to determine the validity of the concept.

 

[gerihifi]

I think its silly. Not enough range in the human voice.

100% agree and through the difference of speaker designs it will more obvious of sound signature change in certain systems, but not as a generic approach.

I'm always comapring changes over several weeks (personal mood, like afternoon or evening listening, as well different genres to get larger frequency range). Of course I've my distinct reference songs and try to bring in 2-3 new titles, to get unbiased impressions.

My 2cents, Gerald

 

[Hedalfa]

misguided. too simplistic and one dimensional. on it's own as a part of the picture, sure. but only a small part. not that it's wrong, just incomplete. for me. and maybe my listening skills are not good enough. i need more time and more examples to have a better feel.

when i make a change and am judging whether it's the right thing to do for a system presentation change, i will listen to 75-100 vocal cuts a few times over a couple of weeks. and many other types of music and formats and in various states of mind.

OTOH if you have a history of using a particular tool and it works and are satisfied with your system development progress, knock yourself out. just not my way. and my system expectations include high energy and large scale music, which this method seems to ignore.

i do respect that for particular set-up questions, that sometimes how a particular recording sounds can be helpful. such as Romy's DPoLS for speaker positioning.

judging individual components, or cartridge set-up, are slightly different cases.

Mike

An unfortunate and inflammatory choice of words, particularly when the approach hasn’t been tried. This response is a choice example of why I am reluctant to bother with Forums. The article never advocated replacing all existing approaches rather like having another tool in the toolbox, it might also be useful. It does have some strengths.

As the article also highlights it’s a different approach and it will need some willingness to make some effort, and a willingness to embark on something unfamiliar. It may also have the benefit of enhancing people’s listening capabilities, which can be helpful.

However, if people have a closed mind, and don’t want to try, it’s a free country, that’s perfectly fine.

 

[Hedalfa]

In my opinion, vocals are not enough to evaluate the quality of an audio system. I like listening to my system and then going to hear a symphony or string quartet or jazz performance, and then listening to that same type of music on my system again. It’s pretty easy to judge whether or not it sounds natural.

For me, the guide and reference is live music not another recording, especially an original and then a cover. I have a bunch of different performances of “nature boy“. Different instruments and different singers. They all have their qualities.

This approach can have merit. Yet one of the problems is most people have a poor sonic memory they can remember how something sounded for very long. Sure some people can hold the sound there memory, though I think youll find it is actually a rare capability.

 

[Hedalfa]

I think its silly. Not enough range in the human voice.

With the caveat that the human voice has been well-recorded, qualities of the reproduction that are not noticed by using other materials can be revealed.

 

[Holmz]

Paul James published today an interesting article proposing a new way to evaluate audio systems:

A New Way to Evaluate Audio Systems

Dedicated music lovers would like to identify and use the best quality audio equipment. That way they could, potentially, have a richer musical experience. Yet, as Norwitz and Qvortrup identify in Are You On The Road To... Audio Hell?,” none of the methods frequently used to evaluate systems...

www.resistormag.com

He proposes using an original vocals track and a cover of that track by a different artist, and evaluating on audio systems based on the extent to which they maximize the perceived sonic differences between the two recordings.

What do you think about this idea?

I was surprised that they have the balls to ask $5 to subscribe to that crap?

I think its silly. Not enough range in the human voice.

On the other hand the human voice is one thing that we have listened to since before birth.
If that does not sound very convincing then it is doubtful that other sounds will also be convincing.

Since a lot of what I like to listen to is vocal, then the voice being accurate is most important to me.

 

[denydog]

I think I agree with Wil's thinking.

Maybe I don't get it. A cover of a song is not meant to mimic the original, so it's bound to sound different in all attributes as far as I know. I find it hard to imagine any system that would come close to not having obvious differences with the original recording vs a cover.

 

[ssfas]

It's not just silly, it's laughable. I looked up a great track from a favourite album by Ute Lemper and the next one above was by Nick Cave! Listened to them both. Love them both. What did they teach me about audio? - NOTHING!


It actually get it wrong in the first paragraph: "none of the methods frequently used to evaluate systems, measurements and specifications, blind tests, double-blind tests, boogie factors, or comparisons to real music, have been definitive." Well, of course none of them are definitive, but it's a good list of the things we can use at different times to make good hifi choices.

 

[treitz3]

It's laughable.

Agreed. A well known audio designer I personally know does the same thing (although, not his only test). I never really "got it".

Me thinks Mike L. got it correct on the first response. It's misguided. Too simplistic and one dimensional. On its own as a part of the picture, sure. but only a small part.

While vocals and the delicate mid range are extremely important in a system, it is all but a small part of what a stereo reproductive effort is, or can be actually capable of. Like a snapshot of a particular time in history. You aren't presented with the whole story.

Tom

 

[Argonaut]

I think its silly. Not enough range in the human voice.

How much range do you want !

• Frequency range of a human voice
  The human voice can range from 20 Hz to 20 kHz, which is the audible frequency spectrum for humans.
• Frequency range of different voice types
  The frequency range of different voice types in classical music are:
  • Soprano: The highest female voice, typically between C4 and C6
  • Mezzo-soprano: The middle female voice, typically between A3 and A5
  • Contralto: The lowest female voice, typically between F3 and F5
  • Tenor: The highest male voice, typically between C3 and C5
  • Baritone: The middle male voice, typically between A2 and A4
  • Bass: The lowest male voice, typically between E2 and E4
• Highest pitch a human can sing
  Brazilian soprano Georgia Brown holds the Guinness World Record for hitting a G in the high 10th octave, which translates to about 25,000 hertz

 

[Mike Lavigne]

Mike

An unfortunate and inflammatory choice of words, particularly when the approach hasn’t been tried. This response is a choice example of why I am reluctant to bother with Forums. The article never advocated replacing all existing approaches rather like having another tool in the toolbox, it might also be useful. It does have some strengths.

As the article also highlights it’s a different approach and it will need some willingness to make some effort, and a willingness to embark on something unfamiliar. It may also have the benefit of enhancing people’s listening capabilities, which can be helpful.

However, if people have a closed mind, and don’t want to try, it’s a free country, that’s perfectly fine.

i said it's not for me. i need more.

i've tried quick hits. many, many, many times. with all sorts of music, in all sorts of ways. it does not get the job done. skims the surface. whatever is learned is hard to determine. i don't have faith in it.

maybe i should not have used the word 'misguided'? maybe.

and Ron's thread title put lots of weight on the method, more than he should have. the thread title should have said 'a tool' to evaluate, not a 'new way' as if it was something comprehensive. it's clearly not that. it's the thread title that was misguided.

 

[Argonaut]

This approach can have merit. Yet one of the problems is most people have a poor sonic memory they can remember how something sounded for very long. Sure some people can hold the sound there memory, though I think youll find it is actually a rare capability.

I think that you meant to write cannot ? And would agree for the most part , In that whilst one may feel an immediate sense of whether a recording of a performance replayed on a particular system sounds Organic , Instinctive , believable , essential , etcetera … using the fallibility of the human memory of a live event , even as recent as a couple of days hence , as one’s only methodology of a systems capability can have its own set of flaws .

 

[treitz3]

Off the beaten path question here - how often would one hear a voice like Geoff Castellucci, Tim Storns or Joe Thompson? Mariah Carey, Jessye Norman or Margaret Price in recordings? These are the extreme examples. Many singers do not have these kinds of octave extensions.

Here is my question. If one were to evaluate a system based upon voices only....why would you do so, if only less than 2% of singers you normally listen too can even hit said octaves?

Tom