Comparative Listening Tests

[PeterA]

It must be my poor communication skills. I'll try once more.

If the pseudo science types tell us emphatically that unless I follow their advice I'm lost with regard to high-end audio, I have problem with such dogmatism. If other types tell us they don't pay much attention to those who don't listen to live music, I fail to see any real difference between that dogmatic stance and that of the pseudo science type.

If there was any validity to your guy's comment, then why shouldn't anybody interpret that to say, those who frequent live performances the most have the most validity regardless of their ability to interpret what they hear when listening to reproduced music? Then to take your point a bit further toward its potential, doesn't that also imply the guy who attended 75 live performances last year obviously knows so much more about reproduced music than the guy who only attended 48 live performance?

IOW, if your guy's comment had any validity whatsoever, then the one who attended 75 live performances has every right to say, he won't pay any attention (regardling reproduced music) to the guy who only attended 48 live performances.

IMO, that's another flavor of equally dogmatic hogwash. Especially when there are those in this forum who frequently attend live performances and have little or no idea what quality of sound they're lsitening to on a playback system. But perhaps it's only equally dogmatic hogwash to me.

If I could only choose between a pseudo science type's dogmatic stance of high-end audio and this dogmatic stance of high-end audio, I'd probably choose the pseudo science type's because I can pretty much reject it in it's entirety and move on. Whereas with this, here I am already having to drill down and disect a less obvious but IMO an equally dogmatic stance of another flavor.

Stehno, I think it is really pretty simple and I am sorry that you seem disturbed by my friend's view and think it is dogmatic. My friend was basically telling me that if someone is attempting to describe the quality of audio equipment or pass a quality judgement on a system, and he has never listened to live music, amplified, or unamplified, then my friend does not pay "MUCH" attention to the guy's opinion. I presume it is because he lacks experience with live music. This is not the same as claiming that one who listens to a lot of live music has the ability, knowledge and experience to judge the quality of an audio system or component, but that at least he has some experience with how a real violin, guitar, drum or voice sounds and can therefore refer to his memory of that sound when making a judgement about the sound of a particular component or system. Though it is certainly no guarantee of his ability to judge the quality of a system, he at least has some starting point, or reference, that the guy who has never heard live music does not have.

I just think that someone who has heard the sound of a live piano is more able to judge if the sound of a system resembles the sound of a live piano more than someone who has never heard that live instrument. Do you really think that that is a dogmatic position and harmful to the cause of increasing knowledge about audio reproduction?

 

[stehno]

Stehno, I think it is really pretty simple and I am sorry that you seem disturbed by my friend's view and think it is dogmatic. My friend was basically telling me that if someone is attempting to describe the quality of audio equipment or pass a quality judgement on a system, and he has never listened to live music, amplified, or unamplified, then my friend does not pay "MUCH" attention to the guy's opinion. I presume it is because he lacks experience with live music. This is not the same as claiming that one who listens to a lot of live music has the ability, knowledge and experience to judge the quality of an audio system or component, but that at least he has some experience with how a real violin, guitar, drum or voice sounds and can therefore refer to his memory of that sound when making a judgement about the sound of a particular component or system. Though it is certainly no guarantee of his ability to judge the quality of a system, he at least has some starting point, or reference, that the guy who has never heard live music does not have.

I just think that someone who has heard the sound of a live piano is more able to judge if the sound of a system resembles the sound of a live piano more than someone who has never heard that live instrument. Do you really think that that is a dogmatic position and harmful to the cause of increasing knowledge about audio reproduction?

Thanks for the explanation, Peter. I kinda' hoped you originally gave the cliff notes version of your friend's comment because there are those who make such blanket statements without any qualifications. And without any context surriounding the original comment as you just did here, it can take on a whole nuther hard line perspective. And some people will no doubt take that to the bank. Hence, I felt compelled to respond.

 

[DaveyF]

I have an interesting question....how many of you have ever tried to tune an instrument? I ask this question, as it relates to the fact that listening and trying to recognize what is the correct note while tuning an instrument is no easy thing! Yet many of us have probably done this on many occasions. Personally, I believe that the sound that we are "expecting" our audio gear to produce has actually very little to do with the Real! So much so, that I think any musician...if he or she listens to our systems, is going to state that there is a chasm of sound that is somehow missing...and then that same person is going to say how wonderful many of our systems sound!! So, the question is what's missing?
Which brings me back to my original question...and a suggestion that even if you have never played an instrument, or for that matter tried to tune one, a great hearing test is to do exactly that. IMHO.

 

[stehno]

I have an interesting question....how many of you have ever tried to tune an instrument? I ask this question, as it relates to the fact that listening and trying to recognize what is the correct note while tuning an instrument is no easy thing! Yet many of us have probably done this on many occasions. Personally, I believe that the sound that we are "expecting" our audio gear to produce has actually very little to do with the Real! So much so, that I think any musician...if he or she listens to our systems, is going to state that there is a chasm of sound that is somehow missing...and then that same person is going to say how wonderful many of our systems sound!! So, the question is what's missing?
Which brings me back to my original question...and a suggestion that even if you have never played an instrument, or for that matter tried to tune one, a great hearing test is to do exactly that. IMHO.

Good point, DaveyF. Except for the one about musicians. There are some musicians who refuse to play around with high-end audio because they've heard enough playback systems and are already convinced reproduced sound can never sound anything like the live performance.

But there's the other side of the coin where some musicians (maybe it's just a general love for music) who are convinced all good looking systems sound just like live music.

In fact, there's a little known secret in the industry where if you want a good cheap endorsement for a product, get a musician or a conductor. And I've seen enough advertising endorsements by musicians and conductors claiming a given product makes the reproduced music indistinguishable from the live performance. YG Acoustics had such a conductor make a claim just like that for a time.

Then there's all the musicians in between.

So IMO it's no guarantee with musicians either I guess is what I'm trying to say.

---------------------------------------------------------------------
Just reread your post. I misunderstood your one point.

Yes, in the case of the musician flip flop, I suspect some musicians are torn between what they initially realize is inaudible or otherwise falls short of the mark in our playback systems (which is much) but their love or passion for music eventually overcomes those shortcomings which are soon forgotten as they engage in the music presentation itself with a little time. I suspect they actually instinctively use their imagination a bit to make up for, overlook, or compensate for the playback system's shortcomings.

 

[KlausR.]

But there's absolutely no explanation for it.

Not correct: there is no explanation that I can provide, or an explanation you can think of. This does not mean that there is no explanation at all!

Unless you can tell me a reason the cable will change SPLs Occam's Razor favors my theory, in which there was no change in SPL.

Take Franck Tchang's little cups of various precious metals: the resonance frequencies of those cups have been determined and for some metals were well above 20 kHz, close to 30 kHz or even above (can't remember exactly), no output at the other frequencies. You agree that for CD these frequencies cannot be excited, yet the various reviews were positive about the effects. So, unless you can tell me a reason why a source that does not contain the necessary frequencies is capable of exciting these cups, the perceived effects invariably have to be imaginary.

 

[Gregadd]

Guessing involves a true random choice. That would make each possible outcome equally likely. Thus there would be no outcome that suggest guessing. Because 7nder a true random choice each outcome is equally likely.
Of course the outcome can be skewed when rinks he knows the answer but is incorrect.

 

[DaveC]

Not correct: there is no explanation that I can provide, or an explanation you can think of. This does not mean that there is no explanation at all!



Take Franck Tchang's little cups of various precious metals: the resonance frequencies of those cups have been determined and for some metals were well above 20 kHz, close to 30 kHz or even above (can't remember exactly), no output at the other frequencies. You agree that for CD these frequencies cannot be excited, yet the various reviews were positive about the effects. So, unless you can tell me a reason why a source that does not contain the necessary frequencies is capable of exciting these cups, the perceived effects invariably have to be imaginary.

Lol, really? I think this has been enough, thanks for playing. Your last arguments are ones you'd never accept yourself. You can't admit you heard a difference and had a preference even when... alright nevermind. You can lead a horse to water...

 

[amirm]

Guessing involves a true random choice. That would make each possible outcome equally likely. Thus there would be no outcome that suggest guessing. Because 7nder a true random choice each outcome is equally likely.
Of course the outcome can be skewed when rinks he knows the answer but is incorrect.

In blind testing we treat the person taking a test as a black box. We don't know what they thought. They may be guessing, randomly selecting, or thinking they know 100% and vote that way.

As with other research, an established method is used to see if the responses have high confidence to be usable. After all, we don't want to include the results of someone who for example got bored and frustrated and just voted A or B for the sake of it. This is where the so called 95% confidence or p<.05 came from. See this article I wrote on statistics of ABX testing.

Statistical analysis must be performed to determine the actual confidence in the results. Too often, as I explain in the article, people use the percentage right answers as the metric but that is a mistake. Even a number like 55% right answers given the right number of trials can indicate 95% or better confidence in the results. Yet lay people often call that as good as "chance" since it is close to 50% toss of a coin.

Personally, I like to target virtually 100% right answers. Then I know for sure I can distinctly recognize one sample versus another.

 

[amirm]

The problem with blind testing is you habe to identify characteristics that you then look for in the blind test. As our friend Don pointed out once you identify a trait you are likely to hear it in both A &B.

That's a problem with our hearing system, not blind testing as I explained. Our short-term memory has immense data rate but very little storage capacity. Can you imagine remembering the nuances of every piece of music you have ever heard? Every conversation you have heard (including its frequency response, noise, etc?). That is incredible amount of data to store. For this reason the short-term memory capacity is measured in seconds. It is that short.

A massively lossy filter throws out all but a fraction of what is in short-term memory and commits the outcome to long term memory. This memory can contain years of information but of course is devoid of mass amount of detail. Normally this process is automatic and the perceptual part of our brain determines what to keep, what not to keep.

What happens in audio evaluation -- blind or sighted -- is that we change what is kept and remembered because we are focused on it. So we swap out a cable then concentrate to hear what is different. Now we hear things and keep them in long term memory that previously was discarded. Not realizing this we attribute the newly discovered detail, air, etc. to the cable, not the fact that we modified how our perception works.

Going back to the other sample as you say can reveal the same difference being there, or not. It all depends on whether you attempt to direct your brain the same way or not. As my test showed earlier, this process is not constant. Playing the same file over and over again results in different outcomes even with similar frame of mind.

 

[Gregadd]

Personally I think consistency is preferable to a "high score." Of course a repeatable high score would be ideal.
Assuming the testee is honest he can tell us what he was thinking.

"

 

[Gregadd]

The problem with most blind tests is that there is no prior attempt to assure there is a trait present in sample A that is missing in sample B. This is specially true in an ABX challenge,where the person designing the test is trying to prove there is no differemce.

 

[amirm]

Personally I think consistency is preferable to a "high score." Of course a repeatable high score would be ideal.
Assuming the testee is honest he can tell us what he was thinking.

"

Agreed.

 

[amirm]

The problem with most blind tests is that there is no prior attempt to assure there is a trait present in sample A that is missing in sample B. This is specially true in an ABX challenge,where the person designing the test is trying to prove there is no differemce.

There are many such tests with provable, objective, measureable difference. Problem is that vast majority of audiophiles can't pass such tests yet believe in easily hearing differences that are not measurable or provable to be there!

 

[jkeny]

Who started up the merry-go-round again? Didn't you know this was behind crime scene tape? Pull the power, quick before it comes off its tracks & creates more genocide

 

[amirm]

I just think that someone who has heard the sound of a live piano is more able to judge if the sound of a system resembles the sound of a live piano more than someone who has never heard that live instrument.

At Axpona there was a grand piano in the lobby of the hotel. As we were getting close to it one of the manufacturer reps sat there and started to play a nice Elton John tune. We walked up to the Piano and enjoyed him doing so. Until he sang that is.

Here is the thing: if anyone's stereo reproduced that piano sound the way it was, they would cut off their wrists and jump out of the tallest building they could. Take Leif and VSA where he says he had a giant soundstage. The live piano had none of that. Heck, it was not even "stereo"! It was rather dull sound resembling nothing of the audiophile recordings we cherish where tremendous amount of post processing is applied to make it sound like what we want Piano to sound like, not what it is in real life. Our preferences have shaped the recorded music to something that is its own art form, separate and distinct from life.

So no, you don't learn a thing from listening to live piano a million times and then listening to someone else's performance on a recording from audiophile perspective. We hide behind such false pretences to make our opinion count more should we be situated that way. It has no logical or physical reality to match it as likeable as the theory is. Or else, musicians would clean our clocks good and have the best audio systems. Which they do not.

 

[amirm]

Let me get us back to what Ron post:

In a post on Al M.’s thread “ZenWave Audio D4 Interconnect” Peter A., describing a comparative listening test at Al’s house, wrote:


[...]

I am not suggesting we stop auditioning components, stop pursuing tweaks or stop listening for significant or subtle differences. I am suggesting that perhaps we should be more realistic and circumspect -- and more skeptical -- about our expressed conclusions. We should attempt to do the best we can do, and to try to remain as intellectually honest as possible, but perhaps we should acknowledge that we may be fooling ourselves about some of our listening conclusions.

I am 100% with him. Who else is?

 

[Gregadd]

There are many such tests with provable, objective, measureable difference. Problem is that vast majority of audiophiles can't pass such tests yet believe in easily hearing differences that are not measurable or provable to be there!

Such tests are not "pass fail" per se.
My point is so many audiophiles walk blindly(pun intended)expecting to discern a difference when in fact there is none. This does not mean the DUT is not superior. It could just mean it is not being sufficiently challenged. I am

 

[ddk]

At Axpona there was a grand piano in the lobby of the hotel. As we were getting close to it one of the manufacturer reps sat there and started to play a nice Elton John tune. We walked up to the Piano and enjoyed him doing so. Until he sang that is.

Here is the thing: if anyone's stereo reproduced that piano sound the way it was, they would cut off their wrists and jump out of the tallest building they could. Take Leif and VSA where he says he had a giant soundstage. The live piano had none of that. Heck, it was not even "stereo"! It was rather dull sound resembling nothing of the audiophile recordings we cherish where tremendous amount of post processing is applied to make it sound like what we want Piano to sound like, not what it is in real life. Our preferences have shaped the recorded music to something that is its own art form, separate and distinct from life.

So no, you don't learn a thing from listening to live piano a million times and then listening to someone else's performance on a recording from audiophile perspective. We hide behind such false pretences to make our opinion count more should we be situated that way. It has no logical or physical reality to match it as likeable as the theory is. Or else, musicians would clean our clocks good and have the best audio systems. Which they do not.

Come on Amir, you're ignoring the different recording environments of recordings, how many are recorded in a dead sounding hotel lobby on an out of tune piano? And you're listening to way too much digital !

david

 

[amirm]

Come on Amir, you're ignoring the different recording environments of recordings, how many are recorded in a dead sounding hotel lobby on an out of tune piano? And you're listening to way too much digital !

david

Well, real piano has no pops and clicks so not sure why anyone listen to LPs if that is the standard.

 

[steve williams]

Come on Amir, you're ignoring the different recording environments of recordings, how many are recorded in a dead sounding hotel lobby on an out of tune piano? And you're listening to way too much digital !

david

+!

Probably the first time he's heard a live piano