Audio Science: Does it explain everything about how something sounds?

[jkeny]

Are you saying that the long-term listening argument is not used in the community to dismiss the results of blind listening tests? Even in this very thread?

Tim

Ah, give up, Tim - there are very good points raised about (short term) blind testing Vs long-term listening - to try to polarise this as you are doing above is just point scoring & shows an inability to engage with the discussion & deal with the issues raised.

Anybody who ignores the fact (yes fact) that long term listening can often reveal unpleasant aspects to the sound that just weren't perceived in short term listening (I'm not even stipulating blind here) is fooling themselves & ignoring reality

Edit: I see that you have edited your post:

Are you saying that there is a specific point in long-term listening where these things begin to be revealed?

Just by asking this question it shows me that you didn't bother to read the replies to Tomolex

Are you able to confirm these things revealed in long-term listening with anything other than personal opinion?

You mean you want me to be able to "confirm" these things in a blind test - lol - I'm sure you see the circular logic here ?

 

[Tony Lauck]

Very nice post Amir, I copied it to my wbf best quotes file

I want to add an abbreviated addition I just did to a different thread I started a while back; item number 10, which goes along with this post:

Some folks claim that it takes long term listening to hear the details and nuances that exist between components etc. They usually proclaim that short term listening does not allow you to tune into these nuances.

---However, the question I ask is, so after a week, or a month or whatever of listening, and now they "decide" or "hear" what is really going on, what led them to now believe their ears at that point in time?

---What biological element all the sudden "showed them the light" and why would they believe this as it seems to not occur at any regular time frame, just all the sudden like?

---If it is a gradual thing then what are their ears doing all along this time...are they adapting to the sound, or are they angrily just getting less and less tolerant of it, and how are they doing that?

There you go, more questions, no answers I suspect....

You may as well ask why some people believe in God and others do not and why some people change their views. There may be answers to these sorts of questions, but they are personal and not a proper subject for a debate on "Audio Science".

 

[Phelonious Ponk]

Ah, give up, Tim - there are very good points raised about (short term) blind testing Vs long-term listening - to try to polarise this as you are doing above is just point scoring & shows an inability to engage with the discussion & deal with the issues raised.

I guess I'll need to go back and review these points. I don't recall anything but conjecture and opinion.

Anybody who ignores the fact (yes fact) that long term listening can often reveal unpleasant aspects to the sound that just weren't perceived in short term listening (I'm not even stipulating blind here) is fooling themselves & ignoring reality

On this we can sort of agree. Listening for more than a few seconds can probably result in fatigue from distortions not detected in A/B switching. Whether or not the source of the fatigue is a component, the entire system, the recordings or the room remains in question, but I don't object.

Edit: I see that you have edited your post:
Just by asking this question it shows me that you didn't bother to read the replies to Tomolex

You mean you want me to be able to "confirm" these things in a blind test - lol - I'm sure you see the circular logic here ?

Confirmation through blind listening tests, no. Substance? Yes.

Tim

 

[Tony Lauck]

Typically, there is no specific point in the listening continuum where these subtleties are revealed. It starts before listening begins, as an argument to dismiss the results of blind listening tests that go against what they think they hear with all their non-auditory biases fully engaged.

Tim

I do not dismiss the results of positive blind tests, such as the ones that Amir passed. The debate is not about these, the debate is about interpretation of negative blind tests.

For me tests with negative results have no value. If one wants to reach a negative conclusion one will have to reformulate it as a positive statement and attempt to prove that. Otherwise, the use of statistical methods is just plain wrong. In general, almost all "science" that uses statistical results is BS mathematics done by people who are either ignorant or hired guns to prove a point. This is everything where the threshold is set at 5%. Real science, such as physics, works at much tighter statistical confidence and does so with an understanding of the relevant probability distributions of unavoidable experimental error.

 

[Phelonious Ponk]

Jon are you talking about this post of Tomelex'?

Very nice post Amir, I copied it to my wbf best quotes file

I want to add an abbreviated addition I just did to a different thread I started a while back; item number 10, which goes along with this post:

Some folks claim that it takes long term listening to hear the details and nuances that exist between components etc. They usually proclaim that short term listening does not allow you to tune into these nuances.

---However, the question I ask is, so after a week, or a month or whatever of listening, and now they "decide" or "hear" what is really going on, what led them to now believe their ears at that point in time?

---What biological element all the sudden "showed them the light" and why would they believe this as it seems to not occur at any regular time frame, just all the sudden like?

---If it is a gradual thing then what are their ears doing all along this time...are they adapting to the sound, or are they angrily just getting less and less tolerant of it, and how are they doing that?

There you go, more questions, no answers I suspect....

And these responses?

Thanks . The biological factors work against the merits of long term testing. Imagine the incredible amount of data our ears are collecting every moment. There is no way we can store it all in our brain. What happens is that short-term memory captures it all but after a few seconds, huge data reduction happens and just the high-level characteristics of what is heard is committed to long term memory. At that point, it becomes impossible to recall nuances of what we hear. That data is lost far more readily than low rate MP3. This means when we are talking about small differences, you have to rely on short-term memory.

I love to know the physiological theory of why people think our hearing system and perception work differently.

tomelex, here is an example of something I notice over time. I hear systems that initially sound detailed and high in resolution. I have a positive impression of how they sound, but after additional, longer term listening, they prove to be fatiguing. I then think that the sense of detail and resolution is probably distortion which I do not notice initially but it becomes more apparent the more time I spend listening. I have found this with Wilson speakers on occasion. I have often heard them sound pretty good, initially, in dealer showrooms and in friend's systems. But after a bit more time listening, I get tired, lose interest, and want to stop listening.

I do not detect this fatigue in short term listening or during rapid A/B tests. But at some point with longer exposure, my ears tell me enough is enough.

Peter, such is the case of those with ears to hear. And the more well-trained one's ears become, the quicker they are to reach that point of enough is enough.

Every once in a blue moon we may run across somebody who has extremely well-trained ears. I know of one such person who a number of years ago was a relatively unknown reviewer. Regrettably, he's perhaps one of the flakier people I know. Nevertheless, when my technology was still in its infancy stages a number of years ago, he came to visit and brought with him his most torturous opera music claiming that exhibitors at audio shows cringe when he pulls out this CD for them to play. I was pretty thrilled when he said this was the first time he's ever heard this piece without any apparent breakup or flattening out and he rather enjoyed his audition time.

Advance about 9 months and I had just inserted a new pair of IC's to audition that were near the full burn-in mark and this gent pays me another visit. He obviously had a memory from his previous visit and starts to listen. Within maybe 30 seconds, he looks at me asks, what did you do to your system? I explained that I upgraded my pre-amp some months back and that it was clearly superior to my prior preamp and that I was auditioning these new IC's. He expressed great displeasure at what he was hearing as we continued listening for about 30 minutes. I then exchanged the new IC's with the old IC's and instantly he said, that's it. In my own defense I'm not going to say whether or not I heard half of what he heard with the new IC's. He was actually a bit angry that I made him suffer so long. But he's probably one in 100,000 who can almost instantaneously distinguish differences in systems he's not that familiar with. I've never encountered anybody with such extremely well-trained ears. A buddy of mine who is very well-seasoned with his own very well-trained ears (much more than mine) even admitted this other gent's hearing is remarkable.

Anyway, the point being that where it may take people like you and me a time to determine something ain't right, there exists an extremist or two out there who can almost instantly tell something's not right and describe in detail why it's not right. Then there's those in between where it may take 5 minutes to 4 hours of continued listening before they realize something's not right and do a fairly reasonable job describing why.

And of course, there are also those who can never under perhaps any circumstance make any distinction whatsoever.

These types with very well-trained ears are the first (and perhaps the last) to admit there stands a huge gulf between the live performance and music reproduced on today's best playback systems. This is a big reason why we need to be so apprehensive about believing what others say about sonic performance (and audio science too). Because from an internet perspective, it's all too easy to assume everybody has the same hearing skills.

Although there needs to be more research on 'dissonance' or threshold-tolerance and behaviour difference between short and long term.
This is one reason a product with say a forward treble may seem to be more detailed/accurate initially but over longer period listeners/owners tend to suffer some kind of dissonance/perceptive niggles/listening fatigue that then result in less satisfaction (could say preference as well then) and listening behaviour changes.

Edit:
Ah just read some others have picked up this type of listening behaviour with themselves and such products, which go beyond speakers, and even to modern mastering of records unfortunately (where not done well).
Of course I should mention some kind of anchoring also needs to be considered in what influences our listening behaviour for short and long term, really if anyone is being critical listener they need a separate 'control' system to reset in a way like palate cleansers do for critical tasting and blending (ok not the same but context sort of similar).

Cheers
Orb

And your subsequent agreement with all but Amir? Go back to Tom's post at the top. See the bolded section? None of the answers to his post, except for Amir's, even addressed his point that "Some folks claim that it takes long term listening to hear the details and nuances that exist between components etc." They usually proclaim that short term listening does not allow you to tune into these nuances." Instead of addressing the claim of greater detail and nuance being revealed by long-term listening, they changed the subject to listening fatigue. I read them, but realized they were answers to a different question than the one Tom posed. Perhaps they're really good answers to some question, but not this one.

Tim

 

[jkeny]

..Confirmation through blind listening tests, no. Substance? Yes.
Tim

What "confirmation" do you want? Name it

Jon are you talking about this post of Tomelex'?

And these responses?

And your subsequent agreement with all but Amir? Go back to Tom's post at the top. See the bolded section? None of the answer to his post, except for Amir's, even addressed his point that "Some folks claim that it takes long term listening to hear the details and nuances that exist between components etc. They usually proclaim that short term listening does not allow you to tune into these nuances." Instead of addressing the claim of greater detail and nuance being revealed by long-term listening, they changed the subject to listening fatigue. I read them, but realized they were answers to a different question than the one Tom raised.

Tim

Tom's question "Some folks claim that it takes long term listening to hear the details and nuances that exist between components etc. They usually proclaim that short term listening does not allow you to tune into these nuances."

The answers given were that one can find oneself no longer liking a component's delivery after living with it for a time, even though it appeared to be stellar when first listened to. How long this might take depends but the point is that short term listening is prone to be mistaken compared to longer term listening. Even more prone to error is blind A/B listening as adequate training in various forms of distortion & how to recognise them isn't normal (is anyone person fully trained in all the different possible distortions & recognition of them - I doubt it). It also requires training to be able to focus/concentrate during such testing and/or to recognise when one has lost focus during the many repetitions required for statistical significance. Not to mention the fact that the studies of auditory attention show that we can only focus on one auditory object at a time so what are we blind testing? Is it the right difference between tracks/components. Are we missing something else that maybe only occurs intermittently? Does the difference change during the playback? Etc. etc. One shot blind testing is just not a suitable mechanism for most of the issues that we have identified & are talking about.

And even with all these issues dealt with (& they seldom are or can't be) you admit that "On this we can sort of agree. Listening for more than a few seconds can probably result in fatigue from distortions not detected in A/B switching. Whether or not the source of the fatigue is a component, the entire system, the recordings or the room remains in question, but I don't object."

 

[Fitzcaraldo215]

I do not dismiss the results of positive blind tests, such as the ones that Amir passed. The debate is not about these, the debate is about interpretation of negative blind tests.

For me tests with negative results have no value. If one wants to reach a negative conclusion one will have to reformulate it as a positive statement and attempt to prove that. Otherwise, the use of statistical methods is just plain wrong. In general, almost all "science" that uses statistical results is BS mathematics done by people who are either ignorant or hired guns to prove a point. This is everything where the threshold is set at 5%. Real science, such as physics, works at much tighter statistical confidence and does so with an understanding of the relevant probability distributions of unavoidable experimental error.

I do not disagree with much of this. But, is there a debate here about negative results in blind tests? I agree, as I am sure many others here including Amir agree, with you that by failing to prove the positive with some level of statistical significance, you have NOT "proven" the negative. Sure, many assume that and do that, but doing so is not true science or statistics. End of story.

I also disagree with you about 95% significance being BS in tests involving human perception. Sure, a higher threshold would be nice, but are significantly higher thresholds used consistently in other fields involving tests of human perception?

Audio science is an applied science, not a "hard" science like physics, which seldom uses human test subjects. Any parallels with higher thresholds in physics are apples and oranges. But, yes, people misuse and misinterpret science and scientific data all the time. That is especially true in audio, because of its largely commercial and highly marketed application to consumers, unlike physics, and other "pure" sciences.

 

[jkeny]

And just to repeat myself, as I believe that this hasn't been registered yet (as evidenced by Tomolex congratulating Amir on his great post about the Clarke paper & Amir accepting his congratulations) - the Clarke paper is flawed (as is the similar Nousaine paper) on long-term listening. Simply put comparing a one box long-term test Vs a side by side box A/B test is comparing apples & oranges (or in Nousaine's case, comparing apples with trained apples).

If this wasn't what was done in the Clarke test & two identical boxes (one with distortion) were given to each participant for home testing then I'm wrong but if it was then please, Amir & Tomolex, have the good grace to admit that it is a flawed test & stop the congratulations or the further use of it as some "proof" that long-term listening is flawed!!

To me, both these papers show exactly the usual problem with blind testing - it's usually agenda driven & full of experimenter's bias as we see in both these papers. The acceptance of the paper by AES doesn't exactly say much for the claims of them being some authority of "audio science" either!

 

[853guy]

That (tonality) is not what you are asked to score. You are asked to score your overall preference for one loudspeaker versus another. Again, this is what we all do day in and day out. We listen and compare and give an opinion of what we like better. You are free to use whatever metric you want in giving that final score. Use emotion, logic, analysis, whatever. You are not given any bounds just like you do in sighted evaluation...

...I am stating what the experience is, not the rules of the test. The rules of the tests are exactly as I mentioned: score them loudspeaker preference from 1 to 10. That is it. Now, sit in the chair and listen and the first thing that hits you in the face is how different tonally each loudspeaker is. I am sure everyone agrees that every loudspeaker has a different tone to it. So it is natural to judge that, given how large and obvious the difference is. High correlation to frequency response measurements shows that this is indeed what people naturally do, without being instructed one way or the other.

But again the tests are pure preference with no restrictions.

Okay, just so I’m clear on this:

Harman subjects its speakers to 70 individual frequency-response measurements at 10 degree intervals in an anechoic chamber and then conducts blind listening test for preference where…

1) The room is of no resemblance to the original space the measurements were taken in

2) The measurements use steady-state signals but the blind testing uses music

3) The speakers are not matched with regard to driver composition, number of drivers, crossover frequency, crossover slopes, nominal impedence, minimum magnitude of impedence, electrical phase angle, cabinet resonances, step response, cabinet construction, etc

4) The subjects are not screened for conflict of interest/hearing irregularities/ability to correctly identify tonal variations under ABX testing

5) The subjects are asked to give an arbitrary numerical rating in which there is no consistency of metric in application

…and then claims high correlation between smooth on/off-axis frequency response and preference where NONE of the above variables are eliminated?

I… I…

If I am testing for preference of a specific variable (frequency response) and want to claim a statistically significant correlation between that variable and preference, but do not eliminate or control all the potential confounding variables that may adversely affect the relationship between the independent variable and dependent variables, I will damage the internal validity of the experiment. In short, I will not have correlation, I will have an anecdote and I will fail Psyc 101.

Do you really feel completely comfortable calling the above testing protocol “audio science”?

You mean if someone presents same or similar spatial averaging loudspeakers to me in a sighted evaluation like we all do, I can't rely on my ears? I will be relying on other senses alone like skin conductance? Can you give an example of two different loudspeakers where my ear would say they sound the same but these other senses say different?

No, you could rely on your ears, but if we were testing two different speakers that sounded similar, and the subject had difficulty articulating the differences, skin conductance response, heart rate, respiratory frequency, skin temperature and facial EMG would perhaps tell us whether these physiological responses were indicative of neurological processes at a sub-conscious level the subject is not aware of.

Remember, the #1 factor in sales of loudspeakers is marketing. Nothing remotely approaches the power of that. So let's not use commercial success in this context. It belies the reality of how the market works...

...Many loudspeaker desigenrs rely on this research in producing their loudspeakers. Their value therefore is established, lack of awareness of high-end consumers notwithstanding.

Which is tantamount to saying even the flakiest of measurements made in the name of science are of value for the purpose of marketing. I agree.

 

[AJ Soundfield]

To me, both these papers show exactly the usual problem with blind testing - it's usually agenda driven & full of experimenter's bias as we see in both these papers.

Vs your "long term" peeking/knowing/fondling/bias overload/mood swings "listening" of the DUT?
So then you also reject your own results of blind tests too. Problematic. Not to be touted. Is this correct John?

cheers,

AJ

 

[jkeny]

Vs your "long term" peeking/knowing/fondling/bias overload/mood swings "listening" of the DUT?
So then you also reject your own results of blind tests too. Problematic. Not to be touted. Is this correct John?

cheers,

AJ

Ah, the usual retort - shift the focus to the other side of the binary debate & try to move the focus away from the flaws I mention!!

So you are willing to suspend your powers of critical analysis for this AES paper & bypass it's flaws to move onto a different argument. I think it was Amir I saw posting that critical analysis should not be reserved for just one side of the argument & suspended for the other - not a truly objectivist method, now is it?

Instead you want to see this AES paper put forth by Amir as "proof" that long-term listening is flawed & you sit by & see it applauded by Tomolex as some sort of reference that he is going to use?

What is your opinion of the paper?

And, BTW, just in case you missed it - I have already answered your biased statements & questions - in essence both are flawed but one (long term listening) is less flawed than the other given the home circumstances where we are doing this listening & irrespective of the many,many difficulties in doing blind testing, it is only revealing of some audible differences, not all audible differences between components.

 

[AJ Soundfield]

And, BTW, just in case you missed it - I have already answered your biased statements & questions - in essence both are flawed but one (long term listening) is less flawed than the other

You keep repeating this falsehood. In a blind test, what one does is Listen. Trusting your ears.
In your "long term" business, you are peeking, fondling, knowing, etc, etc.
The two are incomparable.
If you want to know how something sounds, one listens/trusts their ears in a blind test.
If one want to know what sort of psychogenic melodrama a widget creates for an audiophile mind, by all means, stare at/fondle/etc it for as long as needed. Two different things entirely.

given the home circumstances where we are doing this listening & irrespective of the many,many difficulties in doing blind testing, it is only revealing of some audible differences, not all audible differences between components.

"Audible" as defined by your eyes, touch, moods, mental health, biases, etc? Or "audible" as in soundwaves>ears? What made up definition of words are you using?

 

[jkeny]

So, first off, you refuse to answer my question "What is your opinion of the paper?" & what of my highlighting of it's flaws. Don't wish to discuss this? Prefer to deflect to other issues?

You keep repeating this falsehood. In a blind test, what one does is Listen. Trusting your ears.

And here is the fallacy trotted out again & again "Trusting your ears" "In a blind test, what one does is Listen" This fallacy is perpetrated by those who don't know or don't want to admit to the issues with blind testing preferring to simplify it to these soundbites - the politician's uninformed double talk.

In your "long term" business, you are peeking, fondling, knowing, etc, etc.
The two are incomparable.

Both can be flawed but long-term listening is very much more suited to what we want from audio which is living with a piece of audio equipment over the long term.

If you want to know how something sounds, one listens/trusts their ears in a blind test.

Rubbish as most blind tests are fatally flawed & even if they weren't procedurally flawed they only cover a certain type of difference

If one want to know what sort of psychogenic melodrama a widget creates for an audiophile mind, by all means, stare at/fondle/etc it for as long as needed. Two different things entirely.

Keep repeating your mantra - I'm sure there are a lot who fall into the stupor of it's spell - as evidenced by the parking of critical analysis when looking at Clarke's paper that supports their position

"Audible" as defined by your eyes, touch, moods, mental health, biases, etc? Or "audible" as in soundwaves>ears? What made up definition of words are you using?

Do you really think that what you are testing in a blind test is "soundwaves>ears" & nothing else?? Please, I think you need to educate yourself in psychoacoustics & auditory perception

 

[jkeny]

Why is it always speaker manufacturers that seem so ill-informed about blind tests & yet have such strongly held views? Is it because they see themselves as outside of this need for blind testing?

Can I ask you what blind tests you have done? How do you blind test your speakers? Do you believe in room-treatments & how do you blind test them?

 

[AJ Soundfield]

Don't wish to discuss this?

That is exactly what I am discussing: Audio Science: Does it explain everything about how something sounds?
You want to equate that with "Beliefs: Does it explain everything about how something looks/feels/imagines/causes mood swings" etc, etc.

For there to be a discussion about Audio/Sounds back in Kansas, you don't get to make up word meanings.

 

[jkeny]

I know this poster from another forum & as we can all see he is a practised, point-scoring, forum debater who refuses to engage in a topic - just repeat his mantra as he brings anyone who follows down his particular rabbit hole.
3 posts on this forum is enough to ascertain his M.O.
I'll not be engaging him!

 

[Phelonious Ponk]

What "confirmation" do you want? Name it

Tom's question "Some folks claim that it takes long term listening to hear the details and nuances that exist between components etc. They usually proclaim that short term listening does not allow you to tune into these nuances."

The answers given were that one can find oneself no longer liking a component's delivery after living with it for a time, even though it appeared to be stellar when first listened to. How long this might take depends but the point is that short term listening is prone to be mistaken compared to longer term listening. Even more prone to error is blind A/B listening as adequate training in various forms of distortion & how to recognise them isn't normal (is anyone person fully trained in all the different possible distortions & recognition of them - I doubt it). It also requires training to be able to focus/concentrate during such testing and/or to recognise when one has lost focus during the many repetitions required for statistical significance. Not to mention the fact that the studies of auditory attention show that we can only focus on one auditory object at a time so what are we blind testing? Is it the right difference between tracks/components. Are we missing something else that maybe only occurs intermittently? Does the difference change during the playback? Etc. etc. One shot blind testing is just not a suitable mechanism for most of the issues that we have identified & are talking about.

And even with all these issues dealt with (& they seldom are or can't be) you admit that "On this we can sort of agree. Listening for more than a few seconds can probably result in fatigue from distortions not detected in A/B switching. Whether or not the source of the fatigue is a component, the entire system, the recordings or the room remains in question, but I don't object."

Bold 1) Exactly. The answers you insisted that I re-read were not answers to Tom's post, but a different issue altogether. These answers say long-term listening reveals irritations, not subtlety and detail. Those are completely different things. Nearly opposite things.

Bold 2) Yes. A system with high frequency distortion may not sound right at first listen, but if it is harsh and glaring, it really gets on your nerves after awhile. No argument. But irrelevant.

Tim

 

[Tony Lauck]

I know this poster from another forum & as we can all see he is a practised, point-scoring, forum debater who refuses to engage in a topic - just repeat his mantra as he brings anyone who follows down his particular rabbit hole.
3 posts on this forum is enough to ascertain his M.O.
I'll not be engaging him!

Me, neither!

 

[AJ Soundfield]

I know this poster from another forum

I thoroughly enjoyed our chats, with both you and Amir and was quite disappointed when they ceased.

I'll not be engaging him!

Shame, since we seem to largely agree!
If one wants to know about audio/sound, then trust your ears in a blind test, as Amir did at Harman.
If one has zero trust in their ears, by all means fondle, prime the pump by reading mags about it and fellow story tellers, etc, etc. then stare at a widget, for several weeks (month, years??) and see what sort of psychogenic drama unfolds and makes one happy/satisfied.
Where do we disagree?
You do realize the Harman tests Amir refers to are about audio/sound, yes?

 

[jkeny]

Bold 1) Exactly. The answers you insisted that I re-read were not answers to Tom's post, but a different issue altogether. These answers say long-term listening reveals irritations, not subtlety and detail. Those are completely different things. Nearly opposite things.

Who says that irritations are not the result of issues of subtlety & detail? In fact I believe an example was given of a drop in noise floor not being noticeable in itself, per se, but because it allowed the emotion of the performance to be better perceived. This is precisely because the subtle details are better revealed (or revealed for the first time) - subtle details such as the inflections in a singer's voice or the timing & interplay between different parts of the band.

Bold 2) Yes. A system with high frequency distortion may not sound right at first listen, but if it is harsh and glaring, it really gets on your nerves after awhile. No argument. But irrelevant.

Tim

I think it's that the component may sound interesting & more revealing but on longer listening this is fatiguing - yes I believe that it's false detail due to distortion. But there's the opposite - it may sound very good at first but on longer listening, less engaging, less emotionally interesting. Is this down to a distortion?