Conclusive "Proof" that higher resolution audio sounds different

[Orb]

Snip..... I get that they are testing for the audibility of small differences. I get that the differences are so small that the tests demand listeners trained to hear those specific differences, and listening protocols aimed at finding them. This is exactly why I think these tests are irrelevant to the normal listening experience, and would like to see tests run under real listening conditions, with music lovers as well as trained listeners. Got it?

Tim

As repeated many times no
You are overextending a conclusion because there is NO data showing the tolerance/threshold for human listening behaviour in terms of these variables, same way we do not have tolerance/threshold for human longer term listening/owning behaviour of say digital filters when comparing steep brickwall to say minimum phase (although compounded the coefficiants can provide various solutions).
In this context tolerance is how it would influence behaviour in terms of possibly satisfaction/right-not right/comfortability/etc, while threshold is the number of times/duration/magnutude/frequency/etc it is required to have a perceived-cognitive effect on the listener.
Please note I am just using tolerance/threshold loosely to help put it a bit into context with understanding why one must not extend a conclusion beyond the scope-limitation of a test; this test just shows for some reason Amir and a couple of others can differentiate between original hirez and a "handled" downsampled/decimated file of the original, this is interesting because hirez masterfiles (another consideration is potentially how digital recorded file-stream is also handled in studio for mixing then mastering) can be "handled" many times from studio to the home listener's DAC.
To say what you are Tim, further testing-study would be needed and with some seriously complex controls/environment/system to have any actual worth or relevance.
Handled in this context means actual related software/hardware processes involved and not say download-copying-etc.

Just to add, Amir has never suggested one is better than the other, just that it can make sense having the original native and played as that to ensure integrity is maintained.

Cheers
Orb

 

[jkeny]

BTW, it's probably easy to dismiss this thread as irrelevant but I came across this quote from Isaac Asimov recently which has relevance

The most exciting phrase to hear in science, the one that heralds the most discoveries, is not “Eureka!,” but “That’s funny . . .”

 

[Phelonious Ponk]

jack --

I think the big question that you are asking, correct me if I'm wrong, is that if the difference is now small, why bother at all?

Thanks, but not quite. I'm hoping for another test, one aimed at telling us if untrained listeners, listening to music, and with the only variables being hi-res vs 16/44.1, can hear a difference. I'm asking for a different test, one that I think would be much more relevant and meaningful to audiophiles and music lovers. I've made this very clear, yet john, and now micro, seem to want to insist that I don't understand the tests that are being discussed here. Actually, I think I do I understand them, and their objectives, and that's why I'd like to see a different test.

At the beginning of this part of the conversation, someone said they thought this new test was going to be like Meyer & Moran under BS1116 guidelines. That would be very different than the kind of test Amir ran, with a trained listener identifying a difference that has consistently been inaudible to the untrained, even under the same conditions. That test, that refinement of Meyer & Moran, would include a broad variety of experienced listeners, listening to music. That's the test I'd like to see, because I'd like to see results that might be meaningful to my listening experience.

Orb --

As repeated many times no

Yes, and yet some people continue to insist that I'm saying something else entirely. My apologies for the redundancy.

Tim

 

[jkeny]

Thanks, but not quite. I'm hoping for another test, one aimed at telling us if untrained listeners, listening to music, and with the only variables being hi-res vs 16/44.1, can hear a difference. I'm asking for a different test, one that I think would be much more relevant and meaningful to audiophiles and music lovers. I've made this very clear, yet john, and now micro, seem to want to insist that I don't understand the tests that are being discussed here. Actually, I think I do I understand them, and their objectives, and that's why I'd like to see a different test.

At the beginning of this part of the conversation, someone said they thought this new test was going to be like Meyer & Moran under BS1116 guidelines. That would be very different than the kind of test Amir ran, with a trained listener identifying a difference that has consistently been inaudible to the untrained, even under the same conditions. That test, that refinement of Meyer & Moran, would include a broad variety of experienced listeners, listening to music. That's the test I'd like to see, because I'd like to see results that might be meaningful to my listening experience.

Tim, in the bolded text you have already answered your question - what is important to your listening experience can only be answered by you, yourself, by listening. No test can tell you this! Just as the MP3 codec tests cannot tell what level of compression is audible to you?

The test to answer the question you pose "are these differences meaningful to ordinary listeners in ordinary listening experiences" is so complicated & wide-ranging that it's next to impossible to conduct. To force the issue (by not realising the complexity of what's required) will result in a badly designed test. The highly likely outcome of a badly designed test is a null result as has been discussed here repeatedly. This is what you & Garrett (Max) are being repeatedly told.

Let's take Garrett (Max's) example - he claims to be a non-expert listener (which I believe) & yet he claims to be able to hear the differences between the Winers loopback generational files using an ABX. Yet he listened before running ABX & says he couldn't hear the differences. I've asked him if he can now hear the differences sighted but he hasn't answered. So what is his normal listening experience? Is it now different? Now that he has identified audible differences in the files? If he listens sighted as he had done before will he hear these differences or do they revert back to being inaudible? So is he an example of normal listener or has he now been forever changed?

 

[JackD201]

Thanks, but not quite. I'm hoping for another test, one aimed at telling us if untrained listeners, listening to music, and with the only variables being hi-res vs 16/44.1, can hear a difference. I'm asking for a different test, one that I think would be much more relevant and meaningful to audiophiles and music lovers. I've made this very clear, yet john, and now micro, seem to want to insist that I don't understand the tests that are being discussed here. Actually, I think I do I understand them, and their objectives, and that's why I'd like to see a different test.

At the beginning of this part of the conversation, someone said they thought this new test was going to be like Meyer & Moran under BS1116 guidelines. That would be very different than the kind of test Amir ran, with a trained listener identifying a difference that has consistently been inaudible to the untrained, even under the same conditions. That test, that refinement of Meyer & Moran, would include a broad variety of experienced listeners, listening to music. That's the test I'd like to see, because I'd like to see results that might be meaningful to my listening experience.



Yes, and yet some people continue to insist that I'm saying something else entirely. My apologies for the redundancy.

Tim

My interest in such a test would be academic at best. You know my stand on things. When you can't tell the difference, go with what's cheaper or whatever other reason you can think of, resale value, looks, whatever. The approach makes less money as a seller, saves money as a buyer but best of all makes for better night's sleep. THAT is priceless, especially since I'm at that age where I wake up the same time every morning whatever I do!

 

[Orb]

Thanks, but not quite. I'm hoping for another test, one aimed at telling us if untrained listeners, listening to music, and with the only variables being hi-res vs 16/44.1, can hear a difference. I'm asking for a different test, one that I think would be much more relevant and meaningful to audiophiles and music lovers. I've made this very clear, yet john, and now micro, seem to want to insist that I don't understand the tests that are being discussed here. Actually, I think I do I understand them, and their objectives, and that's why I'd like to see a different test.

At the beginning of this part of the conversation, someone said they thought this new test was going to be like Meyer & Moran under BS1116 guidelines. That would be very different than the kind of test Amir ran, with a trained listener identifying a difference that has consistently been inaudible to the untrained, even under the same conditions. That test, that refinement of Meyer & Moran, would include a broad variety of experienced listeners, listening to music. That's the test I'd like to see, because I'd like to see results that might be meaningful to my listening experience.



Yes, and yet some people continue to insist that I'm saying something else entirely. My apologies for the redundancy.

Tim

That is not what your previous comment I quoted says though Tim that you have insisted upon several times
You keep jumping to a conclusion regarding that specific point I picked up (not same point as what you/Micro/John are debating and does not need me in that as well tbh ) and as I noted in my previous response that is entirely wrong to do so, nothing to do with the debating with others tbh.
Cheers
Orb

 

[Phelonious Ponk]

Tim, in the bolded text you have already answered your question - what is important to your listening experience can only be answered by you, yourself, by listening. No test can tell you this! Just as the MP3 codec tests cannot tell what level of compression is audible to you?

The test to answer the question you pose "are these differences meaningful to ordinary listeners in ordinary listening experiences" is so complicated & wide-ranging that it's next to impossible to conduct. To force the issue (by not realising the complexity of what's required) will result in a badly designed test. The highly likely outcome of a badly designed test is a null result as has been discussed here repeatedly. This is what you & Garrett (Max) are being repeatedly told.

Let's take Garrett (Max's) example - he claims to be a non-expert listener (which I believe) & yet he claims to be able to hear the differences between the Winers loopback generational files using an ABX. Yet he listened before running ABX & says he couldn't hear the differences. I've asked him if he can now hear the differences sighted but he hasn't answered. So what is his normal listening experience? Is it now different? Now that he has identified audible differences in the files? If he listens sighted as he had done before will he hear these differences or do they revert back to being inaudible? So is he an example of normal listener or has he now been forever changed?

John, A) I didn't ask a question B) Change the bold part above to "meaningful when actually listening to music. I stopped reading after the first paragraph. So don't expect any answer to/defense of whatever follows.

Tim

 

[esldude]

In audio quality evaluation, ITU-R BS.1534-1, commonly known as MUSHRA, is widely used for the subjective assessment of intermediate audio quality. Studies have identified limitations of the MUSHRA methodology [1][2], which can influence the robustness to biases and errors introduced during the testing process. Therefore ITU-R BS.1534 was revised to reduce the potential for introduction of systematic errors and biases in the resulting data. These modifications improve the validity and the reliability of data collected with the MUSHRA method. The main changes affect the post screening of listeners, the inclusion of a mandatory mid-range anchor, the number and length of test items as well as statistical analysis. In this paper the changes and reasons for modification are given.
Convention Paper 9172

Of course do note the first sentence. Assessment of intermediate audio quality. Things that would show forth clearly in more rigorous blind testing using bs1116 guidelines. MUSHRA being a shortcut to be done in a simpler, easier, and quicker evaluation. But it has its place I would think. It allows statistically useful results with more differences and fewer participants. It also is specifically not to be used for small sound impairment according to the guidelines.

Basically you are presented with several different versions of sound, and rank them on a 0-100 scale. You do have a reference to compare with. You also have a hidden reference and a highly degraded sound to use as controls. For instance if you rate the hidden reference quite low your results as a useful listener are suspect. Similarly if you rate the highly degraded version quite high on the scale your results are suspect.

 

[Phelonious Ponk]

That is not what your previous comment I quoted says though Tim that you have insisted upon several times
You keep jumping to a conclusion regarding that specific point I picked up (not same point as what you/Micro/John are debating and does not need me in that as well tbh ) and as I noted in my previous response that is entirely wrong to do so, nothing to do with the debating with others tbh.
Cheers
Orb

I'm afraid I'm not following you, Orb. Are you talking about this quote?

Originally Posted by Phelonious Ponk View Post
Snip..... I get that they are testing for the audibility of small differences. I get that the differences are so small that the tests demand listeners trained to hear those specific differences, and listening protocols aimed at finding them. This is exactly why I think these tests are irrelevant to the normal listening experience, and would like to see tests run under real listening conditions, with music lovers as well as trained listeners. Got it?

Tim

I'm not aware of jumping to any conclusions at all. In fact, I'm of the opinion that no one has concluded anything in this thread. Help me out. Where have I reached a conclusion and repeated it?

Tim

 

[jkeny]

snip...............
Basically you are presented with several different versions of sound, and rank them on a 0-100 scale. You do have a reference to compare with. You also have a hidden reference and a highly degraded sound to use as controls. For instance if you rate the hidden reference quite low your results as a useful listener are suspect. Similarly if you rate the highly degraded version quite high on the scale your results are suspect.

These hidden controls are useful to check if the test/testers/setup/procedure/etc. are valid or suspect & should be included in all such tests. Something I have been asking for all along.

I would be interested in what such hidden controls would reveal about the blind tests we see often reported in forums. Anybody care to repeat a previous null blind test get-together?

 

[Orb]

I'm afraid I'm not following you, Orb. Are you talking about this quote?



I'm not aware of jumping to any conclusions at all. In fact, I'm of the opinion that no one has concluded anything in this thread. Help me out. Where have I reached a conclusion and repeated it?

Tim

It is the specific relation you are putting to:

Snip..... I get that they are testing for the audibility of small differences. I get that the differences are so small that the tests demand listeners trained to hear those specific differences, and listening protocols aimed at finding them. This is exactly why I think these tests are irrelevant to the normal listening experience

Reading my response will help why you are jumping to a conclusion/assumption that very small differences do not affect normal listening behaviour.
You have to think about how it relates to a cognitive listening behaviour rather than an experience, behaviour is affected by listening duration (both accumulative over multiple sessions and single duration sessions) and critical the loosely defined used I explained earlier regarding tolerance and threshold and relates to sensitivity to errors/anomalies/etc and strength of reaction to them; part of which can include satisfaction-enjoyment-or cognitive notice or lack of something (and also dissonance) - I touched on subtly different aspects in earlier post and thought to add this bit that I missed originally.

The same concept is potentially applicable to digital filters/dither-noise shaping/steep-slow rolloff/strong-weak alias rejection/etc; these are usually very subtle (putting aside the sin(x)/x designs on FR or NOS related designs).
Cheers
Orb

 

[microstrip]

Thanks, but not quite. I'm hoping for another test, one aimed at telling us if untrained listeners, listening to music, and with the only variables being hi-res vs 16/44.1, can hear a difference. I'm asking for a different test, one that I think would be much more relevant and meaningful to audiophiles and music lovers. I've made this very clear, yet john, and now micro, seem to want to insist that I don't understand the tests that are being discussed here. Actually, I think I do I understand them, and their objectives, and that's why I'd like to see a different test.

At the beginning of this part of the conversation, someone said they thought this new test was going to be like Meyer & Moran under BS1116 guidelines. That would be very different than the kind of test Amir ran, with a trained listener identifying a difference that has consistently been inaudible to the untrained, even under the same conditions. That test, that refinement of Meyer & Moran, would include a broad variety of experienced listeners, listening to music. That's the test I'd like to see, because I'd like to see results that might be meaningful to my listening experience.



Yes, and yet some people continue to insist that I'm saying something else entirely. My apologies for the redundancy.

Tim

No Tim, sorry to tell you I think you do not understand the tests. Otherwise you would understand that dreaming about tests in such way is just a futile exercise. IMHO unless you try to describe the exact implementation, methods and protocols you will never understand it. BS1116 is not a cookbook for beginners. And for the nth time, Amir was free of most of this burden because his tests returned a clear and statistically valid positive result!

Can you clarify your claim of " to see results that might be meaningful to my listening experience"? Are you addressing your preferences?

 

[esldude]

No Tim, sorry to tell you I think you do not understand the tests. Otherwise you would understand that dreaming about tests in such way is just a futile exercise. IMHO unless you try to describe the exact implementation, methods and protocols you will never understand it. BS1116 is not a cookbook for beginners. And for the nth time, Amir was free of most of this burden because his tests returned a clear and statistically valid positive result!

Can you clarify your claim of " to see results that might be meaningful to my listening experience"? Are you addressing your preferences?

Tim's point is pretty simple. If you can only just hear a small difference by isolating a couple seconds of said test, and listening to it repeatedly to finally just manage a positive result indicating you perceived a genuine very small difference, and only if you are a trained or talented listener, then it is pretty far from real world as Tim sees it. It is no longer close enough to his musical listening experience to indicate it likely would be heard under more normal conditions.

Now don't confuse Tim's position as mine (not that there is much disagreement), but it is clear to me that is what Tim is getting at in his recent responses.

 

[Phelonious Ponk]

It is the specific relation you are putting to:

Snip..... I get that they are testing for the audibility of small differences. I get that the differences are so small that the tests demand listeners trained to hear those specific differences, and listening protocols aimed at finding them. This is exactly why I think these tests are irrelevant to the normal listening experience
Reading my response will help why you are jumping to a conclusion/assumption that very small differences do not affect normal listening behaviour.
You have to think about how it relates to a cognitive listening behaviour rather than an experience, behaviour is affected by listening duration (both accumulative over multiple sessions and single duration sessions) and critical the loosely defined used I explained earlier regarding tolerance and threshold and relates to sensitivity to errors/anomalies/etc and strength of reaction to them; part of which can include satisfaction-enjoyment-or cognitive notice lack of - I touched on subtly different aspects in earlier post and thought to add this bit that I missed originally.

The same concept is potentially applicable to digital filters/dither-noise shaping/steep-slow rolloff/strong-weak alias rejection/etc; these are usually very subtle (putting aside the sin(x)/x designs on FR or NOS related designs).
Cheers
Orb

OK, I gotcha. You're right, that's my personal conclusion, my opinion. But I could be wrong and I'm open to that possibility. That's why I'd like to see tests run with experienced, but not trained listeners, to find out what is heard/perceived when no one has been told what to listen for and where to listen for it.

Tim

 

[Orb]

Tim's point is pretty simple. If you can only just hear a small difference by isolating a couple seconds of said test, and listening to it repeatedly to finally just manage a positive result indicating you perceived a genuine very small difference, and only if you are a trained or talented listener, then it is pretty far from real world as Tim sees it. It is no longer close enough to his musical listening experience to indicate it likely would be heard under more normal conditions.

Now don't confuse Tim's position as mine (not that there is much disagreement), but it is clear to me that is what Tim is getting at in his recent responses.

Not so sure it is so clear cut esldude.
Take 0.2db frequency response difference with music as an example (I appreciate such a discussion is also made complex by the pattern/whether flat-at what frequency-etc) and try achieving significant statistical pass as a non-trained listener so approach it casually in an ABX....
To pass this statistically as an ABX can still require a methodical-trained approach, and in a way that also matches the "no longer close enough to his musical listening experience"; however it will have a listening/cognitive behaviour effect on the listener regardless how one feels about this ABX test not being normal listening (so same situation as what I am saying to Tim that one cannot jump to conclusion the effect of what Amir is picking up has on listener behaviour say based on but not limited to duration/fatigue/dissonance-etc including tolerance/threshold aspects I mentioned earlier).
Late so apologies if this post is rather messy.
Cheers
Orb

 

[maxflinn]

Tim's position is pretty clear to me too and I share it.

I think all this analysing of blind-testing is pointless in the grand scheme of things. If your average audio enthusiast wants to compare a couple of products he/she can do so sighted, or level-matched and sighted, or if he /she wants to account for potential expectation bias, level matched with knowledge removed.

No need to get caught up in a positive/null/trained/untrained neurotic mess. A good old sighted then blind-listen will tell your average music lover all he/she needs to know.

 

[jkeny]

Tim's point is pretty simple. If you can only just hear a small difference by isolating a couple seconds of said test, and listening to it repeatedly to finally just manage a positive result indicating you perceived a genuine very small difference, and only if you are a trained or talented listener, then it is pretty far from real world as Tim sees it. It is no longer close enough to his musical listening experience to indicate it likely would be heard under more normal conditions.

Now don't confuse Tim's position as mine (not that there is much disagreement), but it is clear to me that is what Tim is getting at in his recent responses.

I believe Tim is missing the description that Amir gave about this. He said that in listening to the two audio files he could easily identify one from the other (I'm presuming he means listening start to finish of each file) & described what these differences were BUT for the purposes of the ABX test he DECIDED TO isolate a specific sample section in each audio file that he could be sure of identifying differences between the two samples. He focussed on these sections as being the best sections in which to identify the differences he had already heard.

This doesn't mean that these changes can ONLY be identified by isolation & repetition of a small section - it's just that Amir approached the test in this way. There are others who took this test & just went with their gut feel about which file was which & got positive results .i.e they didn't isolate & repeat a small sample section. These people were identifying differences between the files without being able to articulate what these differences were or being able to focus in on the parts of the audio files that best demonstrated these differences. This is possible for Amir because of his experience & training in these matters but ordinary people heard differences too.

So I don't believe you can claim that these differences require sample isolation & repetition to be audible.

The other aspect of what Tim says is that he hopes that music could be used as the audio signal. I believe this was done with other files from AVS - Scott's files. These initially were unmatched in level by 0.2dB but I think this was later corrected. I don't know how many ABX tests were done on these later files & what the results were?

I'm not sure if this satisfies Tim's desire for a test more relevant to general music listening?

 

[esldude]

Not so sure it is so clear cut esldude.
Take 0.2db frequency response difference with music as an example (I appreciate such a discussion is also made complex by the pattern/whether flat-at what frequency-etc) and try achieving significant statistical pass as a non-trained listener so approach it casually in an ABX....
To pass this statistically as an ABX can still require a methodical-trained approach, and in a way that also matches the "no longer close enough to his musical listening experience"; however it will have a listening/cognitive behaviour effect on the listener regardless how one feels about this ABX test not being normal listening (so same situation as what I am saying to Tim that one cannot jump to conclusion the effect of what Amir is picking up has on listener behaviour say based on but not limited to duration/fatigue/dissonance-etc including tolerance/threshold aspects I mentioned earlier).
Late so apologies if this post is rather messy.
Cheers
Orb

I think Tim's position is clear cut. As I cautioned though not my position exactly. And whether his position is correct or not is another matter. But his position isn't so hard to understand.

 

[jkeny]

Max you were one of the people who said they heard no difference when you first listened to the Winer files. Then when you did the ABX test you could identify differences but not by isolating a particular section & repeating it but more by a gut feel for which was which file. Am I right?

Now when you listen to the two files can you tell them apart, (without ABX) whereas you couldn't before?

 

[Tony Lauck]

OK, I gotcha. You're right, that's my personal conclusion, my opinion. But I could be wrong and I'm open to that possibility. That's why I'd like to see tests run with experienced, but not trained listeners, to find out what is heard/perceived when no one has been told what to listen for and where to listen for it.

Tim

The poor "untrained listener" could prove to be no other more than a victim of the marketing slogan, "Perfect sound forever." He may hear no difference the day he shells out his money, but the situation may prove different a few months down the road. This will be well after his check has cleared.

Untrained listeners are the equivalent of jay walkers wearing blindfolds and ear buds. it is fine if these people stay out of the road, but if they frequent audio forums they should expect their comeuppance. One wonders what the excuse might be for an experienced listener who is untrained. Why has he spent all this time listening and not managed to become experienced? Could he perhaps be too stubborn to ask for help? Does he lack for a mentor?

IMO, if you want to know what is heard or perceived you will have to listen critically for yourself and, if necessary train yourself. Otherwise you will be in the same state as a color blind person who is trying to figure out what the sensation of "red" feels like. You won't get the answer out of a book, not even a philosophy book (with apologies to David Chalmers).