Conclusive "Proof" that higher resolution audio sounds different

[Phelonious Ponk]

Tim,
to pass you need to have a very specific approach (tbh I feel that way with any subtle ABX) in that you listen for a cue/s somewhere in the segment, once isolated what you feel is the difference (trait depends upon what the test and source content is) then you focus in on that part/s for the ABX; ideally you want to shorten the critical focus aspect to around 1-3 seconds with the localised segment it is in maybe of 6 seconds for switching-comparison.
Done a few ABX in the past (not hifi but involving audio transmission) and only way I found to get consistent results was to ensure methodical approach to the listening along the brief lines I mention above, Amir also posted more detailed steps on AVSF on how to do this current test - maybe he still knows which post it was and can link it.

Cheers
Orb

So, basically, only those who know how to execute this very clinical listening process and do so with isolated passages where the differences are most pronounced (i.e.: audible) can hear the difference between RB and hi-res. That speaks volumes to how incredibly subtle the difference is, how unlikely it is to have any real impact on the enjoyment of recordings, and how little it should matter to most of us. With that at the bottom line, I guess I don't really care if it's IMD or additional detail. I'm never going to hear it anyway.

Should the industry still make the hi-res files available for download? Maybe. But I can think of a few things that should be way ahead of that on their to do list. Thanks, Orb. That's settled; for me anyway.

Tim

 

[rbbert]

Consider the fact (well established in behavioral sciences) that even a small difference once noticed tends to assume increasing importance (or annoyance) as time goes on. With something as subjective as music listening that may be even more so.

 

[Stereoeditor]

Here are two spectral plots of the Meridian Prime D/A headphone amplifier decoding 96kHz-sampled data representing tones at 39kHz and 41kHz, each at -6dBFS, the combined waveform peaking at 0dBFS.

The Meridian Prime has an AC power supply. I next measured a Meridian Explorer D/A headphone amplifier ($299), which is powered from the 5V USB bus.

View attachment Meex HFIM39k41k Wide.pdf
You can see that with the 39+41kHz signal at 0dBFS (above), there are audio-band products visible as high as -50dBFS. However, the oscilloscope reveals that the amplifier is starting to clip with this maximum-level signal. Reducing the level to -10dBFS (below), which is still above the level of the jangling keys in Arny Krueger's file, results in any audio-band products dropping to below -100dB and the higher-order products above the audio-band disappearing.

View attachment Meex HFIM39k41k Wide-10.pdf
I repeated these tests with a bus-powered AudioQuest Dragonfly ($149), with very similar results to the Meridian Explorer. So, given that musical signals never have ultrasonic content at anything close to 0dBFS, I think it appropriate, other than with pathologically poor-performing products, to rule out added intermodulation distortion as being the reason people can detect differences between 44.1kHz and 96kHz-sampled versions of the same music.

John Atkinson
Editor, Stereophile

 

[Phelonious Ponk]

Consider the fact (well established in behavioral sciences) that even a small difference once noticed tends to assume increasing importance (or annoyance) as time goes on. With something as subjective as music listening that may be even more so.

Let's hope, for Amir's sake, that's not the case. I'd hate for it to diminish his listening experience, and it doesn't seem likely that a lot of hi-res content is forthcoming, given what we've learned here.

Tim

 

[esldude]

Sorry to interrupt, but most of this technical discussion is over my head, but I'm very interested to know if anyone, here or elsewhere, has duplicated Amir's results. I can't hear this stuff; never can. It would be good to know that's normal, not a personal defect.

Tim

I now have. Using the approach hinted at by Amirm and described by Orb. I ended up focusing on the 2.5-4.9 second segment of the jangling keys. It ends just after a very distinct high pitched tink sound. I got 17 of 20 in Foobar. This using the original files supplied by Arny. 96/24 vs the 44/16 files.

I would listen to A then B, A then B, A then B and then X. I can't describe what I was listening to. It sounded exactly the same. However I got a feeling X was either like B or not. Chose accordingly and got 17 of 20 correct.

I made 3 attempts using Arny's original file downsampled by me in Audacity. I used shaped dither. I could not detect between those. My best result was 13 of 20.

Using Anry's ultrasonic test tone pairs there is no audible IM in what I am using.

I think you do need to be very relaxed, rested and in a quiet environment. When I had decided upon a small segment I thought sounded different, I started a test, got 7 in a row correct, the AC in the house came on (just a quiet though perceptible background noise), and my results became random. I waited until late at night with less noise and the AC off to get the 17/20 result above.

This was using inexpensive headphones and a laptop as I currently have no other equipment at my disposal.

 

[Orb]

So, basically, only those who know how to execute this very clinical listening process and do so with isolated passages where the differences are most pronounced (i.e.: audible) can hear the difference between RB and hi-res. That speaks volumes to how incredibly subtle the difference is, how unlikely it is to have any real impact on the enjoyment of recordings, and how little it should matter to most of us. With that at the bottom line, I guess I don't really care if it's IMD or additional detail. I'm never going to hear it anyway.

Should the industry still make the hi-res files available for download? Maybe. But I can think of a few things that should be way ahead of that on their to do list. Thanks, Orb. That's settled; for me anyway.

Tim

Not what I am really saying Tim
Look back and mentioned quite a few times why this approach MUST be taken for subtle ABX, this does not correlate to long term listening tolerances-thresholds-preferences so it would be wrong to take the conclusion as far as you did -IMO anyway.
To keep it brief; many unfortunately underestimate what it takes to do blind subtle comparison testing, but once know what your looking for it gets easier.

Hey kudos to you esldude
And also thanks for trying to a specific listening approach and sharing that experience.

Cheers
Orb

 

[Orb]

JA,
serious kudos and much appreciation, thanks.
The Meridian Explorer D/A headphone amplifier and AudioQuest Dragonfly should be seen as worst case examples due to what has been mentioned about well designed-engineered and to a specification/price.
No-one should be surprised how both of those behaved, which is still within performance if keys recorded/played back at right level or considerate use of test tones (which has been mentioned before means not pushing into stress testing/clipping/pushing to distortion the product).
Fits exactly with what David Griesinger presented and what we would expect from looking at both normal IM tests.

Anyway thanks again John.
Orb

 

[Phelonious Ponk]

Not what I am really saying Tim
Look back and mentioned quite a few times why this approach MUST be taken for subtle ABX, this does not correlate to long term listening tolerances-thresholds-preferences so it would be wrong to take the conclusion as far as you did -IMO anyway.
To keep it brief; many unfortunately underestimate what it takes to do blind subtle comparison testing, but once know what your looking for it gets easier.
Hey kudos to you esldude
And also thanks for trying to a specific listening approach.

Cheers
Orb

Perhaps its not what you were saying, orb, but it is really the only conclusion we can come to with what we've learned here, so I'll take it until those long term listening tests come along. Actually that shouldn't be hard. Just take a big believer in hi res, down sample some music he's familiar with, has been listening to long-term, and see if he can differentiate between the files in normal, pleasure listening.

Tim

 

[Orb]

Perhaps its not what you were saying, orb, but it is really the only conclusion we can come to with what we've learned here, so I'll take it until those long term listening tests come along. Actually that shouldn't be hard. Just take a big believer in hi res, down sample some music he's familiar with, has been listening to long-term, and see if he can differentiate between the files in normal, pleasure listening.

Tim

Err nooo.
You are reading too much into it, again subtle ABX is NOT as easy as you think, and maybe that is why so many hobbyists fail blind tests even for amps and DACs.
I repeat it will NOT tell you anything about long term listening behaviour and satisfaction-enjoyment level or tolerances-thresholds; which for most is what triggers the itch to change equipment.
All it shows is that audible differences can be consistently identified, now the question is why and widening the scope of investigation after the why; does it affect long term listening or "preferences"....
That was the crux of this thread; the narrative pushed by some is that objectively CD and hirez are the same in terms of sound produced, however Amir is showing how a trained listener can pass these "impossible" tests.
Also this does not necessarily mean hirez is much better apart from potential fact the native file is more transparent for some reason (whether that be downsampling/effects of mixing-mastering digital related effects such as implementation of dither-etc in the studio/amongst other things).
Sorry for "/" but the reason can be many for now.

Thanks
Orb

 

[Orb]

Consider the fact (well established in behavioral sciences) that even a small difference once noticed tends to assume increasing importance (or annoyance) as time goes on. With something as subjective as music listening that may be even more so.

Agreed.
Thanks
Orb

 

[amirm]

So, basically, only those who know how to execute this very clinical listening process and do so with isolated passages where the differences are most pronounced (i.e.: audible) can hear the difference between RB and hi-res. That speaks volumes to how incredibly subtle the difference is, how unlikely it is to have any real impact on the enjoyment of recordings, and how little it should matter to most of us. With that at the bottom line, I guess I don't really care if it's IMD or additional detail. I'm never going to hear it anyway.

Let's keep in mind the standard that is put upon us in such tests. To rule out "probability of chance" you need to constantly guess right. If you don't, you will have to run huge number of trials. Just miss a couple and you may have to do twice as many trials just to get back to square one!

This is one of the reasons such precision and technique is required. My goal is 100% repeatability so that I can be done with the test quickly and be put out of my misery of listening to keys jingling for the hundredth time .

 

[amirm]

Another factor: negative placebo. Because you are in a test challenge, you can easily second guess yourself out of differences that you are hearing. This is one of the unfortunate side effects of testing for the purposes of the world to see.

 

[jkeny]

Tim,
I think you may also be missing the implications of these results? As Amir has pointed out before - no longer can someone who hears a difference between RB & high-res files be dismissed with a "prove it with a blind test". For whatever reasons, yet to be fully determined, ArnyK's "keys" files & Scott's music files are providing very strong, statistical evidence that rb & high-res versions of these files are distinguishable.

Orb's comments about how difficult it is to get a positive ABX result for subtle differences should be borne in mind for all blind tests. The results have pretty much no bearing on long term listening pleasure i.e it doesn't ruin one's enjoyment of audio. It also has no bearing on whether these audible differences will or will not be determined in long term listening. My personal opinion is that long term listening would probably prove to be a better differentiator of these differences but more along the lines of a feeling that one file is just not quiet right or the other file just sounds better, without having specific sections that could be pointed to for quick A/B listening.

 

[Phelonious Ponk]

I think all of that reinforces what seems obvious, to me anyway -- these differences are insignificant to the point of meaningless. If somebody does some long-term listening and manages to show solid evidence that, over time, casual listening for pleasure can reveal what was only immediately audible through this definitively unnatural listening process, I'll be happy to be wrong. But that seems more like wishful thinking than a real possibility.

Tim

 

[jkeny]

Another factor: negative placebo. Because you are in a test challenge, you can easily second guess yourself out of differences that you are hearing. This is one of the unfortunate side effects of testing for the purposes of the world to see.

I already reported, I think, one guy relating how he couldn't hear any differences in Winer's 20 pass generational file & it was only when other forum members started to report they could that he actually made a more concerted effort & found out he could. He didn't know how but the ABX results proved again & again that he intuitively picked the correct file 100% of the time. He moved down to 1 pass & this also was 100% correct. He learned from this that his negative bias was such a strong bias that it was preventing him from hearing 20 pass differences.

I believe this to be an interesting lesson in negative bias & stress (fear of ridicule) preventing someone from hearing the differences. The ABX results table takes away any stress over the result & take away the stress of having to say what the differences being heard are which I believe can be a large factor in blocking the correct kind of listening - relaxed listening.

It's something akin to inattentional blindness in psychology - when asked to focus on a video scene & count the number of times a ball is passed from one person to another in a complex way. An Ape appearing in the background waving arms around is not noticed by many viewers. If not tasked with the counting job, the ape would have been easily noticed when watching the video.

From Wiki: "When it simply becomes impossible for one to attend to all the stimuli in a given situation, a temporary blindness effect can take place as a result; that is, individuals fail to see objects or stimuli that are unexpected and quite often salient.

Inattentional blindness also has an effect on people’s perception. There have been numerous experiments performed that demonstrate this phenomenon.[3]"

 

[jkeny]

I think all of that reinforces what seems obvious, to me anyway -- these differences are insignificant to the point of meaningless. If somebody does some long-term listening and manages to show solid evidence that, over time, casual listening for pleasure can reveal what was only immediately audible through this definitively unnatural listening process, I'll be happy to be wrong. But that seems more like wishful thinking than a real possibility.

Tim

Huh?
Isn't that what a lot of people say - high-res audio sounds different (usually better) to them over the long term? Is that not a real possibility or is it wishful thinking given that we now have these positive results confirming that there are audible differences?

I think you are missing what I & Orb are saying - the fact that it is difficult to hear this in quick A/B listening may well be a shortcoming of A/B testing & not that it is ACTUALLY as difficult to hear the difference, long term, but much more difficult to "prove". It's just that these long term reported differences have been dismissed in the past as "wishful thinking" or delusional or ....

But you are right in that the differences are not that great - the same as the differences between DACs are not that great; the differences between transports are not that great; the differences between amps are not that great. (I'm leaving speakers & rooms out of this.) But add up all these differences together - do they cumuatively amount to something of more import? I would suggest they do. Others might want to focus on speakers & rooms & that's absolutely correct too & obviously the bigger hit but it doesn't mean the rest should be dismissed - we have Ethan Winer around to do that

 

[microstrip]

Tim,
I think you may also be missing the implications of these results? As Amir has pointed out before - no longer can someone who hears a difference between RB & high-res files be dismissed with a "prove it with a blind test". For whatever reasons, yet to be fully determined, ArnyK's "keys" files & Scott's music files are providing very strong, statistical evidence that rb & high-res versions of these files are distinguishable.

Jkeny,

Excellent summary. However those loving a flat soundstage and blurred transients will want to conclude otherwise.

I would expect that all the pro-digital people would be very happy and celebrating Amir results and would maximize the importance of the differences, as these findings will allow them to challenge the pro-analogue team and their superiority claims. Astonishingly some of them prefer to minimize the importance of the perceived differences!

 

[Orb]

I think all of that reinforces what seems obvious, to me anyway -- these differences are insignificant to the point of meaningless. If somebody does some long-term listening and manages to show solid evidence that, over time, casual listening for pleasure can reveal what was only immediately audible through this definitively unnatural listening process, I'll be happy to be wrong. But that seems more like wishful thinking than a real possibility.

Tim

Tim,
it really is not that simple and to provide another example to show this.
Case in point I know of only one multiple test done with THD using ABX and blind comparison listening.
It required the listeners to be trained, spend a lot of time understanding the trait to listen for....
And still thd with statistical significant result only could be achieved down to 3% and no lower.
So going by your logic we should buy any amp including tubes with okish distortion because a trained listener could not consistently identify anything below 3% with music.

Separately Keith Howard much earlier than this also investigated thd using 4 different patterns; two of which had just over 3%.
In his own words and bearing in mind he is very used to doing these kind of listening tests:

This type of listening exercise can rapidly drive you nuts, particularly as none of the distortion patterns proved to have a gross effect on sound quality.
Nor would you expect them to, given the tiny, difficult-to-spot differences revealed by spectral analysis of the undistorted and distorted music signals. But after a number of listening sessions—kept short to keep my ears fresh—over both headphones and loudspeakers, I did feel that I could detect differences.
I found two of the recordings—of the violin and the harpsichord—particularly insightful, I surmise because both instruments have a rich enough harmonic structure to generate significant numbers of intermodulation products.

The most important finding was that none of the different patterns of nonlinearity sounded in any way preferable to the undistorted reference.

But you are very vocal about thd, so going by the fact it is hard to pass blind comparison test even at 3% thd with music using trained listeners, applying the same logic as you do to this thread means you also should not care about which amp apart from the "weakest" tube SETs used beyond spec

Cheers
Orb

 

[Phelonious Ponk]

Huh?
Isn't that what a lot of people say - high-res audio sounds different (usually better) to them over the long term? Is that not a real possibility or is it wishful thinking given that we now have these positive results confirming that there are audible differences?

I think you are missing what I & Orb are saying - the fact that it is difficult to hear this in quick A/B listening may well be a shortcoming of A/B testing & not that it is ACTUALLY as difficult to hear the difference but much more difficult to "prove". It's just that these reported differences have been dismissed in the past as "wishful thinking" or delusional or ....

But you are right in that the differences are not that great - the same as the differences between DACs are not that great; the differences between transports are not that great; the differences between amps are not that great. (I'm leaving speakers & rooms out of this.) But add up all these differences together - do they cumuatively amount to something of more import? I would suggest they do. Others might want to focus on speakers & rooms & that's absolutely correct too & obviously the bigger hit but it doesn't mean the rest should be dismissed - we have Ethan Winer around to do that

As always, John, there is more than one way to view this. One is on face value. Using what is a proven method for discerning subtle differences, the differences were only discernible when isolated and analyzed by listeners trained to identify them. The other is to assume that the very analytical process that made the differences audible to the trained listener makes them inaudible to others, but that these subtle differences they can't hear when they try, will become obvious, over time, when they don't. I get what you and Orb are saying. But it sounds like you guys are talking yourselves into it to me.

It should be easy enough to test. Find some audiophiles with hi-res files they're very familiar with from long-term listening. Downsample the files to 16/44.1. Then test the hi-res files against the down sampled ones. Don't rush them or put them in a strained situation. Let them listen as long as they want, go back and forth as much as they like. Take as many breaks as they like. See if they can beat the flip of a coin.

I don't imagine anyone is going to be doing that test.

Tim

 

[Phelonious Ponk]

Tim,
it really is not that simple and to provide another example to show this.
Case in point I know of only one multiple test done with THD using ABX and blind comparison listening.
It required the listeners to be trained, spend a lot of time understanding the trait to listen for....
And still thd with statistical significant result only could be achieved down to 3% and no lower.
So going by your logic we should buy any amp including tubes with okish distortion because a trained listener could not consistently identify anything below 3% with music

.

Well, first of all Orb, there are plenty of people in this hobby who think THD is not important. I don't really know, honestly. I just know it's very easy to avoid THD, and because of that D on the end, I think avoiding it is a pretty good idea. Redbook? A lot harder to avoid if you want to listen to much music. And all my logic is telling you is that we should pay the most attention to that which is the most audible.

Look, I'm not trying to talk you into anything, Orb. A trained listener, through a listening process so analytical that it eliminated the possibility of actually listening to the music, was able to differentiate hi-res from Redbook. The conclusion I'm reaching -- that if that's what it takes to hear them, these differences must be extremely subtle -- is based on the facts of Amir's experience as presented. Any conclusion that this result confirms that hi-res is clearly superior to RB, and that these differences would be much more easily heard during casual, long-term listening, is pure conjecture. But it's what you hear, so you believe it. I am missing the "flat soundstage and smeared transients," as micro so elegantly put it, so I've decided to go with what has actually been shown.

Tim