Conclusive "Proof" that higher resolution audio sounds different

[esldude]

Yes, running through Audio Diffmaker shows an offset of 13.53usec between Arny's 24/96 & the downsampled tracks. This is the equivalent of about half a sample - I doubt that it's audible although I do know that the just noticeable Interaural time difference is around 4usecs. But what we're talking about here is a shift of both stereo channels by 13.53usecs, not shifting one of the stereo channels.


I'm confused here about what files you are referencing - I thought the AVS files were ArnyK's files, no?

BTW, be careful with AudioDiffmaker - it seems to have some problems:
- it seemingly aligns the two files getting rid of the offset between them but introduces some shift anomaly in the first 0.177secs of the aligned file. This causes a reduced reported null value in Diffmaker.
- I brought the aligned file (16/44) from Diffmaker into Audition, inverted it & nulled it against the original 24/96 file. What should be seen is a good null for all frequencies up to 22K but you don't. What you see in the first 0.177secs is a bad null fluctuating -60dB, -70dB pretty much jumping all over as you move the pointer into the file & look at the frequency analysis plot. If you select the whole file & scan it for freq plot, you get about the same null figure as Diffmaker reports, -80dB. However, if you skip the first 0.177secs & scan the rest of the file you get a much better null of about -100 or -110dB.
- so my conclusion is that if there is an offset between files, Diffmaker makes a mess of aligning it which results in a lower null.
- one other thing is that there seems to be timing drift between the files which if not turned on in Diffmaker gives a much worse null of -30dB or so

Yes, I am aware of Diffmaker issues. So much so I never use it without confirming the info in at least one other way. In fact I didn't use diffmaker for these files yet. Dumped the jangling keys files into Audacity and lined up the nearest bits. What I found was the more downsampling done the more difference in the alignment (in this case to the nearest bit). It was more than one sample it was several samples, and I think in one case several dozen. Now in addition to that perhaps once files are aligned Diffmaker also found an additional sub-sample time difference.

When I said Arny's file nulled better it was the -100db or so null I was referring to. The other files (musical AVS files) even when lined up to the nearest bit don't null that well, and clearly show some timing drift from beginning to end. Which makes me think they have been thru an extra AD/DA loop. If you took the higher resolution file and downsampled I don't see where you would get timing drift. And you should get much better nulling at the lower frequencies.

So Amirm's comments about such testing are not changed by this. When so many fail an ABX and he passes it with large enough sample numbers as he has it shows not surprisingly that some hear better than others in some ways. But also there is no night and day difference. Plenty of us have some other tests we would like to see Amirm do of course. The problem being he is the only one who has consistently made this discernment.

It would mean even more if he can do this with files with a better cleaner provenance. I don't know what SRC Arny or the AVS files were used with, but some much cleaner ones seem to be available. I believe Amirm mentioned doing the jangling keys test in the past. So maybe he is just hearing something different most of the rest of us miss. Would be nice to figure out what. It certainly is very interesting.

 

[esldude]

No, I am not going by any delay and such. I listen to a complete segment, and then the other segment. I am not doing instantaneous switching as the whole track plays. So any delay does not play a role in my findings.

I should say that I have done no mechanical analysis of the files prior to listening to them. And even now, have not done much investigation.

Okay so that answers that question. Thanks for replying. Very interesting results.

 

[esldude]

...................
Just tried to run two of Scotts files through Diffmaker to check for offset, timing drift, etc but it reports our of memory. Any other way to do this?

Scotts files do appear to have timing drift though not very much. So have them been through an AD/DA loop??

One way to determine this is lining up both files to the nearest bit. Then of course null them. You can in this case see the null start out not so bad and slowly, but surely worsen as you go further into the recording. Which by itself may or may not effect audibility, but it does make you think it has been thru the analog realm one extra time. In Audacity you can use "Change Speed" to make very small speed changes. You can then move which part of the file nulls best from the beginning to somewhere in the middle with worse nulls at each end. Pretty clear indicator the timing is off just a bit between files. I didn't do this with Scott's files. But the null worsens as you go. It doesn't appear to be a very large change. You also can amplify and listen to the residual. If it is tinny, and thin it usually is a timing issue. Because timing differences cause a 6db per octave upward slope to frequency response in the resulting residual null.

When I did this to Scott's files the residual sounds a bit thinner, but not by much. So I suspect it is primarily a tiny difference in gain between the two files with an even tinier timing difference. Don't know this explains them being ABXable by Amirm, but it would have been nicer if the two files were more similar other than sample rate.

 

[amirm]

From what I recall, Scott's files (created by Mark) used SoundForge. They found that the resampler there reduces volume a bit to avoid clipping. So they compensated by turning up the volume (.2 db?).

 

[Orb]

IF you're right, that's way down there, Orb. Here's the article: http://people.xiph.org/~xiphmont/demo/neil-young.html. Again, whatever it is, I can't hear it. A few good masters would do my listening pleasure a lot more good than a hard drive full of hi-res files. A new, quality re-mastering movement would transform it. I hope, for Amir's sake, that when he's just listening to music for pleasure, he doesn't hear this stuff either. That would be cursed ears, not golden ones.

Tim

Just to add Tim, what I posted was a real IM measurement on the Asus integrated DAC Headphone done by JA and shown on Stereophile, while Xiph mentions a theoretical product (so not actual real product with real measurement).
As ABX are being done with headphones on most of the cases then we should put aside power amps and loudspeakers, especially as the tests in the context of this thread was passed by everyone using headphones anyway.
So we need IM measurements for DAC and headphone amp or as I showed earlier with the ASUS ideally an integrated DAC Headphone product.
Another example on Stereophile that is integrated DAC/Headphone amp and incredibly compact (so has some serious design restrictions and meaning performance will be more compromised when one also considers retail price as part of the design) is the Meridian Explorer USB D/A processor/headphone.

Fig.10 Meridian Explorer, line output, HF intermodulation spectrum, DC–30kHz, 19+20kHz at 0dBFS into 100k ohms (left channel blue, right red) (linear frequency scale).
Tested for high-frequency intermodulation (fig.10 shows the line output spectrum; the headphone output was similar, if somewhat worse), the Explorer shows only mild rejection of the imaging products at 24.1 and 25.1kHz, though actual intermodulation products are low in level. Again, the left channel is slightly worse than the right.

So again we can see weaker alias rejection causing additional tones (which can compound IM) but IM distortion sub 20khz is still nearly 80db down on the actual tone; for normal music/sounds this means it will be seriously low, for jangling keys it would be seriously lower than the keys sub 20khz sound, which if you read Dr David Griesinger presentation linked below highlights IM to be audible required sub 20khz content to be removed and the amp clipping (for a quality product and of interest is the fact we are not clipping due to using headphone amps in this instance and furthermore at normal listening levels anyway as mentioned before by Amir).
Anyway personally I would put more weight on David Griesinger investigation in 2003 than that of Xiph in this case: http://www.davidgriesinger.com/intermod.ppt
David is a pretty notable scientist/engineer in the audio world - not to be confused with another David Griesinger who is a Production Sound Mixer, pretty unfortunate both are in audio lol

Cheers
Orb

 

[esldude]

From what I recall, Scott's files (created by Mark) used SoundForge. They found that the resampler there reduces volume a bit to avoid clipping. So they compensated by turning up the volume (.2 db?).

Looked at these files a bit more. The Just My Imagination file nulled considerably better if the B version was given a .2 db boost. While Mosaic nulled better if given a .2db cut. So I am wondering about the conversion process on these.

Also the Arny Jangling Keys seem not to have used a very good SRC process. Using his original full bandwidth file and the doing the conversion in Audacity resulted in a much nicer null between the files. Everything lines up, no gain change needed, just invert and null out with much better results.

Makes me wonder if you aren't hearing sample rate conversion artifacts more so than any difference in the rates themselves.

Does not change the factors you have commented upon like how public ABX testing isn't reliable enough to rule out things or the fact you are the only participant to correctly discern a difference while many others have not. Indicating listener training/experience/ability do matter.

Would be nice if you would repeat perhaps the Jangling Keys test with different conversion used. Would be happy to convert and send you the files. But that is asking a lot. I also can see you aren't going to be able to repeat the test to everyone's satisfaction, and certainly understand your not wishing to do such.

 

[amirm]

Makes me wonder if you aren't hearing sample rate conversion artifacts more so than any difference in the rates themselves.

Just to be clear, this was my bar to clear. That is, prove that if produced in high-res, there are ways to degrade the sound on the way to 16/44.1. To the extent all of these conversions are audible to me, then we have accomplished that, and with it, created justification for getting the original mastered file.

Going beyond this and proving value of the ultrasonics, etc. is a "reach goal."

Would be nice if you would repeat perhaps the Jangling Keys test with different conversion used. Would be happy to convert and send you the files. But that is asking a lot. I also can see you aren't going to be able to repeat the test to everyone's satisfaction, and certainly understand your not wishing to do such.

Thanks. I can do a high quality sample rate conversion with the tools I have. So when I have time, I will do that and repeat the results.

 

[amirm]

OK, curiosity got the cat.

Even though the dogs are going nuts and the TV is playing, I did a resample in Audition CC with TPDF dither from 24/96 6o 16/44. Initially I had a hard time consistently telling them apart but after a couple of tries, managed to find the critical segment:

-------

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/24 20:27:41

File A: C:\Users\Amir\Music\Arnys Filter Test\keys jangling amir-converted 4416 2496.wav
File B: C:\Users\Amir\Music\Arnys Filter Test\keys jangling full band 2496.wav

20:27:41 : Test started.
20:28:07 : 00/01 100.0%
20:28:25 : 00/02 100.0%
20:28:55 : 01/03 87.5%
20:29:02 : 02/04 68.8%
20:29:12 : 03/05 50.0%
20:29:20 : 04/06 34.4%
20:29:27 : 05/07 22.7%
20:29:36 : 06/08 14.5%
20:29:44 : 07/09 9.0%
20:29:55 : 08/10 5.5%
20:30:00 : 09/11 3.3%
20:30:07 : 10/12 1.9%
20:30:16 : 11/13 1.1%
20:30:22 : 12/14 0.6%
20:30:29 : 13/15 0.4%
20:30:36 : 14/16 0.2%
20:30:41 : 15/17 0.1%
20:30:53 : 16/18 0.1%
20:31:03 : 17/19 0.0%
20:31:07 : Test finished.

----------
Total: 17/19 (0.0%)

Of course I forgot to save the segment markers and now have to start over to test with any other resamplers . This means I can't go back now and compare it to Arny's version but I am fairly certain this resampler is cleaner than the old CoolEdit he used.

 

[Orb]

Nice work esldude.
I must admit,
1st place I would look and consider potential issues would be the transparency of the downsampling for bit depth and sampling rate, which of course ties in with your response Amir in post #208.
Still feel IM is a red herring they are raising tbh (for well designed equipment and NOT stressed beyond operating spec such as near or actual clipping/played extremely loud/etc).

Amir, is it possible you are hearing the TPDF dither when downsampling?
Keith Howard ages ago did an article-investigation on dither and when it is applied more often/reiterated he did notice a very slight effect (although this should be taken as more anecdotal than scientifically thorough), appreciate TPDF is the ideal dither in this instance just thinking aloud outside that of possible downsampling artifacts/issues is all.
BTW any idea what filter (fast or slow rolloff/minimum phase-linear/etc) is used by your playback/DAC?

Cheers
Orb

 

[Stereoeditor]

Still feel IM is a red herring they are raising tbh (for well designed equipment and NOT stressed beyond operating spec such as near or actual clipping/played extremely loud/etc).

That's my feeling also. I don't test for audioband IM with high-level tones close to Nyquist with 96k sampled material because real-life music just doesn't have content like that. Other than NOS DACs or DACs using reconstruction filters optimized for time-domain behavior, as shown by the graphs shown in this thread, there isn't any misbehavior that results in audioband spuriae.

John Atkinson
Editor, Stereophile

 

[amirm]

Nice work esldude.
I must admit,
1st place I would look and consider potential issues would be the transparency of the downsampling for bit depth and sampling rate, which of course ties in with your response Amir in post #208.
Still feel IM is a red herring they are raising tbh (for well designed equipment and NOT stressed beyond operating spec such as near or actual clipping/played extremely loud/etc).

Amir, is it possible you are hearing the TPDF dither when downsampling?
Keith Howard ages ago did an article-investigation on dither and when it is applied more often/reiterated he did notice a very slight effect (although this should be taken as more anecdotal than scientifically thorough), appreciate TPDF is the ideal dither in this instance just thinking aloud outside that of possible downsampling artifacts/issues is all.
BTW any idea what filter (fast or slow rolloff/minimum phase-linear/etc) is used by your playback/DAC?

Cheers
Orb

You know as I was creating this last test I did think about effect of dither. Due to shortness of time, I just did that one conversion with TPDF. As time permits, I will repeat without dither.

John, did I read an article from you in stereophile that adding dither to a 24-bit subjectively reduced fidelity?

 

[Orb]

That's my feeling also. I don't test for audioband IM with high-level tones close to Nyquist with 96k sampled material because real-life music just doesn't have content like that. Other than NOS DACs or DACs using reconstruction filters optimized for time-domain behavior, as shown by the graphs shown in this thread, there isn't any misbehavior that results in audioband spuriae.

John Atkinson
Editor, Stereophile

And also the 19khz+20khz is part of international standards, so can appreciate why it is used.
Cheers
Orb

 

[Orb]

You know as I was creating this last test I did think about effect of dither. Due to shortness of time, I just did that one conversion with TPDF. As time permits, I will repeat without dither.

John, did I read an article from you in stereophile that adding dither to a 24-bit subjectively reduced fidelity?

That was one of Keith Howard's articles, but since then he followed this up further in a more detailed way.
Unfortunately this was only published in HiFiNews; might also been presented at one of the European AES workshops/conferences as well (cannot remember - I blame the hot weather for lowering my IQ, I must be a troll from Terry Pratchett's Discworld ).

Cheers
Orb

 

[arnyk]

I am actually thinking of proposing a paper and get some of this data on record. It is remarkable how our leading engineering/research organization has done so little to bring light to this issue.

First you might want to find someone else who can duplicate your work, Amir.

Getting someone else well known to proctor the test might be a good idea, too.

I'd say write that paper, and watch the AES review board make the identical same comments. ;-)

Several other interesting things:

(1) The basic technique and recordings involved have been around for 14 years or more and downloaded and tried by 100s if not thousands of people. No exceptional results have come forward until now.
(2) Speaking of technique, if memory serves you used Foobar2000 as your music player after ripping its design and ease of use a new termination for its digestive system.
(3) A number of other people have tried to duplicate your results, but when an monitoring system IM test was added, their results fell out of the story.

 

[arnyk]

Another challenge put forth and another set of results .

------------------------------


OK guys. You must be playing with me. You really can't tell the difference here? Here are my quick results. I skipped down to 5th generation (five times going from analog to digital and back):
=========
foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/18 06:34:21

File A: C:\Users\Amir\Music\Ethan Soundblaster\sb20x_original.wav
File B: C:\Users\Amir\Music\Ethan Soundblaster\sb20x_pass5.wav

06:34:21 : Test started.
06:35:00 : 01/01 50.0%
06:35:10 : 01/02 75.0%
06:35:21 : 01/03 87.5%
06:35:46 : 02/04 68.8%
06:35:58 : 03/05 50.0%
06:36:19 : 03/06 65.6% <----- Difference found
06:36:28 : 04/07 50.0%
06:36:40 : 05/08 36.3%
06:36:51 : 06/09 25.4%
06:37:02 : 07/10 17.2%
06:37:11 : 08/11 11.3%
06:37:25 : 09/12 7.3%
06:37:36 : 10/13 4.6%
06:37:47 : 11/14 2.9%
06:37:58 : 12/15 1.8%
06:38:10 : 13/16 1.1%
06:38:24 : 14/17 0.6%
06:38:34 : 15/18 0.4%
06:38:50 : 16/19 0.2%
06:38:58 : 17/20 0.1%
06:39:12 : 18/21 0.1%
06:39:21 : 19/22 0.0%
06:39:38 : Test finished.

----------
Total: 19/22 (0.0%)

Above I am showing my search for critical section. So when I tested the single generational loss (i.e. "most difficult") I knew what to listen for:

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/18 06:40:07

File A: C:\Users\Amir\Music\Ethan Soundblaster\sb20x_original.wav
File B: C:\Users\Amir\Music\Ethan Soundblaster\sb20x_pass1.wav

06:40:07 : Test started.
06:41:03 : 01/01 50.0%
06:41:16 : 02/02 25.0%
06:41:24 : 03/03 12.5%
06:41:33 : 04/04 6.3%
06:41:53 : 05/05 3.1%
06:42:02 : 06/06 1.6%
06:42:22 : 07/07 0.8%
06:42:34 : 08/08 0.4%
06:42:43 : 09/09 0.2%
06:42:56 : 10/10 0.1%
06:43:08 : 11/11 0.0%
06:43:16 : Test finished.

----------
Total: 11/11 (0.0%)

So I started at 6:34 AM and finished at 6:43 AM for a total of 9 minutes. What I am supposed to do for the rest of the hour Arny?

By the way, I am traveling and the headphone I have is my Shure IEM. So now we have results across three different headphones.

Arny do you honestly not hear the difference???

By the way the techno clip Ethan picked is not ideally suited for this type of testing yet the audible difference is there.

Looks like typical biting of hand that feeds...

What I don't see mentioned is the fact that Ethan's Pass1 files differ from the origional file in overall level by over 0.4 dB, and that I provided corrected files for Amir to test with. I don't recall seeing any results from those tests.

 

[arnyk]

Thanks Orb. My comment in my post was directed at Arny. As the bandleader on what is audible and what is not, he just can't have such poor listening ability. To make matters worse, he keeps posting these challenges as if the others before it had not backfired on him.

First off Amir, you've made this all about you, but I haven't made this all about me. For one thing good scientists know that its not all about them.

The findings and tests I've been talking about go back 14 years or more (when my hearing was a lot better FWIW) and have been duplicated by a great many people.

Right now we've got one exceptional result.

It appears that you have biased the members of this forum by witholding certain critical information, such as how many people at AVS tried and failed, the existence of a set of subjective IM test files, and the fact that AFAIK just about everybody who first reported positive results later on found audible IM in their systems. At least I'm about half ways through reviewing this thread, and don't see hide nor hair of any of the critical factual information that has come to light.

 

[arnyk]

And also the 19khz+20khz is part of international standards, so can appreciate why it is used.
Cheers
Orb

What we found at AVS is that IM varies across the ultrasonic range. The subjective tests I set up used a series of twin tones, about 4 KHz apart, across the whole range from 22 KHz to 44 KHz. Different pairs sometimes give different results, and there's apparently no rule that says that the tests based on the highest frequency ranges are the toughest tests.

Furthermore, one headphone amp that was reported to fail pretty severely seems to be found in a highly regarded higher end BD player with a palindromic name.

I encourage people to use the test files that have the IM test tones attached, and make sure that they don't adjust levels between running the test tones and trying the actual keys jangling material.

 

[arnyk]

My thanks also for performing blind testing on this matter, Amir. Finally, it would appear, someone has put the Meyer-Moran test in its proper context.

As for Arny Krueger, he never gives an inch, no matter how much evidence mounts up proving him incorrect. I first encountered him in the Usenet days when he was publicly criticizing me as an "ex-car mechanic with no formal education in audio engineering." I politely corrected him, explaining that I did indeed have a formal education that included audio engineering and that he was confusing me with then-Stereophile publisher Larry Archibald, who was indeed once a car mechanic (though Larry also had a degree in English from Harvard). Rather than acknowledge his error, Arny doubled-down, insisted that his statement was correct, and has continued in that vein ever since. :-(

Intersting how much can be made out of casual comments that are decades old. The comment was made in error at the time. For the record John, I do understand the difference in career paths between you and Larry, and at this point I'd say that calling you a car mechanic impugns an honest profession. ;-)

 

[Orb]

What we found at AVS is that IM varies across the ultrasonic range. The subjective tests I set up used a series of twin tones, about 4 KHz apart, across the whole range from 22 KHz to 44 KHz. Different pairs sometimes give different results, and there's apparently no rule that says that the tests based on the highest frequency ranges are the toughest tests.

Furthermore, one headphone amp that was reported to fail pretty severely seems to be found in a highly regarded higher end BD player with a palindromic name.

I encourage people to use the test files that have the IM test tones attached, and make sure that they don't adjust levels between running the test tones and trying the actual keys jangling material.

Oh I really do understand how it works Arny, btw most of the tests proposed into looking at IM forget musical content (putting keys to the side) is at least 40db-70db lower at ultrasonic levels, this is further compounded that IM will be at least 60-80db lower in sub 20khz region for a good product.
Another aspect compounding this is to hear the sub 20khz IM one has to really turn the volume up, so hence you are now seriously stress testing an amp that has near full ultrasonic energy while the IM spectra amplitude is much lower than even normal music (again lets take keys out of this for now).
Everyone I have read on various forums using that test had to turn up the volume really loud; meaning they are seriously stress testing/or clipping the amp/or pushing it into non-linear behaviour due to the high crest factor-energy difference between audible sub 20khz and the extreme ultrasonic tones (usually close to or at 0dbfs).

BTW which product headphone amp failed your measurements and are those measurements on the web; am seriously interested to know.
Furthermore did that headphone pass the 19+20khz standard when pushed to same limit as higher tones therefore extreme loud volumes?

The tests you do seem (I appreciate IMO) to comply with Dr David Griesinger conclusions that to hear IMD caused by ultrasonics even from keys one has to push a good/well designed product into clipping/non-linear behaviour.
As I showed earlier with the Asus integrated DAC-Headphone it has exceptional low IM measurements (albeit using organisation international standards test measurement for IM rather than ones proprosed on forums).
Asus is a good example to use because they are one of the worlds largest PC component manufacturers out there including critically motherboards with integrated audio/lan/usb/etc.

Just to add though, should be noted Amir and maybe a couple of others (appreciate again not conclusive from a mass test but one meaning some seem to be identifying differences that were not meant to exist so one has to look at more than just IM) also did the ABX with music, which one would have to accept would not have the unusual ultrasonic amplitude/energy such as jangling keys or near 0dbfs dual ultrasonic tones.
While we cannot say this for all, we do know Amir mentions he did playback closer to normal listening levels than extreme loud/stress test component level.

Edit:
Just to add, I am not categorically saying IM is not the cause, but it is very unlikely compared to other possibilities and especially if those doing the ABX are using well designed products and listening in a way to not push components into clipping or non-linear behaviour.
Thanks
Orb

 

[arnyk]

does anyone know of ABX software that I could try this out with on a mac?

For the PC: Foobar2000 with the ABX plug in, a free download that google will point you to in a heartbeat.

For the Mac, there are several the best of which may be the one that comes with Apple's "Mastered for iTunes" package, another readily googlable free download.

The current keys jangling file with test tones at the end are at: https://www.dropbox.com/s/ylrjezd7vc11leo/keys jangling with test tones.zip

The last 4.1 seconds of both files should sound the same: 1 second of a low 4 KHz test tone, a click, silence, a click, silence, a click silence, and a click and 0.1 seconds of silence. If this area does not sound the same in both files, the probable cause is audible IM in your monitoring path.