Conclusive "Proof" that higher resolution audio sounds different

[sasully]

John, you said 'long-term sighted comparisons can reveal audible differences not revealed by controlled ABX testing'

Not - long term listening reveals more differences than improperly constructed blind testing

Indeed. Not to mention that the claim about long-term sighted listening vs 'controlled ABX' has never been demonstrated. (Tom Nousaine wrote up at least one 'long term sighted listening' test that did *not* demonstrate the benefits.)

Long term sighted listening and controlled tests don't preclude each other. In fact, the purported benefits of LTSL would need to be validated by a DBT, in any credible research.

 

[Phelonious Ponk]

John, you said 'long-term sighted comparisons can reveal audible differences not revealed by controlled ABX testing'

Not - long term listening reveals more differences than improperly constructed blind testing



But clearly the more variables you remove, the more valid the test. Why then would sighted tests be better than any form of blind tests in terms of identifying differences?

Max, this is excellent, as have been your last several posts. For my part, I'm relaxed, checking in from time to time but not paying particular attention to the details of this thread because I see that you are keeping the contradictions exposed and the fallacies countered. Really, anything I'd have to say would be redundant. Thank you.

Reminds me of a time when I was sitting in a corporate conference room; a gaggle of young MBAs had all just commented broadly on a brand refresh. None of them, of course, knew anything about design or had a clue what they were going on about, and none of them had asked the opinion of the one person in the room who did, the rather brilliant Art Director who made my job easier on a daily basis. When the MBAs finished pontificating one of them finally turned to me instead of the Art Director - probably only because there was an S in front of my VP - and asked, "Tim, what do you think?" I turned to the Art Director and said, "I don't know. Bonnie, what do I think?"

She was brilliant; I didn't even have to speak.

Tim

 

[sasully]

So we shouldn't trust our ears in conclusion?

Not unreservedly, no, we shouldn't.

Is that such a difficult thing to accept?

 

[sasully]

Poppy Crumb

correction: Crum

 

[sasully]

You might try not to generalize about folks hearing John.

I do not distrust my ears, however, your own research paper (and Orbs) (Perpestives on...)referenced above spells it out in black and white that "there is clear evid ence that cognitive processes such as attention, memory,emotion, and motivation impact the auditory processing of sound."

That's not about hearing, its about the ear/brain interface,

i would say this is a distinction without meaning, for listeners. There is no 'hearing' without the 'ear/brain interface'. You can't subjectively separate them, you can't 'trust your ears', you can only trust your 'ear/brain interface' ...your auditory system.

 

[jkeny]

ABX testing is a type of testing which has strengths in revealing particular aspects within the sound field. Some differences are better revealed in a less strained, less focused, more holistic listening. When switching rapidly between two samples our ears take some time to settle to particular aspects of the new sound field. Our perceptions are just built like that - as I said, we don't make good human instruments.

But remember most of the cries for "proof" usually are for blind tests, not ABX tests. So my point about half arsed blind tests still stand

 

[sasully]

ABX testing is a type of testing which has strengths in revealing particular aspects within the sound field. Some differences are better revealed in a less strained, less focused, more holistic listening. When switching rapidly between two samples our ears take some time to settle to particular aspects of the new sound field. Our perceptions are just built like that - as I said, we don't make good human instruments.

If you mean, using short samples is worse for discrimination of audible difference than long ones, perceptual testing science doesn't support you. You are, of course, still free to use long samples in an ABX if you think it will help you.

If you mean, minimizing switching interval between the end of A and the start of B is worse for discrimination of difference than long ones, perceptual testing science doesn't support you. You are, of course, free to use a long switching interval, in an ABX if you think it will help you (though you won't be able to use fooABX for that).

The fundamental requirements are 'blinding' and 'level matching'. The other recommendations -- short samples, minimal switching interval -- are there to *increase* a subject's chances of detecting a real but subtle difference. NB if I read Amir's claims right, he was focusing on short segments of the files in question.


And in any case, your claim that 'these sound different' after your long, relaxed, sighted, holistic listening, should be supportable by a subsequent DBT . After all it means you have 'trained' yourself to hear a difference between the sound fields, right? So why would a subsequent ABX (or other DBT) override that training? And if you don't use some sort of subsequent DBT, how would you validate that the difference you perceived in long, relaxed, sighted, holistic listening, was real and not just another example of cognitive bias?

But remember most of the cries for "proof" usually are for blind tests, not ABX tests. So my point about half arsed blind tests still stand

ABX-type sensory tests are called for when difference is claimed; preference claims are tested with different blind protocols.

 

[amirm]

A stance which could quickly lead to nonsense , depending on how 'best' is defined. E.g., if 'best' means the format that most accurately captures the highest frequencies at the deepest bitdepth, the 'best' , with current tech, would be 192kHz/24bit. Now go peddle that claim to, say, Dan Lavry or Bruno Putzys and see what they say.

I was clear in my reference. Here is your post again which I was responding to:

Lossy encoding will never be acceptable as an archiving format. But as a consumer format, it is *effectively* 'transparent to most listeners, if done right.

The best is not defined as "transparent to most listeners." MP3 at 128 kbps is transparent to most listeners. That doesn't make it the best by any stretch.

It is an ideal way to define 'best' for those peddling 'audiophile' hardware, though. Whatever measures best is best! (Except when it's vinyl, of course. Vinyl is always best. )

Looks like you have forgotten what you had written. There was no talk of "measure best," "deepest bit depth," etc. You said your test of transparency is what is "effectively transparent to most people" and I said that is a good enough standard, not what is best. What is best is one that is transparent for everyone. If I can for example hear a degradation, then that version of the file by definition is not the best. That you can't or a million others can't, is neither here nor there.

 

[amirm]

The fundamental requirements are 'blinding' and 'level matching'. The other recommendations -- short samples, minimal switching interval -- are there to *increase* a subject's chances of detecting a real but subtle difference. NB if I read Amir's claims right, he was focusing on short segments of the files in question.

Do you work in the PR department? I made no "claim." I showed data. Data is not a claim. Data is data. I was told to try to pass an ABX test using foobar2000 and I did. That is not a claim. You can't put a negative spin on data by calling it a "claim." By your notion I could create a cloud of doubt on Meyer and Moran test and for that matter any ABX test by saying it was a "claim." So let's not play clever word games.

 

[jkeny]

If you mean, using short samples is worse for discrimination of audible difference than long ones, perceptual testing science doesn't support you. You are, of course, still free to use long samples in an ABX if you think it will help you.

No, I don't mean this

If you mean, minimizing switching interval between the end of A and the start of B is worse for discrimination of difference than long ones, perceptual testing science doesn't support you. You are, of course, free to use a long switching interval, in an ABX if you think it will help you (though you won't be able to use fooABX for that).

No, I don't mean this

The fundamental requirements are 'blinding' and 'level matching'. The other recommendations -- short samples, minimal switching interval -- are there to *increase* a subject's chances of detecting a real but subtle difference. NB if I read Amir's claims right, he was focusing on short segments of the files in question.

Indeed & your premise is that fast switching between samples "*increase* a subject's chances of detecting a real but subtle difference." I'm saying that there are other differences that don't fit this model - this ABX approach has strengths for spotting differences in frequency & amplitude. There's more to the music than this.

And in any case, your claim that 'these sound different' after your long, relaxed, sighted, holistic listening, should be supportable by a subsequent DBT .

No, that's a circular argument - you are saying that the only way to verify/prove LTSL is valid is by using the very test I'm saying is not suitable for revealing the differences that LTSL can reveal. I already said some type of longitudinal study is what's required - one in which some people used ABX to choose their system & some people who used LTSL. Let them live with their choice of a while & then to poll these people over time to see if they are still feel their choice was the correct one.

After all it means you have 'trained' yourself to hear a difference between the sound fields, right? So why would a subsequent ABX (or other DBT) override that training? And if you don't use some sort of subsequent DBT, how would you validate that the difference you perceived in long, relaxed, sighted, holistic listening, was real and not just another example of cognitive bias?

ABX-type sensory tests are called for when difference is claimed; preference claims are tested with different blind protocols.

As somebody already said, they didn't know of anybody who had actually chosen a system based on their ABX or blind results.

 

[jkeny]

Indeed. Not to mention that the claim about long-term sighted listening vs 'controlled ABX' has never been demonstrated. (Tom Nousaine wrote up at least one 'long term sighted listening' test that did *not* demonstrate the benefits.)

Long term sighted listening and controlled tests don't preclude each other. In fact, the purported benefits of LTSL would need to be validated by a DBT, in any credible research.

Thanks for the link. Interesting article. The one issue I have with both comparison of LTL(Long Term Listening) Vs ABX is that according to the report they trained the listeners prior to the ABX but didn't use any pre-conditioning for the LTL. As a follow-up control I would have used the trained listeners in LTL test to see if they could NOW identify the distortion box in a LTL - in other words was it the training that differentiated the results?

I might also have tried to test something other than frequency or amplitude differences!

 

[esldude]

I posted this in reply to you earlier in the thread
"Yes, running through Audio Diffmaker shows an offset of 13.53usec between Arny's 24/96 & the downsampled tracks. This is the equivalent of about half a sample - I doubt that it's audible although I do know that the just noticeable Interaural time difference is around 4usecs. But what we're talking about here is a shift of both stereo channels by 13.53usecs, not shifting one of the stereo channels. "

"- one other thing is that there seems to be timing drift between the files which if not turned on in Diffmaker gives a much worse null of -30dB or so"

Do both of these factors explain the difference in the graphs that you have shown?
Would these differences would be audible?
- a half sample offset of the resampled file
- a timing drift between the two files

Anyone can prove this for themselves:
- download Dfiffmaker
- run a difference between ArnyK's 24/96 file & his downsampled 16/44 file
- do it with different settings in Diffmaker "enable time alignment" turned on & again with it turned off
- compare the nulling values between these two runs

For further checking
- There are intermediate files created by Diffmaker which are the aligned versions of the 16/44 file
- bring that into Adobe Audition & null it against the original 24/96 file
- you will see that if you skip the first 0.187 seconds of the file (some Diffmaker glitch in the aligned file), the nulling is around -100dB for all frequencies <20KHz
- this might be about 10dB worse than the nulling you get when you run resampling using a modern resampler
- Did ArnyK's report that there is a 0.2dB difference between the original file & downsampled files & would this explain the 10dB null diff?

Diffmaker sometimes does wonderful things almost beyond belief in its ability to compare files. And then sometimes seems to get quite confused. So I view it with suspicion until I can more or less confirm things another way. It does not appear there is anything like a 13 usec offset. If there is one it looks to be about 3 usec if looked at in other ways than Diffmaker.

Now a subsample timing offset still might produce the reconstructed file almost perfectly. But it will corrupt nulling in digital audio editors. A timing drift would be similar. The files will differ though likely in a manner not audible.

Beyond trying to do forensics further on an old file from years ago, seems sensible to me to bypass the issue. Resample the full bandwidth file with a good modern resampler. No alignment, timing, level or other problems. You get great nulls easy as pie. Do your testing with those. As there is clearly some oddities about Arny's resampling just resample anew.

 

[esldude]

But it can also be H/W one when considering the complete chain at both studio and also at home; both can "mess" around with source file.
One aspect still not ruled out is the dither that a few of us went into a bit detail earlier on in this thread (including aspects of TPDF and other types, when used-implications,etc).
Esldude out of curiosity which dither did you use when downsampling-decimation (sampling rate and bit depth)?

Thanks
Orb

I used shaped dither. I later redid it with TPDF dither. I did not ABX test the TPDF file yet. But getting a residual and listening to the amplified result you still get noise with no evidence of the jangling keys.

 

[jkeny]

Diffmaker sometimes does wonderful things almost beyond belief in its ability to compare files. And then sometimes seems to get quite confused. So I view it with suspicion until I can more or less confirm things another way. It does not appear there is anything like a 13 usec offset. If there is one it looks to be about 3 usec if looked at in other ways than Diffmaker.

I agree with you about Diffmaker & try to verify it's results independently. What other way did you use to measure the offset?

Now a subsample timing offset still might produce the reconstructed file almost perfectly. But it will corrupt nulling in digital audio editors. A timing drift would be similar. The files will differ though likely in a manner not audible.

Agreed & could it also produce the type of graphs you showed for the old files?

Beyond trying to do forensics further on an old file from years ago, seems sensible to me to bypass the issue. Resample the full bandwidth file with a good modern resampler. No alignment, timing, level or other problems. You get great nulls easy as pie. Do your testing with those. As there is clearly some oddities about Arny's resampling just resample anew.

I agree to try with your own, modern, blameless resampler is a good idea to verify the test results - something that Amir already did & posted positive results, yet again. I doubt from what you just said above that you can declare that the original files are flawed in some way & that this is the cause of the audible differences.

I think you got into discussion with Amir about what dither he used? AFAIR, he used standard TPDF & you used something less common? In some ways, this further proves his point - deliver the high-res files & let us downsample them with whatever dither & resampler we choose - why pay for others resampling or dither mistakes.

I would be interested in further analysis of the original files to possibly understand the limits of our hearing, the problems with resampling, etc. These to me seem to be fruitful areas of investigation.

 

[FrantzM]

Not unreservedly, no, we shouldn't.

Is that such a difficult thing to accept?

not at all.

 

[maxflinn]

Max, this is excellent, as have been your last several posts. For my part, I'm relaxed, checking in from time to time but not paying particular attention to the details of this thread because I see that you are keeping the contradictions exposed and the fallacies countered. Really, anything I'd have to say would be redundant. Thank you.

Thanks Tim. I hope I've been both logical and polite

Reminds me of a time when I was sitting in a corporate conference room; a gaggle of young MBAs had all just commented broadly on a brand refresh. None of them, of course, knew anything about design or had a clue what they were going on about, and none of them had asked the opinion of the one person in the room who did, the rather brilliant Art Director who made my job easier on a daily basis. When the MBAs finished pontificating one of them finally turned to me instead of the Art Director - probably only because there was an S in front of my VP - and asked, "Tim, what do you think?" I turned to the Art Director and said, "I don't know. Bonnie, what do I think?"

She was brilliant; I didn't even have to speak.

Tim

Haha, I guess they felt that they were taking the initiative

 

[maxflinn]

Indeed. Not to mention that the claim about long-term sighted listening vs 'controlled ABX' has never been demonstrated. (Tom Nousaine wrote up at least one 'long term sighted listening' test that did *not* demonstrate the benefits.)

Not demonstrated yet many still believe. Common in HiFi it seems.

Long term sighted listening and controlled tests don't preclude each other. In fact, the purported benefits of LTSL would need to be validated by a DBT, in any credible research.

Agreed.

 

[Phelonious Ponk]

Thanks Tim. I hope I've been both logical and polite

Absolutely.

Haha, I guess they felt that they were taking the initiative

Something like that. Everyone has an opinion. It's not a problem until you run into those who endeavor discredit any data that disagrees with theirs.

Tim

 

[JackD201]

Ok guys, how do you decide WHAT to ABX?

 

[Don Hills]

correction: Crum

Autocorrect and auto-tune. Both crimes agaisnt humanity.