Conclusive "Proof" that higher resolution audio sounds different

[maxflinn]

I may have missed it, but has anyone actually said that they don't accept the positive results from the ABX tests, posted here and in JK's other thread? Personally, I think they prove that differences between the files in question are audible.

The problem as I see it is that any null result from any other test, no matter how many variables were accounted for do not seem to be accepted by many, namely John, who states that most every such test was half-arsed.

Yet, the least reliable form of comparative testing for differences- long-term sighted, is said by John to be better for allowing listeners to identify some differences that even the most stringent double-blind ABX tests does not, even though there is zero evidence of this, no variables whatsoever are accounted for and the issue of the fallibility of audio memory is yet one more variable to add on top.

So long-term - no variables accounted for = best, or at least better in some ways, allegedly.

Short-term Blind A/B or ABX - Many if not all variables accounted for = worst (unless differences are found, then it's valid)

Is it only me seeing all these contradictions?

 

[Orb]

Amir,
interesting posts by RayDunzl and JJ over on AVSF in the associated thread.
Not a member there so cannot discuss but maybe some here may know anyway.
But what signal level/DB is required to create audible to the listener "beats" (really envelope modulation I think) combined from ultrasonic and subsonic tone?
I would think it needs to be such a level the electronics would be clipping, and proably creating IMD products by that point anyway, interesting anyway (highly likely not relevant but would be cool if someone tested signal-db levels required to achieve audibility or the maths showing loudness-energy required) due to its nature-behaviour that does not measure as heard and aspect I totally forgot about.

Cheers
Orb

 

[jkeny]

I may have missed it, but has anyone actually said that they don't accept the positive results from the ABX tests, posted here and in JK's other thread? Personally, I think they prove that differences between the files in question are audible.

Correct but, in the correct spirit of scientific investigation, the positive results are being investigated. The way the scientific principle works is by trying to find flaws in the test - it's nothing personal against Amir or others, it's just science experiments. So far nothing concrete has been found. Could the test have been better constructed - yes. One obvious change - including controls to verify the tests ability to return known positive & known negative results.

The problem as I see it is that any null result from any other test, no matter how many variables were accounted for do not seem to be accepted by many, namely John, who states that most every such test was half-arsed.

Firstly, again in the scientific method, a null test is no result - it's the equivalent of no verdict in law. Secondly, the same forensic investigation is used on such tests & if flaws are found then they are itemised. This is what I'm doing & the "tests" of which you speak are found very flawed. So why should we accept their "results" as evidence of anything?

Yet, the least reliable form of comparative testing for differences- long-term sighted,

have you evidence which proves your claim?

is said by John to be better for allowing listeners to identify some differences that even the most stringent double-blind ABX tests does not, even though there is zero evidence of this,

There is more than me stating this & yes, we have always stated that there is no hard evidence, just evidence born of experience - it is my opinion & others too - we are being honest here

no variables whatsoever are accounted for and the issue of the fallibility of audio memory is yet one more variable to add on top.

Sure there are lots of uncontrolled variables but long term averaging can reduce the significance of these variables? Whereas with a short, once off, so called "blind test" with uncontrolled variables - there is no averaging

So long-term - no variables accounted for = best, or at least better in some ways, allegedly.

Yes, better in some areas - the areas that are more important for our long term pleasure with our audio listening

Short-term Blind A/B or ABX - Many if not all variables accounted for = worst (unless differences are found, then it's valid)

Yes, not so good at determining how we will like a device over the long term. Differences in ABX tells us nothing about preference, as Amir has said, many times

Is it only me seeing all these contradictions?

Yes there are many contradictions in what you say & no, I don't think you see them which might be the cause of your confusion

 

[Whatmore]

I may have missed it, but has anyone actually said that they don't accept the positive results from the ABX tests, posted here and in JK's other thread? Personally, I think they prove that differences between the files in question are audible.

The problem as I see it is that any null result from any other test, no matter how many variables were accounted for do not seem to be accepted by many, namely John, who states that most every such test was half-arsed.

Yet, the least reliable form of comparative testing for differences- long-term sighted, is said by John to be better for allowing listeners to identify some differences that even the most stringent double-blind ABX tests does not, even though there is zero evidence of this, no variables whatsoever are accounted for and the issue of the fallibility of audio memory is yet one more variable to add on top.

So long-term - no variables accounted for = best, or at least better in some ways, allegedly.

Short-term Blind A/B or ABX - Many if not all variables accounted for = worst (unless differences are found, then it's valid)

Is it only me seeing all these contradictions?

bingo!

 

[Whatmore]

Firstly, again in the scientific method, a null test is no result - it's the equivalent of no verdict in law.

What about Type 1 errors?

Also you haven't answered my question in post #504. Or if you have can you please try again because I don't see an answer?

 

[jkeny]

What about Type 1 errors?

If the tests are not valid, & nobody has so far shown one of these tests as valid, despite numerous calls do so, then it doesn't even get to be considered statistically.

Come on, guys, give it up - when asked for a valid test, no reply; when asked for a list of "competently designed" DACs, no reply

Also you haven't answered my question in post #504. Or if you have can you please try again because I don't see an answer?

I answered it with reality, not with hypotheticals. We can all create hypotheticals - it's irrelevant to the real world.
Come on, guys, give it up - I'm off - spent long enough sparring with you guys!

 

[Whatmore]

If the tests are not valid, & nobody has so far shown one of these tests as valid, then it doesn't even get to be considered statistically. Come on, guys, give it up


I answered it with reality, not with hypotheticals. We can all create hypotheticals - it's irrelevant to the real world.
Come on,guys, give it up - I'm off - spent long enough sparring with you guys!

let's imagine a world without hypotheticals then

Are you seriously telling me hand on heart, that a test where you shut your eyes is irrelevant to the real world?

 

[Phelonious Ponk]

Valid means valid to be called a blind test, Tim. If you want to call it something else, fine - I suggest you call it a "half-arsed, pretend test"
If you challenge others to present "proof" that they heard something then stop being disingenuous - suggest a valid test - not some makie-uppie test that changes depending on what's available.
If you can't meet sufficient standards to ensure that it's a valid test & that it has a chance of proving something then fine call it an anecdotal report. So in other words demand another anecdotal report instead of the pretense of demanding a valid test

Stop all the pretense, it's gone on long enough, for god's sake. For years now these half-arsed blind tests have been presented as "proof" that there is no difference .....blah,blah

I'm calling you on it - show me a forum organised valid blind test.

Well john, if you're going to call me on it, you're going to have to show where I've ever presented the result of an informal, or any singular blind test as proof of anything. No single blind test, under no set of controls, constitutes proof, and to my knowledge none of this navel-gazing minutia audiophiles wring their hands over has been reviewed, challenged, re-tested and confirmed to anywhere near the level required for "proof." Probably because no one with the chops and the resources cares.

Have I ever challenged you to test yourself, blind, to verify what you heard sighted? Yes. That's because I believed, and I still believe, that given the same controls (or lack thereof) removing opportunity for bias beats inviting it. That if you change just one variable (sight) and get a different result, that variable was probably (which is different from proof) what caused the difference. You disagree. I think that's established, and I've respectfully invited you to agree to disagree a couple of times already. You can't seem to do that. And you can't seem to resist getting aggressive and personal when you don't get the last word, so take it now, but please use it to summarize your position, not mine. Thank you.

Tim

 

[Stereoeditor]

Most people got so used to debating negative ABX results during tens of years that they do not know anymore how to deal with a positive result ...

Amen to that thought!

John Atkinson
Editor, Stereophile

 

[maxflinn]

Firstly, again in the scientific method, a null test is no result - it's the equivalent of no verdict in law. Secondly, the same forensic investigation is used on such tests & if flaws are found then they are itemised. This is what I'm doing & the "tests" of which you speak are found very flawed. So why should we accept their "results" as evidence of anything?

I don't believe I've mentioned any specific tests that resulted in a null, bar Vital's.

have you evidence which proves your claim?

You've outlined the evidence yourself by declaring all the things that are necessary for a test to be valid. long-term sighted listening does not involve any of these things, so by default based on your own strict criteria for blind-testing, long-term sighted has to be the least reliable.

There is more than me stating this & yes, we have always stated that there is no hard evidence

Indeed, and there is no other kind of evidence either, nor can any logic be put forward that may explain how long-term sighted listening can be better at differentiating, therefore the notion that long-term listening is more valid than all other means of testing, or a better way to find some differences that cannot be identified using any other method is fanciful at best. Why do you keep stating it?

Yes, better in some areas - the areas that are more important for our long term pleasure with our audio listening

But as already demonstrated by us both, you inadvertently, it is not better in any way as a means for differentiation due to the fact that it accounts for none of the variables requiring accountability for valid ABX testing, such as outlined by yourself, many times.

A continued belief, in light of this, that long-term sighted testing has validity as a means of differentiating, can only be faith-based!

Blind testing or any other kind of testing does not deal with 'pleasure'.

Yes, not so good at determining how we will like a device over the long term. Differences in ABX tells us nothing about preference, as Amir has said, many times

Blind-testing is a means used to find differences, not to find what you like or prefer.

Yes there are many contradictions in what you say & no, I don't think you see them which might be the cause of your confusion

I'm not so sure that it's me contradicting myself, and I'm not confused either. Perhaps you are

 

[FrantzM]

John


What I find interesting in your arguments (and you are not alone with that line of reasoning) is the insistence and stridency about the non validity of forums blind tests. THis is a classic argumentation fallacy. I will spare you the name of it but no one termed those test conclusive or scirntifc they did however show that the confidence that we, audiophiles have on our hearing acuity or ability is not based on provable facts.

So let me try to see if I understand your position here. A few questions, you are of course free not to answer any of those but in the name of civility and since after all you all, participating in a Discussion Forum, your input/replies/answers would be much appreciated.

There goes my question
Am I correct in reading this point from you: Casual ABX are not valid as scientific proofs? (BTW No disagreements there).

How do you explain this:
With knowledge: Night and Day.
Knowledge removed or withheld: Not able to perceive differences or even confusion.

microstrip

The evaluation of the BADA were conducted sighted. i tried however to match the level as carefully as possible. It seems to me as one of the best DAC I have heard regardless of price. A friend of mine also have it and we conducted some tests with Ayre, MSB, MBL, EMM Lab, ARC and it was to our ears and eyes the better. match level within 1 dB with test tones.

As with many electronics I do find their signature unmistakable once you learn to listen to what make them different. Not all the time but in my case most of the time.

@everyone else

As an aside We had an eminent scientist here on the WBF. We have had quite a few emphasis on "had" as we seem to have run them out so strident some were against some of their (scientific) findings or conclusions. This person, Dr Sean Olive and others developed a software that would train people to become better listener . Strangely enough not too many here reported using it, while they continue to bellow that they "trust their ears". I have been using it for a while actually, I also believe one needs to re-train oneself from time to time to keep the (learned) ability.
It would help people distinguish quite a few artifacts even with knowledge removed .

CLICK HERE TO DOWNLOAD

 

[amirm]

Most people got so used to debating negative ABX results during tens of years that they do not know anymore how to deal with a positive result ...

Indeed. On AVS, we had a person who passed the test and then said the results didn't mean anything. He said he spent an hour searching for a specific spot and then found it. I told him that is what he was supposed to do, i.e. learn how to tell the difference. He got super upset at me for saying that!

 

[Phelonious Ponk]

Indeed. On AVS, we had a person who passed the test and then said the results didn't mean anything. He said he spent an hour searching for a specific spot and then found it. I told him that is what he was supposed to do, i.e. learn how to tell the difference. He got super upset at me for saying that!

Perhaps, but I don't think that has happened here, has it Amir? In spite of the fact that we have very limited, casual ABX here, with few of the controls being demanded by John, even in spite of your playful, exaggeration in the thread title, haven't WBF "objectivists universally accepted your results? So who are these "most people" who can't deal with a positive result?

Tim

 

[esldude]

Post removed upon reflection.

 

[Johnny Vinyl]

removed

I still see 26 pages!

 

[jkeny]

John


What I find interesting in your arguments (and you are not alone with that line of reasoning) is the insistence and stridency about the non validity of forums blind tests.

This seems to be the misunderstanding with what I'm saying. I'm saying that casual blind tests are no better, nor worse than sighted blind tests. I use both & have been involved in both & quite honestly neither one nor the other is better. So, to me they are both anecdotal. I do favour long-term listening, however.

THis is a classic argumentation fallacy. I will spare you the name of it but no one termed those test conclusive or scirntifc they did however show that the confidence that we, audiophiles have on our hearing acuity or ability is not based on provable facts.

Ah, so you are saying one proves something that the other doesn't. So you do hold one test superior to another whereas I hold them both at the same level. Yes, if some difference I find too close to call, I will use a blind test to check it. Does this make blind testing better, no - it's just another angle to test it from. I could just as easily have used long term listening & formed an opinion over time about the differences
Maybe what I'm reacting to is the usual tactics that apply on forums where what people relate that they hear is challenged with asking for a blind test. I find this of no more use than asking them for another anecdote.

So let me try to see if I understand your position here. A few questions, you are of course free not to answer any of those but in the name of civility and since after all you all, participating in a Discussion Forum, your input/replies/answers would be much appreciated.

There goes my question
Am I correct in reading this point from you: Casual ABX are not valid as scientific proofs? (BTW No disagreements there).

How do you explain this:
With knowledge: Night and Day.
Knowledge removed or withheld: Not able to perceive differences or even confusion.

Well, I can't recall having had this experience myself so I don't know but simply saying that nothing has changed, except sightedness, is not necessarily correct. If it was correct then why would the criteria for valid blind tests be necessary? Why not just take away knowledge & that's a blind test? People seem to forget that the reason for the test criteria is, not just to pass some scientific high-falutin ideal, it's there because these are all the recognised factors that play a part in influencing the outcome of the test. So you can't just remove one factor & when the test result changes, claim that it was that factor alone which caused the change in result - what I meant to say was, you can't claim that the result is now correct. You have no control over the other factors so you are flying blind & cannot deduce cause & effect as simply as you are stating.

How about we stop talking about hypotheticals & take a look at reality. There were many null ABX test results of differences before the current positive ones. What factor(s) account for the change to positive ones now appearing? Why did Max not hear differences when he played the files sighted but could pass the ABX test? Why did Vital, on another forum, have the same experience? What do you think might explain these changes?

I think examining these questions might be more fruitful than the cyclic argument that is filling these pages recently.

EDIT: in colors above

 

[amirm]

Let me add some perspective on how it "felt" to be taking these tests:

1. It was so easy to forget which track was A or B. In other words, I would hear the difference yet attribute it to the wrong sample A or B.

2. Second guessing was deadly. Often I would find the difference, get it right 5 or 6 times in a row. Then I would wonder if I had correctly classified A or B and would change my mind only to guess 100% wrong from then on.

3. Despite finding the difference reliably, if I let myself, I could "undo" the differential I was hearing. That is, I could read the characteristics of the alternate track into the one I was listening to.

4. As I would get closer to 0% chance of guessing, I would get more and more nervous before hitting the button "next trial." The anxiety was that if I guessed one wrong, it would revert the probability stats and would force me to run a bunch more trials to get down to 0% chance. As the results show, I could identify with 100% certainty the difference yet the nerves would get in the way, causing me to second guess. And at any rate, not being in the best mood to run the tests.

5. I would pass the test 4 to 5 times in a row, only to then keep getting it wrong from then on. If I could not reverse the trend, I would know that I must have been imagining the difference. I would re-listen and still find the same difference. I would then imagine the difference not being there and it would vanish! Then on demand I could bring back the difference in my mind. I would usually abandon the segment as not revealing enough.

As differences get small, our ability to avoid these errors becomes more limited to none. Training and discipline helps but even there, I faced all of the above challenges as I was taking these tests. You can see examples of that in my initial failed attempts in some of these tests.

Put another way, there is something to the fact that when testing for small difference, and the outcome being some kind of race that is scored online, and needing to achieve perfect score, does create an abnormal situation relative to how we usually enjoy music without having to report on it. Instead of the test being purely about what we hear, a lot more gets involved beyond just the ear.

 

[still-one]

As Amir has stated knowing what you are looking for can make all the difference.

http://gizmodo.com/this-audio-illusion-will-make-you-never-trust-your-ears-1593113324

 

[Orb]

Thanks for the link,
mainly because in the comments a poster provided this research lab at Northwestern University : http://www.soc.northwestern.edu/brainvolts/projects/music/
Used to follow/given papers by a few Uni research labs, but was not aware of this.
Thanks again
Orb

 

[amirm]

As Amir has stated knowing what you are looking for can make all the difference.

http://gizmodo.com/this-audio-illusion-will-make-you-never-trust-your-ears-1593113324

Wow, that is a clever demo and experiment.