I am. Allow me to re-phrase my sloppy analogy -- think Meyer and Moran with better controls.
Tim
Conclusive "Proof" that higher resolution audio sounds different
- Thread starter amirm
- Start date
I am. Allow me to re-phrase my sloppy analogy -- think Meyer and Moran with better controls.
Tim
No it has not. And it is a good question. The one I was actually interested in rather than pretending I was for another reason. I believe it will be the change from 24 to 16 bit. But no it has not been identified for Amir. Is he even interested in that part of it?
Speaking as the perpetrator of that test, I agree.
The people who do things like that are being truly adult about audio. All I can say is: "Become the man!". ;-)
This is exactly the testing described in the Meridian paper being presented at next month's AES Convention, plus the restriction of the tests to trained listeners.
Looking back at your contributions to this thread, it looks as if your objection centers on the used of trained listeners in formal tests because doing so predicts nothing about the application of the test results to "normal" listeners. The fallacy in that objection lies in the fact that, as long as that "normal" listener doe not having hearing damage, he can be trained to hear small differences by experience. Something that wouldn't have bothered him at one point on his life later becomes intolerable. So you have to separate the ultimate audibility of a factor from preference. The former is objective, the latter is both subjective ie, individual, and a moving target for any particular listener.
John Atkinson
Editor, Stereophile
Last edited:
I can share concrete data on this front as I have a number of times before. During my life as a trained listener, I was beat by totally untrained listeners from time to time. One example was one of our partners which kept complaining about a high frequency distortion. I couldn't hear it, nor could anyone in the codec team. We brought him and in front of us he kept asking that famous line, "can't you hear that?" We went back and investigated the code and indeed he was right. We fixed it and he was then happy. We were happier because we hired him to be part of our codec test on the spot.
So the above is a case where someone was even better than a trained listener. As I explained our ideas of transparency need to be universal if we want that claim to stick. Using trained listeners means we are testing at a very strict level which should cover almost all of the population as far as sensitivity to distortions.
As I explained we don't need to test other parts of the population when it comes to small distortions. We *know* they won't hear the difference from countless experiments. Throwing them in the experiment would potentially subject us to Simpson's paradox. This is where the sum of the results hides the true underlying data and generates the reverse outcome. The only thing putting a lot more untrained listeners in this pile is to increase the probability of Simpson's paradox.
Now, once we find a difference using trained listeners, then we can move to testing the next tier down using proper pre-screening tests.
I think you are misunderstanding my position/situation. I did enter the testing with any advanced notions. Arny and others persented the tests, and in the case of Arny he made it look like I wouldn't dare running and reporting the results. So I ran the tests. The interest there was nothing more than what it should have been: running the test to see if I could or could not detect any differences. I am confident that is the case with your testing unless you explain otherwise.
I create this thread to highlight that we had gotten ahead of ourselves. So sure that no one would pass such tests that we put them forward at every breath as the litmus test of the person being right about any audio observation. Worse yet, without proper data, we had believed outcomes we had not even tested ourselves! No one for example thought there would be any difference between listeners. That if the person creating the test couldn't hear the difference, others could not either. All of these things were wrong and need to be discussed and internalized.
No one should be anxious about these discussions unless they were doing the same above, claiming in forum posts that no one had ever passed a double blind test in this and that. I specifically put the "proof" in quotation mark and explained that this is what this discussion is about. But the fear continues to put a lump in people's throat that this means the end of objectivity, end of validity of blind tests, end of good enough vs high-end, etc., etc. These are good discussions to have given this new data but isn't where I aimed the discussion, nor do I believe in those outcomes.
In this context, no. Not at all. People are anxious to change the topic while not agreeing on where we are now. Do we believe there are people with better listening abilities than others? Do we believe that we have falsified statements that in 14 years no one could have told the difference in Arny's test? Do we still believe the differences are orders of magnitude below threshold of hearing? Do we believe in absolute necessity of trained listeners in these tests? Do we believe how essential it is to have controls in these test? Do we still talk about Meyer and Moran as a valid test?
Let's have people specifically say they agree or disagree and then we can move forward. If by what you are saying that we have not made any forward progress, then no, we have to stay here until we make that progress. There is concrete data that should inform our opinion and if it does not, then further testing is of no value just the same.
Amir, I think you are more optimistic about people accepting the implications of these results than is warranted. We have reached post 1355 in this thread & the same posts are being rehashed by pretty much the same people as were posted at the start of the thread - the desire to deflect from examining the validity of most of these tests & instead an attempt to refocus on the usual invalid tests of "normal people listening to normal music". Really, no learning has been achieved on this thread by these people & at this stage I don't believe any learning is possible for them. It's a shame but that is the nature of what is being revealed in this thread.
Lest anyone forgets, "real music" was used in Winer's test which also produced positive results after being heralded as a test that no one had passed in >10 years or whatever. These positive results were produced by non-trained people, some of whom were skeptics. Scott's files were also music for which people have produced positive ABX results.
The desire for this testing of "normal people listening to normal music" is just another way of suggesting that only the usual half-arsed blind tests are the ones that actually of importance. Why? Well, it's obvious - because they invariably produce null results which has been the fodder of objectivists for so long that they can't give it up.
Lest anyone forgets, "real music" was used in Winer's test which also produced positive results after being heralded as a test that no one had passed in >10 years or whatever. These positive results were produced by non-trained people, some of whom were skeptics. Scott's files were also music for which people have produced positive ABX results.
The desire for this testing of "normal people listening to normal music" is just another way of suggesting that only the usual half-arsed blind tests are the ones that actually of importance. Why? Well, it's obvious - because they invariably produce null results which has been the fodder of objectivists for so long that they can't give it up.
Amir and jkeny bring up a very valid point about "rehasing" the same old same old.
My question is this.
After 1,300 + posts, is there any consensus on anything?
My question is this.
After 1,300 + posts, is there any consensus on anything?
I spent my entire career as a professional communicator, and given the evidence of the jobs I landed, I think I might have been pretty good at it. But I must have left those skills in my last desk when I took my retirement; I'm having an awfully hard time getting people to understand a very simple idea...
Yeah, I think I got what the Meridian paper was about from previous posts. And I'm very interested in the results.
I don't object to the use of trained listeners at all. I think trained listeners are probably essential to identify very small differences. But I also think the use of experienced listeners who are not trained to focus in on specific differences is essential to understanding the real-world impact those differences may have on listeners. I'm not asking anyone to abandon the use of trained listeners, I'm suggesting that it would be really useful, would provide a very practical perspective, to add tests that don't use them exclusively. And I'm having a really hard time understanding why this is so hard to understand.
You've shared concrete data showing what untrained listeners hear relative to the small differences trained listeners identified? I missed that.
Nice anecdote; no data.
I really must be communicating horribly. I'm not suggesting anyone should throw untrained listeners into a trained listener test; I'm suggesting another test...
...I'm suggesting this!!! But now I'm confused. If you "know" that untrained listeners won't hear a difference, from countless studies, why would you bother moving on to testing the next tier down?
And by the way, a bit of advice -- if you would still like to see audio made available at full resolution, don't tell people you "know" that untrained listeners won't be able to hear the difference. If they believe you, no business with any sense at all will put any resources into bringing that product to market.
Tim
Yeah, I think I got what the Meridian paper was about from previous posts. And I'm very interested in the results.
I don't object to the use of trained listeners at all. I think trained listeners are probably essential to identify very small differences. But I also think the use of experienced listeners who are not trained to focus in on specific differences is essential to understanding the real-world impact those differences may have on listeners. I'm not asking anyone to abandon the use of trained listeners, I'm suggesting that it would be really useful, would provide a very practical perspective, to add tests that don't use them exclusively. And I'm having a really hard time understanding why this is so hard to understand.
You've shared concrete data showing what untrained listeners hear relative to the small differences trained listeners identified? I missed that.
Nice anecdote; no data.
I really must be communicating horribly. I'm not suggesting anyone should throw untrained listeners into a trained listener test; I'm suggesting another test...
...I'm suggesting this!!! But now I'm confused. If you "know" that untrained listeners won't hear a difference, from countless studies, why would you bother moving on to testing the next tier down?
And by the way, a bit of advice -- if you would still like to see audio made available at full resolution, don't tell people you "know" that untrained listeners won't be able to hear the difference. If they believe you, no business with any sense at all will put any resources into bringing that product to market.
Tim
Tim, this is pretty much what you said in post 27 of the thread
And this one from post 33 is along the same lines:
I don't understand the desire for this "normal people listening normally" test. It's such a fuzzy definition that it fails as a basis for a thesis to test & any attempt to apply the necessary rigour to it which would allow such testing would immediately disqualify it from "normal people listening normally". So to me it is a complete misunderstanding of testing & the need for rigour in testing - it only proves that no understanding has been reached here - It is an impossible test!!
And this one from post 33 is along the same lines:
I don't understand the desire for this "normal people listening normally" test. It's such a fuzzy definition that it fails as a basis for a thesis to test & any attempt to apply the necessary rigour to it which would allow such testing would immediately disqualify it from "normal people listening normally". So to me it is a complete misunderstanding of testing & the need for rigour in testing - it only proves that no understanding has been reached here - It is an impossible test!!
Like esldude, I would like to get back to the actual testing of what might be underlying the results, as I'm sure many others would also. Unless the possible underlying factors are investigated & analysed, this test will become written off as IMD or dither or resampler artifacts. These are actually the more interesting aspects of these positive results but as long as there are people trying to return to status quo testing as a more valid method, we will be stuck in this loop & no progress made. .
Maybe a separate thread should be started to analyse these factors?
Maybe a separate thread should be started to analyse these factors?
This is rich. The King Of The Strawman quotes a post of mine, then attributes a "normal people listening normally" quote to my post, even though nowhere in my post do I use that language or, for that matter, any other language which can even loosely be interpreted as such.
Then we see in his next post the King Of The Strawman asks about underlying factors. So what we have here is the King Of The Strawman takes my post out of context and ignores my earlier and, indeed, my first post in this thread (a question which I then repeated because Amir's first reply did not address my question) where I specifically ask Amir why he believes he passed the tests. There is no shame.
Leave me out of your intellectually dishonest quest, replete with false equivalencies and a complete and utter misunderstanding of the scientific method, lest we bend the forum rules which I, and I repeat, I, initiated. I have no patience for moronic tendencies.
Then we see in his next post the King Of The Strawman asks about underlying factors. So what we have here is the King Of The Strawman takes my post out of context and ignores my earlier and, indeed, my first post in this thread (a question which I then repeated because Amir's first reply did not address my question) where I specifically ask Amir why he believes he passed the tests. There is no shame.
Leave me out of your intellectually dishonest quest, replete with false equivalencies and a complete and utter misunderstanding of the scientific method, lest we bend the forum rules which I, and I repeat, I, initiated. I have no patience for moronic tendencies.
Firstly, can you stop with the personal, ad hom stuff - it's a reversion to your former attitude with me on this forum as a super-moderator which to be honest was pretty shocking & atrocious. Secondly, I said "post 33 is along the same lines:" so your evidence is moot but yes maybe your post is all over the place & you would care to elucidate on it as it seems to me more about trying to demean some nebulous target group than make any real point?
Oh, you mean the post where you state that "Before reaching any hang your hat here conclusions, I'd like to see more tests run with real music, of course, but also with different microphones, ADCs and DACs." A post which Amir corrected as Scott's tests were with music files " Hi Ron. All three of Scott's clips are real music from AIX records. They are not artificial sounds.
These tests are proper Ron. They are level matched and carefully created. Per above, they are also real music. And I used a common ABX comparator." To which you shifted your point of attack to "I was speaking, perhaps ineloquently, to the claims we've all read that long term and not short term listening is what is required to reliably and repeatably detect certain differences." No shame
How ironic that you flagrantly break the forum rules with this statement.
My posts speak for themselves, your ongoing misinterpretations of them, as King Of The Strawman, notwithstanding. The readers can decide for themselves.
Let me repeat: leave me out of your quest. I have no interest in moronic tendencies. Steve, I apologize, but as we've discussed privately in the past, John is, well... special.
Let me repeat: leave me out of your quest. I have no interest in moronic tendencies. Steve, I apologize, but as we've discussed privately in the past, John is, well... special.
It's no excuse to apologise to Steve & yet call my posting "moronic tendencies" - this does not excuse your ad hominem attacks on me.
I will quote your posts if I wish & you can clarify them if you wish but enough of the personal attacks, please !!
And John, I don't understand what you don't understand. Do you somehow think all the tools of keeping bias and error out of a test become ineffective if the listeners are not trained in what to listen for? Do you think there is no value in knowing what experienced listeners hear? Do you think there is nothing to be tested but absolute audibility? I don't think I'm the one who misunderstands testing.
Tim
John is right. Let's back off from just declaring the other side wrong. Please focus on sharing more information than on each other.
There is no point in hanging around waiting for people to change their minds. They won't. They should be left in the dust at the side of the road.
There are some real questions to be answered. For example, although evidence has been presented that shows that people heard differences, the evidence is far from compelling that the differences were caused by the resolution of the digital formats involved. The differences could have been due to implementation limitations in the format converter software or to as yet unknown artifacts in the playback equipment.
One way to advance is to better characterize what people can hear and to do so along separate directions, e.g. bit depth / dither algorithms would be one direction and sample rate conversion / filters would be a separate direction. One can get around the handicap of "null" results by using different parameters, e.g. how many bits?
There are some real questions to be answered. For example, although evidence has been presented that shows that people heard differences, the evidence is far from compelling that the differences were caused by the resolution of the digital formats involved. The differences could have been due to implementation limitations in the format converter software or to as yet unknown artifacts in the playback equipment.
One way to advance is to better characterize what people can hear and to do so along separate directions, e.g. bit depth / dither algorithms would be one direction and sample rate conversion / filters would be a separate direction. One can get around the handicap of "null" results by using different parameters, e.g. how many bits?
Similar threads
