Conclusive "Proof" that higher resolution audio sounds different

[amirm]

If wishes were fishes...

In this case experience is speaking.

How do you do that when the experimenter knows the right answers and can affect the outcome of the test?

"Can" is not the same as "will." When in school and your teacher gave you a multiple choice quiz, did you know the answer everytime because he affected the outcome? I teach classes and I never have trouble keeping the answer to myself.

Ever since Clever Hans the answer for thoughtful people has been: "The human ability to control its personal bias is questionable at best".

Which is why I said if you want to screw up the outcome of a listening test, simply go upstream and cook the test. I gave examples of this already. It is that bias which is insidious. It is hidden behind the mask of "DBT."

Back to your point, it is insufficient to show possibility. You have to show actuality. I gave you a simple single blind scenario: proctor is behind a screen and simply switching samples. No talking. No visual contact. Please explain how you can induce an outcome willingly let alone unconsciously.

Typical of the many golden eared audiophile attempts to stigmatize DBTs for problems that they share with any any subjective test or more general than that, any audio test.

Typical of "objectivity" zealots who use these talking points to dismiss the results of any test they don't like. In this case, Ivan says he has seen blind tests and we are supposed to automatically assume they are all invalid due to that fact alone? That they are similar to sighted tests? Pls leave these arguments for other less informed discussions.

Just to clarify an obvious misapprehension, DBT is not a complete experimental design, it is a critical part of one.

Unfortunately as I explained the zealots take it as not just a "part," but the whole deal. They are blinded by those three letters which they understand. And ignore what is underneath which they don't. Meyer and Moran followed DBT, yet forgot the basics of checking to see if they had high-res content. Blinded by DBT....

Anybody who reads BS 1116 should be able to see that. BS1116 says that the test should be a DBT and then it specifies a whole lot of other test conditions to come up with a semblence of a good experimental design.

We have established that there is not one single blind test thrown around on forums on this topic that remotely followed BS1116. So now you want to use it to decide what is and is not proper testing?

But let's go there as I don't think you have really read the document. Here is the definitions page:

Blind test
A test in which the only source of information for the subject about the trials is the stimuli.

Double blind test
A blind test in which there is no possibility of uncontrolled interactions between experimenter and the listening test.

I showed with a specific example of the curtain and no communication how the intent of double blind test can be met. I stopped the interactions between experimenter and listening test.

And this is what it says about its use:

4 Test method
To conduct subjective assessments in the case of systems generating small impairments, it is necessary to select an
appropriate method. The “double-blind triple-stimulus with hidden reference” method has been found to be especially
sensitive, stable and to permit accurate detection of small impairments. Therefore, it should be used for this kind of test.


It says nothing about other tests being the same as sighted testing which is the argument you were making. Nothing about this discussion says that you should not use double blind tests or that it doesn't have value. Instead, we are discussing the lack of value which you assigned to single blind tests.

 

[Orb]

If wishes were fishes...

Arny just quoting your response to Amir stating SBT can be the real thing.
You used the example of Clever Hans to present why SBT is not valid, however I also presented that screened SBT related to Clever Hans actually overcame those shortcomings (where animals passed original test as per Clever Hans they actually failed the screened/visual block SBT one done by a university study).
Therefore either Clever Hans example is not valid on your part and would then remove my own example, or it is valid and therefore it is possible for SBT to be conducted in a way to be the real thing

In summary, it is how one does the SBT.
Thanks
Orb

 

[amirm]

Absence of evidence is't evidence of absence. I've read the paper cited above and it pretty well describes the tests many of us have been doing all along.

Well, you argued the paper years before you actually read it. Here is our back and forth on that: http://www.avsforum.com/forum/91-au...cott-s-hi-res-audio-test-15.html#post24795939. When I asked you if you had read the paper, this is what you said:

Actually, I have never had the paper to read. That's why I emailed the potentially erroneous comments to Dr. Vanderkooy directly for his review.

You post that less than 2 months ago. You had no problem years back arguing the paper without having it in hand or reading it.

I see people's inability to read the paper being exploited to create false impressions.

The paper is very clear: Under pristine conditions it may just be possible to hear residual channel noise. Dr. Vanderkooy is saying that based on evidence he has.

 

[Ron Party]

It just continues to sadden me to read the ongoing intellectual dishonesty by BOTH sides, i.e., both Arny and Amir. It does nothing to advance the cause; it has everything to do with personality deficiencies. Do either of you ever bother to look in a mirror? Do you recognize your own boundaries? Do you have an ounce of humility?

Look, bottom line: nothing, I repeat, nothing, has been proven and nothing is conclusive. Allow me to put either one of you on the stand and it would be easy pickings to cross-examine, impeach and completely discredit your testimony. At best, and this is a stretch, all we have is new data. Credibility and reliability of the data have not yet been established and, at this time, cannot be established. What an enormous waste of intellect and energy. Phew.

 

[Stereoeditor]

Actually, I have never had the paper to read. That's why I emailed the potentially erroneous comments to Dr. Vanderkooy directly for his review.

You had no problem years back arguing the paper without having it in hand or reading it.

This is all eerily familiar.

Back in the Usenet days, Arny Krueger posted criticisms of a blind test that had been published in Stereophile. When I queried him about statements he had made about the test that were factually incorrect, he responded that though he hadn't yet read the article, when he did he was sure that his criticisms would be proved correct

John Atkinson
Editor, Stereophile

 

[esldude]

It just continues to sadden me to read the ongoing intellectual dishonesty by BOTH sides, i.e., both Arny and Amir. It does nothing to advance the cause; it has everything to do with personality deficiencies. Do either of you ever bother to look in a mirror? Do you recognize your own boundaries? Do you have an ounce of humility?

Look, bottom line: nothing, I repeat, nothing, has been proven and nothing is conclusive. Allow me to put either one of you on the stand and it would be easy pickings to cross-examine, impeach and completely discredit your testimony. At best, and this is a stretch, all we have is new data. Credibility and reliability of the data have not yet been established and, at this time, cannot be established. What an enormous waste of intellect and energy. Phew.

Big +1

 

[jkeny]

It just continues to sadden me to read the ongoing intellectual dishonesty by BOTH sides, i.e., both Arny and Amir. It does nothing to advance the cause; it has everything to do with personality deficiencies. Do either of you ever bother to look in a mirror? Do you recognize your own boundaries? Do you have an ounce of humility?

Look, bottom line: nothing, I repeat, nothing, has been proven and nothing is conclusive. Allow me to put either one of you on the stand and it would be easy pickings to cross-examine, impeach and completely discredit your testimony. At best, and this is a stretch, all we have is new data. Credibility and reliability of the data have not yet been established and, at this time, cannot be established. What an enormous waste of intellect and energy. Phew.

Wow, both you & esldude really do seem to miss the whole point of this thread & seem to wish it would just go away & the status quo prevail. It's now not about the data - it's about the test itself!

Amir's point is correct - DBT has been used, blindly & unthinkingly, by objectivists as some golden measure of "proof". That three letter acronym has become the most abused TLA on audio forums, almost as mis-directing as another TLA - THD.

The attempt, in this thread to introduce some thinking about DBTs has resulted in a lot of resistance & very revealing attitudes - attitudes which prove exactly what Amir is stating - the bias of the test designer & the resultant biased design of the test. Examples of this attitude couldn't be better illustrated than by Arnyk's posts, Tim's posts & Maxflinn's post - some of whom had never heard of BS1116 before, others who have not read it before & others who still refuse to read it. I suspect that these are representative of the common range of objectivists. The possibility that tests designed by people with these attitudes will have any objectivity is very slim. The ongoing reluctance by some to learn is a sad indictment. The complete opposite to the scientific principle that they would so espouse.

It's interesting that the longer this thread goes on, the more revealing it becomes.

I mentioned before my thread title on another forum entitled "sorting out evidence-based from faith-based" which had exactly the same trajectory as this one. Close it down, some protested & it too was very revealing of the attitudes held by the very same people who like to call themselves "objectivists"

 

[thedudeabides]

Sounds like there could be a fair bit of subjectivity within certain factions of the "O" crowd.

 

[jkeny]

Sounds like there could be a fair bit of subjectivity amongst the "O" crowd.

Subjectivity being masqueraded as objectivity is another thing altogether - it's called self-delusion
This is the critique levelled at the "S" crowd for so long that we realise just how "deluded" we all are
Time for the "O" crowd to wake up

Edit: Notice how I don't say "S" are right & "O" are wrong - just that we are all prone to delusion & unless some real attempt is made at controlling this in tests (not just some token gesture) then these "O" tests should be considered of the same value as anecdotal "S" reports. I know this is probably considered an insult by the "O" crowd (& has been argued already incessantly in this thread) but again this shows how much of a sciency-kinda-thingy, superiority delusion they have been suffering from.

 

[Mike Lavigne]

my son-in-law the Phd in Physics did his post-doc work at U of Chicago and he was working on a radio teliscope research project where a team at Cal Tech was suppose to design a piece of testing gear that would be taken to Chile and be part of the project. the Cal Tech team sent this piece of gear to Chicago 3 or 4 times but it never worked properly to perform the measurments so the project could proceed. there was another team at Princeton with their part of the project too. so three teams working on this funded grant project but they could not get it the test gear to work correctly.

the problem was that the U of Chicago team would not lower their standards of measurment to move forward since then the test would not have the potential to get a meaningful, valid result. yet, the other teams wanted to proceed to get the funding to continue and to protect their credibility.

my son-in-law then decided to go into the private sector to avoid all the egos and politics. he had originally planned to stay in the academic world.

i realize i'm relating his viewpoint and there are other perspectives on what occured, but it seems that objectivity is always an issue.

 

[jkeny]

What, you mean science, as practised, isn't always objective? Shock & horror
Fair play to the U of Chicago boys but I'm not naive enough to imagine that they didn't have ulterior motives for holding to scientific rigour

 

[FrantzM]

my son-in-law the Phd in Physics did his post-doc work at U of Chicago and he was working on a radio teliscope research project where a team at Cal Tech was suppose to design a piece of testing gear that would be taken to Chile and be part of the project. the Cal Tech team sent this piece of gear to Chicago 3 or 4 times but it never worked properly to perform the measurments so the project could proceed. there was another team at Princeton with their part of the project too. so three teams working on this funded grant project but they could not get it the test gear to work correctly.

the problem was that the U of Chicago team would not lower their standards of measurment to move forward since then the test would not have the potential to get a meaningful, valid result. yet, the other teams wanted to proceed to get the funding to continue and to protect their credibility.

my son-in-law then decided to go into the private sector to avoid all the egos and politics. he had originally planned to stay in the academic world.

i realize i'm relating his viewpoint and there are other perspectives on what occured, but it seems that objectivity is always an issue.

Mike

Completely OT but how different does he think the the private sector would be from Academia?

Same: Humans being Humans... Back to the scheduled discussion

 

[Mike Lavigne]

for the last 5 years he has headed a team of scientists that do research and product development at a local think tank. we do talk about the challenges of his job as far as politics and such. the think tank is a subsidary of a large overseas conglomerate and so there are some disconnects involving that conglomerate's views on progress his team makes. and some personality issues with his local superiors. so there are real people and real sorts of things involved. but mostly there is little intellectual hypocracy involved, it more getting what he does to fit into the company. he does not feel like the system is constipated and toothless.

and the product of his team evidently is successful for the organization....as far as i can tell. he seems happy doing what he is doing. he says he does not miss academia.

 

[Orb]

Mike

Completely OT but how different does he think the the private sector would be from Academia?

Same: Humans being Humans... Back to the scheduled discussion

Can be very different and I have been lucky to visit/been involved with quite a few private sector research-engineering labs, then there is the partnership schemes/projects between private sector and universities singularly or linked.
At the core research level of the private companies it is about results/advancements in frameworks-standards (or at times bespoke inhouse) and products-technologies-solutions.

As an example why private companies can be more beneficial; many university developed technologies/solutions are then "spinned" off into a private joint venture company both for funding and to further results.
That said egos or more likely strong personalities exist in all aspects of life, especially when it involves teams of people with PhDs/research degrees from my experience

Cheers
Orb


.

 

[Phelonious Ponk]

Amir's point is correct - DBT has been used, blindly & unthinkingly, by objectivists as some golden measure of "proof".

Really? That's Amir's point? I think it's yours. And is anyone here blindingly, unthinkingly saying that a DBT is some golden measure of "proof?" Because here is where this conversation is taking place and this is the community participating in it. You have once again argued against a position no one has taken...

That three letter acronym has become the most abused TLA on audio forums, almost as mis-directing as another TLA - THD.

...and this time you've puffed it up to make a moot point look like an epic battle. Have some "objectivists" looked at the results of casual DBTs on forums and claimed them to be proof? I'm sure. There is a lot of misunderstanding and no small measure of willful ignorance of what constitutes a statistically valid conclusion in the audiophile world, much less "proof."

It's interesting that the longer this thread goes on, the more revealing it becomes.

Well, we can agree again.

Tim

 

[Ron Party]

Wow, both you & esldude really do seem to miss the whole point of this thread & seem to wish it would just go away & the status quo prevail. It's now not about the data - it's about the test itself!

I had promised myself I would not respond to anything you posted, ever ... we have history here at WBF from my days as a Super Mod which I would have preferred not to revisit ... but this one I cannot let go uncorrected.

This thread has morphed into many subjects as do almost every thread where the 2 As bash each other with personal insults and intellectual dishonesty. That you have chosen one of the morphed subjects to attack me and ignored the OP is your shortcoming, not mine. Argue with someone who has espoused that which you incorrectly attribute to me. I will have no part in it. It is just another example of the intellectual dishonesty running rampant here. The OP speaks for itself. The King of the Strawman seemingly is gone from this forum. Hail to the new King.

 

[esldude]

Wow, both you & esldude really do seem to miss the whole point of this thread & seem to wish it would just go away & the status quo prevail. It's now not about the data - it's about the test itself!

Has nothing to do with the topic for my part. It is clear there is personal animosity between Arny and Amir. When both were taking part the tone changed somewhat. And nothing since then has contributed a thing. It goes back some years it appears so I assign blame to neither party in this particular thread. But it would be nice to stop with the petty bickering on tiny little points that fit with the term angels dancing on the head of a pin.

I am all for good tests and the tinier the differences the more constraints on doing the test rightly and well. Forum tests will never match academic research or even corporate funded testing done well. It can serve for some purposes. Finding new pscyho-acoustic principles and proving them is not likely one of them.

As for DBT being hurled as a curse or spell that is one reason I take little part in Hydrogen Audio forums. They will ban many forms of reasonable evidence while being quite slack in DBT's. DBT like any other test done poorly or done with too much bias is not going to give useful results in an objective sense. That is why I complained about the various test files so much. Level mismatches and timing mismatches are way too sloppy and too easily avoided. Just throwing out DBT and calling it good does no one any good.

 

[Ron Party]

Amir's point is correct - DBT has been used, blindly & unthinkingly, by objectivists as some golden measure of "proof".

Really? That's Amir's point? I think it's yours. And is anyone here blindingly, unthinkingly saying that a DBT is some golden measure of "proof?" Because here is where this conversation is taking place and this is the community participating in it. You have once again argued against a position no one has taken...

Tim, I see you beat me to the punch. Quick left hand you have.

It's amazing to me that people do not understand the scientific method. Were JJ here he'd be screaming.

 

[jkeny]

Amir's point is correct - DBT has been used, blindly & unthinkingly, by objectivists as some golden measure of "proof".

Really? That's Amir's point? I think it's yours. And is anyone here blindingly, unthinkingly saying that a DBT is some golden measure of "proof?" Because here is where this conversation is taking place and this is the community participating in it. You have once again argued against a position no one has taken...

Tim, I'll let Amir speak for himself but that is my interpretation of what he is saying - The accumulation of such DBT results is portrayed as proof (or very close to it) - do you deny this? How many times have you read the suggestion by some "O" that a subjectivist go do a DBT & then report back to be even considered. Note, no mention ever made of a rigorous DBT, just a DBT

...and this time you've puffed it up to make a moot point look like an epic battle. Have some "objectivists" looked at the results of casual DBTs on forums and claimed them to be proof? I'm sure. There is a lot of misunderstanding and no small measure of willful ignorance of what constitutes a statistically valid conclusion in the audiophile world, much less "proof."

Well, I've asked you above a question which addresses this same point that you are making - "proof" it is perceived to be by many. You may choose now to diminish this perception but do you deny it is rampant?

Well, we can agree again.

Tim

Good, glad you are getting something out of it, although I suspect you are trying to be sardonic.

 

[jkeny]

I had promised myself I would not respond to anything you posted, ever ... we have history here at WBF from my days as a Super Mod which I would have preferred not to revisit ... but this one I cannot let go uncorrected.

Yes, I remember it well & the die was cast then, I suspect.

This thread has morphed into many subjects as do almost every thread where the 2 As bash each other with personal insults and intellectual dishonesty. That you have chosen one of the morphed subjects to attack me and ignored the OP is your shortcoming, not mine. Argue with someone who has espoused that which you incorrectly attribute to me. I will have no part in it. It is just another example of the intellectual dishonesty running rampant here. The OP speaks for itself. The King of the Strawman seemingly is gone from this forum. Hail to the new King.

If you didn't understand the nature & ramifications of this thread from the start, don't blame me for your failing. I'll give you that intellectual dishonesty is one of the themes of the thread - indeed The intellectual dishonesty of biasing tests to achieve the desired result - the intellectual dishonesty of not caring about test rigour. Yes these are all cogent to the thread. Do you mean some other intellectual dishonesty or wish to stay with a fuzzy definition without being specific & declare that you were misinterpreted when somebody replies to you?