The Upgrade Company

[mep]

A predictable result. Proving the futility of ABX testing.

Thanks for trying.

I think the real lesson here was that there was no big difference in SQ between a stock unit and a modded unit with blue-goo.

 

[Gregadd]

I have not problem with that conclusion. The problem is that that would be the expected bias held by some. Failure to identify the products under ABX does not prove the null hypothesis.

 

[pepar]

See,
http://www.whatsbestforum.com/showthread.php?3732-Is-ABX-finally-Obsolete

Yep, what a load of ... cobblers.

 

[pepar]

I think the real lesson here was that there was no big difference in SQ between a stock unit and a modded unit with blue-goo.

Of course, and what's with denying rigorous tried-and-true testing with one unknown author and another whose entire magazine is based taking advertising dollars from goo manufacturers?

 

[amirm]

Picking up here after a long dry spell, but we did a listening test. Here is the disclosure on how and the results. I'm sure that there will be comments on "ya shoulda done this, or that" but we did the set we could under the circumstances.
Jeff

That's really going beyond call of duty Jeff to run such detailed tests and sharing the results. Much appreciated.

As you have seen from the reaction, when doing these tests, you will make one or the other camp unhappy . As a minimum, you know to your own satisfaction if there is a differential or not.

BTW, on the point you raised regarding hearing more detail. I can take a song, without changing anything in it in successive plays, and imagine more or less detail on demand! In other words, I can play a song and convince myself the detail is not there. Then pretend the opposite and hear more detail. It is quite uncanny how you can read such fidelity differentials into the experience even when you are self aware. When testing different devices, there can be a real difference but disambiguating between the two is very, very difficult. You could test this theory by doing the test sighted. I suspect given the results you have, you will hear no difference. Then convince yourself there is a difference in the modded one and likely you will hear it then!

 

[Kal Rubinson]

See,
http://www.whatsbestforum.com/showthread.php?3732-Is-ABX-finally-Obsolete

What a joke. This contains no proof of anything.

 

[pepar]

That's really going beyond call of duty Jeff to run such detailed tests and sharing the results. Much appreciated.

As you have seen from the reaction, when doing these tests, you will make one or the other camp unhappy . As a minimum, you know to your own satisfaction if there is a differential or not.

BTW, on the point you raised regarding hearing more detail. I can take a song, without changing anything in it in successive plays, and imagine more or less detail on demand! In other words, I can play a song and convince myself the detail is not there. Then pretend the opposite and hear more detail. It is quite uncanny how you can read such fidelity differentials into the experience even when you are self aware. When testing different devices, there can be a real difference but disambiguating between the two is very, very difficult. You could test this theory by doing the test sighted. I suspect given the results you have, you will hear no difference. Then convince yourself there is a difference in the modded one and likely you will hear it then!

If only I could convince myself of ... anything. But it seems to happen on a level of awareness that is inaccessible to conscious and purposeful thought. If you were insane would you know it? If you thought raising power and signal cables off the floor on little trusses improved the sound ....

 

[Kal Rubinson]

I have not problem with that conclusion.

OK.

The problem is that that would be the expected bias held by some.

Are you inferring that there was an expected bias in favor of a null result?

Failure to identify the products under ABX does not prove the null hypothesis.

But it does not prove there is a difference (presuming there is statistical validity) despite the sighted discriminations of some of the subjects.
Are you presuming there is a difference?

 

[fas42]

BTW, on the point you raised regarding hearing more detail. I can take a song, without changing anything in it in successive plays, and imagine more or less detail on demand! In other words, I can play a song and convince myself the detail is not there. Then pretend the opposite and hear more detail. It is quite uncanny how you can read such fidelity differentials into the experience even when you are self aware. When testing different devices, there can be a real difference but disambiguating between the two is very, very difficult. You could test this theory by doing the test sighted. I suspect given the results you have, you will hear no difference. Then convince yourself there is a difference in the modded one and likely you will hear it then!

Familiarity with the material makes a huge difference. If you know the musical structure of the track being played intimately, your mind fills in the gaps in fidelity beautifully, and the ABX becomes relatively useless. This is why you can enjoy appalling quality in a favourite song on a junk car radio, say.

Far better to use something completely unknown; your mind is struggling to understand the musical content, so the ear/brain is working overtime to analyse, digest all the sound cues coming in.

I had this very experience testing a completely unknown Bruce Springsteen track with varying levels of compression: at first the variations were obvious, but as I began to "learn" the music this became harder and harder to pick. My mind knew at each point what "should" be coming next, and if the reality didn't quite match up to that expectation the gaps in quality were to a large degree "papered" over ...

Frank

 

[MylesBAstor]

Of course, and what's with denying rigorous tried-and-true testing with one unknown author and another whose entire magazine is based taking advertising dollars from goo manufacturers?

Could you please expand on what you are calling goo manufacturers?

The problem is that this conspiracy theory has ZERO proof. It's all innuendos, heresay evidence and paranoia. If you've read the stats that John Atkinson's has posted on Audio Asylum, there's absolutely no correlation between Stereophile's advertising and reviews. But that's obviously not enough for people.

Nor should there be since any well run mag will have a Chinese Wall between the ad and review dept. Oh, there's plenty of examples of good reviews in mags w/o advertising too. And of advertisers who yanked ads because they weren't happy with the review.

 

[MylesBAstor]

That's really going beyond call of duty Jeff to run such detailed tests and sharing the results. Much appreciated.

As you have seen from the reaction, when doing these tests, you will make one or the other camp unhappy . As a minimum, you know to your own satisfaction if there is a differential or not.

BTW, on the point you raised regarding hearing more detail. I can take a song, without changing anything in it in successive plays, and imagine more or less detail on demand! In other words, I can play a song and convince myself the detail is not there. Then pretend the opposite and hear more detail. It is quite uncanny how you can read such fidelity differentials into the experience even when you are self aware. When testing different devices, there can be a real difference but disambiguating between the two is very, very difficult. You could test this theory by doing the test sighted. I suspect given the results you have, you will hear no difference. Then convince yourself there is a difference in the modded one and likely you will hear it then!

Or as you have talked about more than once Amir, certain sonic artifacts will only show up with certain types of music.

 

[Gregadd]

OK.

Glad we agree on something

Are you inferring that there was an expected bias in favor of a null result?

Referiing to something as beiing treated with Blue Goo did seem to imply some negative connotations.

But it does not prove there is a difference (presuming there is statistical validity) despite the sighted discriminations of some of the subjects.

Yes,presuming statistical validity is a big stretch. Failing to prove there is a difference does not mean you have proved there is no differrence. It just means the whole thing is a wash.

Are you presuming there is a difference?

I could not care less. I just wanted to point out that the null hypothesis was not proven. Please correct me if I'm wrong.

 

[fas42]

And another thought. The ABX depends on human beings thinking and behaving like computers. And the last time I checked, that wasn't the case ...

Computer programmers know this problem well: it is extremely difficult to make a "mechanical", computing process emulate the "smartness" of human thinking or ability to recognise patterns; and people get very bored and overloaded doing computing type work: doing ABX's is forcing people to behave like a mechanical contraption, and it's a poor fit, it doesn't have a good chance of reaching the "truth".

Frank

 

[pepar]

And another thought. The ABX depends on human beings thinking and behaving like computers. And the last time I checked, that wasn't the case ...

Computer programmers know this problem well: it is extremely difficult to make a "mechanical", computing process emulate the "smartness" of human thinking or ability to recognise patterns; and people get very bored and overloaded doing computing type work: doing ABX's is forcing people to behave like a mechanical contraption, and it's a poor fit, it doesn't have a good chance of reaching the "truth".

Frank

Awww, OK, the I-just-don't-test-well argument.

 

[pepar]

Nor should there be since any well run mag will have a Chinese Wall between the ad and review dept.

Shirley, you jest?

 

[pepar]

Could you please expand on what you are calling goo manufacturers?

Clearly, there are a lot of goo consumers on this thread.

 

[mep]

Somebody seems to be overly sensitive.

 

[Kal Rubinson]

Glad we agree on something

Referiing to something as beiing treated with Blue Goo did seem to imply some negative connotations.

I agree with that but, if the test is blind, why should it matter?

Yes,presuming statistical validity is a big stretch. Failing to prove there is a difference does not mean you have proved there is no differrence. It just means the whole thing is a wash.

I could not care less. I just wanted to point out that the null hypothesis was not proven. Please correct me if I'm wrong.

Not at all. The test failed to prove a difference. It did not prove there is none. So, we are back to square one unless the anticipated statistical analysis informs us otherwise.

 

[Kal Rubinson]

And another thought. The ABX depends on human beings thinking and behaving like computers.

Red herring. "Which of these things is not like the other?"

Computer programmers know this problem well: it is extremely difficult to make a "mechanical", computing process emulate the "smartness" of human thinking or ability to recognise patterns;

This is pattern/feature recognition and it is something at which human beings excell.

...and people get very bored and overloaded doing computing type work

Only if the process is protracted.

doing ABX's is forcing people to behave like a mechanical contraption, and it's a poor fit, it doesn't have a good chance of reaching the "truth".

Just an assertion. Competent sensory and psychometric testing is well established.

 

[MylesBAstor]

Shirley, you jest?

Surely you have proof? Or lot's of just as usual casting aspersions against magazines.