Tips for ABX Tests

[microstrip]

(...)
If you would bother to look around a bit, you will find that I have grumped both gently and rather loudly at people who scream "PROVE YOUR PREFERENCE IS RIGHT". There is no right to preference, everyone has one, and as long as it doesn't pass the tip of someone else's nose, so be it.

But should we accept that when people are asked to express their preference on sound quality in unbiased conditions there is a significant convergence of preference in a "RIGHT " type of sound reproduction?

 

[microstrip]

If I understand you correctly, you're saying that for 2 samples of audio, in "blind" listening, there can simultaneously be a statistically significant preference for one over the other, and no significant difference when tested via ABX. I find that unlikely. I would want to see the actual results of the testing before entertaining the possibility that both cases are correct. Personally, I would look at it from another angle: Based on what we know about how we hear, is it possible to generate audio signals which produce the result you have described?

Well, the game was just analyzing the situation entertaining the possibility that both cases are correct to find what type of circumstances (if any can exist) that could justify it.

 

[Groucho]

I would say that either one or both experiments are flawed in some way, or you have hit upon a (hypothetical) situation where the task of identifying a match is much harder than identifying a personal preference. Maybe the hypothetical test subjects are relaxed about one task and more anxious with the other, which I think may be where you're coming from? I don't see why that is not possible. All tests presumably come with caveats regarding explaining to the subjects what they are trying to achieve, getting them into a relaxed frame of mind etc. All of these are a little subjective so a test with a negative result cannot guarantee that it would necessarily be a negative result on another day.

 

[Gregadd]

A review is not just preferential it is also observational. It becomes preference if there is a conclusion ranking it against other products. It becomes of value based on experience, exposure to equipment and source material. They have far more access to equipment and source material than most. Not to mention more time. it's not always the case.

For example if I say the amp exhibits an edgy high end that's an observation; When you add for that reason I cant live with it. That's a preference. It's not just about arbitrary and capricious there has to be a reason.

Excercise due diligence? No thanks. I got better things to do. j.j. just trying to demonstrate I have no personal vendetta. I can't because I don't know you. You have the last word if you wish. .

 

[Gregadd]

So whose the quack? What's the profession and what was the quackery? Oh yeah who spouted it?


>First, an uninformed person can spout quack science."

 

[Gregadd]

I wonder if in AB tests there should be an option of I don't know?

 

[j_j]

But should we accept that when people are asked to express their preference on sound quality in unbiased conditions there is a significant convergence of preference in a "RIGHT " type of sound reproduction?

No. You can't. "Right" is personal. There are at least 3 clusters (meaning groups of people) with very specific preferences that I know of (I don't mean audiophiles, either, they are split as well, which accounts for some of the controversy). It's something that could be analyzed if there was the funding to run the test.

 

[j_j]

I think I was clear stating that the preference tests were not sighted ...

Ok, if not sighted, then one needs to put positive and negative controls into both and find out what's going on.

 

[j_j]

. j.j. just trying to demonstrate I have no personal vendetta.

Your words already represent such.

 

[NorthStar]

Like Amir said before; for true confidence let rely on our OWN ABX testing, instead of someone's else. ...There lies the clarity and the most realistic and trustworthy study.

 

[Gregadd]

Your words already represent such.

Let's put the whole quote in there.

"Excercise due diligence? No thanks. I got better things to do. j.j. just trying to demonstrate I have no personal vendetta. I can't because I don't know you. You have the last word if you wish."

 

[microstrip]

No. You can't. "Right" is personal. There are at least 3 clusters (meaning groups of people) with very specific preferences that I know of (I don't mean audiophiles, either, they are split as well, which accounts for some of the controversy). It's something that could be analyzed if there was the funding to run the test.

Well I knew essentially about two groups - the professionals and the consumers. Can you help with your suggested clusters? Where in this debate should we situate F. Toole studies, referring that there are no such think as ethnic or national group preference in listeners? I have seen very different studies on this subject, even people relating preferences in sound reproduction with the sounding of the country language.

 

[j_j]

Let's put the whole quote in there.

"Excercise due diligence? No thanks. I got better things to do. j.j. just trying to demonstrate I have no personal vendetta. I can't because I don't know you. You have the last word if you wish."

Selective quoting and feigned context suits you no better than your previous personal abuse.

 

[j_j]

Well I knew essentially about two groups - the professionals and the consumers. Can you help with your suggested clusters? Where in this debate should we situate F. Toole studies, referring that there are no such think as ethnic or national group preference in listeners? I have seen very different studies on this subject, even people relating preferences in sound reproduction with the sounding of the country language.

I know that there are people who like VERY direct sound (i.e. headphones), people (many of whom are concert goers, classical, folk, blues, ...) who like a direct to reverberant ratio something like row 8, and then there is a large group that likes very, very diffuse sound.

How well they cluster is not well established. Get me the funding (it won't be easy) and I'll tell you.

 

[treitz3]

Greg, what do you hope to achieve with this thread? Is it actually tips or just your point of view? In other words, what is it exactly that you are arguing for or toward?

Tom

 

[Gregadd]

Good point Tom.

I think peopel should avoid ABX. Uf they chose nt they shoold not take it lightly.

I hoped to point out some pitfalls and give some suggestions on ow to deal with them. I did not contemplate having to defend myself personally. I think we made some points. This is a hot button topic.

 

[Phelonious Ponk]

Greg, what do you hope to achieve with this thread? Is it actually tips or just your point of view? In other words, what is it exactly that you are arguing for or toward?

Tom

A bit of very recent history, for the original post in this thread should answer all your questions:

Tips
1. Never take an ABX test.
2. Go back to Tip 1. and follow that advice.
3. For some reason you failed to follow Tips 1 and 2. You might want to check on what happened to Michael Fremer he successfully negotiated an amplifier test.
4. Even if you have publicly accepted an ABX challenge it's not to late to get out. yes you'll take some flack for withdrawing. The challenger will claim you knew you were wrong and the test would expose it. Not to worry they would not respect you or support your position anyway.
Get out now before it's to late!

It's a strategic snark. A blatant opportunity to snipe the opposition and promote the OP's opinion. If I ever start a thread entitled "Tips for getting the most out of your vinyl rig" it will be as serious and sincere, as credible, as this one.

Tim

 

[NorthStar]

I just think that sometimes we get enthralled in our own composure's state, without much giving up on our grip.

_____________

I don't rely on someone's else opinion without solid foundations. ...Same for most other people.
As for ABX testing, I don't truly care; it does nothing to my sensor, my vibrating chords, my temporal pleasure.

 

[j_j]

It's a strategic snark. A blatant opportunity to snipe the opposition and promote the OP's opinion.Tim

More than that, it's a pre-emptive attack on anyone who might start a thread titled "how to do a DBT correctly".

This thread has, for the foreseeable future, utterly eliminated any possibility of such a thread being useful.

And that, frankly, is why I think it was started. I think it was started intentionally to pre-empt any rational discussion.

 

[treitz3]

Tips
1. Never take an ABX test.
2. Go back to Tip 1. and follow that advice.
3. For some reason you failed to follow Tips 1 and 2. You might want to check on what happened to Michael Fremer he successfully negotiated an amplifier test.
4. Even if you have publicly accepted an ABX challenge it's not to late to get out. yes you'll take some flack for withdrawing. The challenger will claim you knew you were wrong and the test would expose it. Not to worry they would not respect you or support your position anyway.
Get out now before it's to late!

OK Tough Guy you are hell bent on proceeding with the test.

5. It is essential you have the negative position. When you obtain the statistically insignificant results you can claim they failed. Make them prove there is no audible difference between cables. Don't let them suck you into proving there is a difference.
If you can't do that withdraw from the test.
6. Don't let the test become personal. Make them prove that there is no audible difference. Don't fall into the trap of proving you can't hear a difference.
7. Don't let the challenger design or conduct the test. The reasons for that should be obvios. Remember despite what they say they want you to fail
8. Demand to know all the details in advance. The only thing that shoud be secret is the identiy of x during the test. If they try to switch anything on the day test don't let them
9. Practice, practice, practice. Make sure that the characteristic you claim exist are actually present in device A and not present in device B. in the music to be played after levels have been matched.
10. Call thier bluff. That's how you get here in the first place. Take every advantage they offer. Make sure it is infact an advantage.
11 Don't guess. You have already identified the chahracteristics you are looking for. Take your time and locate them

This is why it was started. The OP. Whether I or the management team agree? That's up to you and your thoughts..

Tom