Is ABX finally Obsolete

Status
Not open for further replies.
You're actually seriously asking me that question, no joke?

I've known about the ASTM since I was a little kid in the 50's, reading the books in my daddy's bookcase.





Still you aren't joking? Saying that a standard does not apply to a particular situation (which is what I did) is of course vastly different than saying that the standard is wrong. Another way to look at it is that I used an ASTM document as my authority for saying that an ASTM document does not apply to a situation that said document is mute about.





Again, far-reaching conclusions based on exactly what evidence? I wrote a post about the applicability of a certain document, not a PhD thesis about subjective testing. If the fact that I didn't write a PhD thesis for your pleasure, disturbs you perhaps you need to get the moderator update the terms of service to make that a prerequisite!



Where did I say all that?



I've seen the opinion granted that ABX is just a slight variation on a triangle test.




Straw man argument as I have already explanied.



To me, your ignorance of what I've said in a variety of contexts and forums is noted. So is your ongoing willingness to make far-reaching accusations based on what is at best described as highly limited knowlege.
As usual your responses fall back to being derogatory, when someone else does know the science and has a fair few papers based on those other methodologies.
As I do not know anything, how come you have never explained ROC, decision strategies, how same-different is a standard that is still used today, what 2IAX and other methodologies are,etc.
But to clarify, from what I understand the triangle test is more than just a slight variation, the purpose is to select the most different and not necessarily used in the same way, and they are not the only ones to state triangle and ABX create fatigue or observers defaulting to wrong cognitive decision strategy (other papers not specifically ASTM).

You ask about my conclusions..
But I have been consistent in this, it is about unknowns that have still not be proven either way in relation to ABX, but the behaviour of an observer is known so what I am opening the discussion to is not pie in the sky but a possibility of an underlying unknown.
Why do you think scientific studies find it critical to develop models and the area under ROC for such studies relating to JND?

Anyway the only one making accusations so far is you, and possibly now in a similar way to how you debate this subject with JA.
Edit:
Ignore a lot of this banter between both myself and Arny on this as it all started with a post being read before edited.
Thanks
Orb
 
Last edited:
Arny

I enjoy reading your posts but honestly I am going to warn you (as Amir did) that it serves no purpose to demean members.

Part of it was my fault Steve,
I edited my post to remove some of the heat but Arny had already picked that one - which can be seen in my quoted post of Arny's.
But the heat was taken out quite a bit before my very last edit, so I guess Arny saw the post clicked edit and waited a fair amount of time before posting - maybe blowing off steam *shrugs*.

Apologies

Orb
 
It's obviously a heated topic but very interesting nonetheless reading the polarity of members' opinions. Allowing emotions to spill over and throwing insults serves no purpose in advancing the discussion
 
I would like to respectfully submit that your study of *all* of the people who are doing DBTs for the purpose of audio evaluation appears to a tiny bit less than comprehensive.

You're jumping to far-reaching conclusions based on what? A few PR papers released by just one company?

I happen to know of nobody who routinely does ABX tests who thinks that it is hard to do them with consistency, and I know dozens of people who routinely do ABX tests.

I am glad you decided to converse with me . Could you please refer me to those who routinely do ABX, the methods, and thier results. I believe AMIR has already made that request. I beleive it was you who said you do "spot tests."
 
In high end audio the purpose of testing a hypthesis it to sell the product at hand. We all know that, right?

The whole problem with ABX and other DBT methods. amd in fact all of science is the same - they don't reliably produce the conclusions that lead to the greatest profits.
Many in the high end are dedicated to making quality products and advancing the field of audio reproduction. Spectral for example. Harmon seems to take failed companies;apply scientific methods and do pretty well.
Indeed it would appear to me that the notion of "all amps sound the same", would open the door to sales based on bells and whistles and the ad copy with the large breasted girl.
 
Isn't that the hidden agenda behind all of this critcizm of ABX?

There is no hidden agenda. I do have the courage of my convictions.
 
It's obviously a heated topic but very interesting nonetheless reading the polarity of members' opinions. Allowing emotions to spill over and throwing insults serves no purpose in advancing the discussion

Agreed and apologies for my part in it.
Coming back on subject it is kind of funny, what we have forgotten in this discussion that was mentioned a few pages ago by myself (and I had forgotten this as well) is that the right methodology should be used for what the focus is.
The problem as I see it is that there are possible several different debating points or perspectives relating to ABX going on currently.
IMO those who want to discuss ABX should mention what ABX task they feel is involved, by this I mean is someone talking about very specific test such as "all amps sound the same", more about listening to music where one may have added distortion/noise/etc, comparing wav to mp3, various codecs, etc.

To me this is absolutely critical because the criteria relating to the JND of testing "all amps sound the same" is not exactly the same as as doing a wav to mp3 comparison in general or added noise to one stimuli,etc.
IMO the latter two examples can rely on greater amounts of musical passages as in those examples we are looking for human acceptable thresholds.
But, in the "all amps sound the same" debate this is very specific to proving there are differences even if they are difficult to observe - possibly at the very edge of JND in short term listening (extended listening behaviour does not come into play that is seperate to test concentration fatigue).

So for those who are skeptic of ABX in the "all amps sound the same" debates may be right in saying the methodology-use of ABX needs to convey JND at its most difficult to observe, and possibly accomodate scientific approaches from other methodologies or use them in conjunction (with their ROC) and compare results.
Also thinking this through as I said, possibly the ideal scenario for such on the edge of observation is to prove any differences by using polyphonic chords or notes from musical instrument or instruments-groups-symphonies.
This may fit in with the difficulty slider (quite noticeable to barely there if at all) and sensitivity relating to the tougher JND and detection.
Also % accuracy needs to consider that the figure drops as difficulty increases, this is shown in research papers and dscussions in JND-ROC-etc.

Thanks
Orb
 
To me this is absolutely critical because the criteria relating to the JND of testing "all amps sound the same" is not exactly the same as as doing a wav to mp3 comparison in general or added noise to one stimuli,etc.

It is my experience that the essence of listening tests is the comparison of two sounds. Where those sounds come from and by what path is irrelevant as long as those sources and paths do not obscure the essencie of the comparison which is sound quality.

IMO the latter two examples can rely on greater amounts of musical passages as in those examples we are looking for human acceptable thresholds.

How is the use of musical passages a game-changer except of course that there is no doubt that the threshold of hearing for complex sounds is generally higher, than that for more complex sounds.

What prevents information that was obtained by means of pure tones being used to synthesize a threshold for complex sounds by including the effects of masking?

But, in the "all amps sound the same" debate this is very specific to proving there are differences even if they are difficult to observe - possibly at the very edge of JND in short term listening (extended listening behaviour does not come into play that is seperate to test concentration fatigue).

<I would like to point out to the moderators that I find the use of the phrase "all amps sound the same" and phrases like it to be highly objectionable in this context and ones like it.>

I'd rather busy myself elsewhere than to be routinely taunted like this. :-(
 
Indeed it would appear to me that the notion of "all amps sound the same", would open the door to sales based on bells and whistles and the ad copy with the large breasted girl.

For the many amps that do sound very much alike, there is little future, and no brand differentiation, in admitting that, regardless of the size of the girl's bells and whistles. The big companies with big marketing departments know this; they know that they need you to choose their brand, because the bells, whistles and girls, if they become the draw, will quickly be on their competitor's products, in their competitors ads. So they create differentiators, even if they are mostly lost on mortal ears.

High-end companies, of course, have much more modest marketing needs. Some are truly differentiated because they deliberately deviate from accuracy. Others get most of their marketing for free, supplied by Audiophiles on internet discussion boards.

Tim
 
Arny,
the "all amps sound the same" was NOT directed at you, in fact I have not quoted you in any of that post and was doing a general discussion.
Please take it in its context, where I stated in that post 2nd line down:
orb said:
The problem as I see it is that there are possible several different debating points or perspectives relating to ABX going on currently.
You are not generating all the debating points or perspectives.
In this case I am talking about where we have the debate of amps sound the same and ABX shows this, others then point to what they feel are concerns.

But unfortunately the response now loses the context and point of what I was saying.
In that the ABX task needs to be broken down depending upon its focus, and so any perspectives or debates raised by a poster needs to consider this.
The approach between very difficult JND proving whether two amps cannot be differentiated, and observation of added noise in a stimuli or codec, is not the same or should be considered to be approached differently.
Where one is very specific to proving for some whether a Class A and Class AB amp are so similar they cannot be audibly told apart.

If anyone has problems or is unsure of my earlier post please say and I will try to rectify that.
Thanks
Orb
 
<I would like to point out to the moderators that I find the use of the phrase "all amps sound the same" and phrases like it to be highly objectionable in this context and ones like it.>

I'd rather busy myself elsewhere than to be routinely taunted like this. :-(

Try not to take it to heart. I can't count how many times I've been told that I believe all amps sound the same. And that just scratches the surface of what I've been told I believe. Most recently, it seems, I'm a proponent of ABX, simply because I won't accept a very poor argument for its obsolesence. The truth is that I'm a proponent of blind comparisons, of which ABX is just one rather minor variation. But I digress...

There are a few here, once they've place you on the other side, who will put in your mouth all the words that have most offended their audiophile sensibilities, whether you've actually said them or not. There are just as many who largely agree with you. It's a pretty balanced place, I think. Much more so than most Audiophile boards. I can think of a couple where you would have been banned 140 posts ago.

Tim
 
In that the ABX task needs to be broken down depending upon its focus

There ya go! If we had a thumbs up emoticon here I'd use a whole lot of them.
 
How is the use of musical passages a game-changer except of course that there is no doubt that the threshold of hearing for complex sounds is generally higher, than that for more complex sounds.

What prevents information that was obtained by means of pure tones being used to synthesize a threshold for complex sounds by including the effects of masking?
I am not sure i follow the meaning Arny, following this logic if a real instrument has its tones or polyphonic chords masked, this would affect existing abx test involving music and may then mask possible differences.
But if two amps sound different, then it will be reflected in the presentation of the sound generated by instruments (albeit via recording), ensuring the noise floor/etc is within acceptable values.
As we are talking about JND specifically in this instance (well the context of my post was), how we use sound-music does not need to be the same as the other two examples of say observer listening to added noise stimuli,testing codecs,etc.
Many JND scientific tests focus on melodies and tones, also the challenge is as I mentioned; the more difficult the sensitivity (so tougher to differentiate) the % accuracy drops lower, and possibly this is compounded by using full music instead of approaching this from other JND scientific studies.
Therefore it is possible to condense the test down to using well recorded polyphonic chords and tones from real instruments.

Not being critical just a thought, which would reflect those scientific studies focusing on auditory aspects of JND.
To clarify I am talking about ABX methodology reflecting the purpose of the test and that it does not necessarily mean it is always the same, this needs to be taken in context of the previous posts.

Cheers
Orb
 
Try not to take it to heart. I can't count how many times I've been told that I believe all amps sound the same. And that just scratches the surface of what I've been told I believe. Most recently, it seems, I'm a proponent of ABX, simply because I won't accept a very poor argument for its obsolesence. The truth is that I'm a proponent of blind comparisons, of which ABX is just one rather minor variation. But I digress...

There are a few here, once they've place you on the other side, who will put in your mouth all the words that have most offended their audiophile sensibilities, whether you've actually said them or not. There are just as many who largely agree with you. It's a pretty balanced place, I think. Much more so than most Audiophile boards. I can think of a couple where you would have been banned 140 posts ago.

Tim

The fact that there are places that are more intolerant doesn't make this place a picture of tolerance.
 
The fact that there are places that are more intolerant doesn't make this place a picture of tolerance.

Well its comments like that which makes me wonder if you are overstepping your welcome here. We do strive to be a better forum but that's just my opinion.
 
Isn't that the hidden agenda behind all of this critcizm of ABX?

No. the agenda is to come up with a test that proves something. As stated by Mr. Atkinson in your debate nothing gets proved.
 
Well, first off I agree that setting up and running a proper dbt is hard work. I did it a few years ago. And, it was NOT ABX, just blind AB I spose you'd call it.

It took a lot of time and effort, and the Amirs point always applies, due to human nature (he calls it the make ego) it will always be interpreted by the reader according to his previously held position.

The good news is, that for a very short while at least, those who actually partook gad their eyes opened...a few beers was all it took however before it started to become that it was easy to tell them apart...that fish just grew bigger and bigger as the beers went down! (after the test I mean..it became a party)

To accomplish the test (so the listener had full control over what was what and when) we used a switch box at the LP. Well, as you can imagine, to some that was a fatal flaw yada yada.

There was a 'flaw'..or at least a weakness, that I too found when doing the test. As it involved two completely different front ends tied to one unchanging speaker, it also involved cuing up two identical cd's (as best we could time wise) and flickming between the two. That meant IF, for example, you thought there was a certain passage that might have been revealing, you could not re-listen to that passage on the other...as there were two distinct cds playing in two different players (did that make sense?)

So, if you wanted, learning was available. I learnt stuff too.

To a very large extent I too am dismayed that most forum discussion is not about learning, it is about maintaining rightness. Seriously, what is the point of that? I realise that is a personal thing, so others may not feel the same way.

To the 'deniers' (is there a better word??) why not TRY a level matched blind test?? I don't care what it is you do, forget ABX or ABC or NBC or CBC, just ensure the two most important factors, 'blind' and 'level matched'.

I assure you (and herein lies the problem I suspect) you WILL be shocked. Can you still hear differences?? YES, you can. People did hear differences in my very own test, that was not a problem for me..it might have been surprising yes, but I am always wanting to learn.

So funnily enough, THAT was one of the major thing to come out of my test, a blind test CAN and DOES still produce differences contrary to the oft produced canard. I'd say the differences were very much like that graph of seans, they were still there but by crikey the skewing provided by 'sight' (or knowledge) were gone!

Well, as the speakers were constant the differences were much less than it appears in seans graph.

So it's not a matter of 'all amps sound the same', or whatever, it is all about being brave enough to listen with the ears only. A lot are not brave enough to even try, and then they probably go on to trying to market multi kw amps or something. AND, you'd think that even for business considerations you'd at least do a bit of homework, well if you were normal that is.

Amir is completely correct on another matter. I earl;ier spoke of human nature, well sadly that applies to all on the net (I like to think I don't exhibit this as much, but I KNOW I do and must!!). Not only do we have the 'deniers' denying some BT results, we also have the 'acceptors' denying some...those that they TOO do not accept as it clashes with their pre-existing beliefs. What I mean is they will uncritically accept a result they expect or agree with, yet go to town finding faults in procedure of methodology in a contrary result.

Why is one group not subjected to the same scrutiny for properness or competence than the other IF truth is what we are after?

That, I suspect, is what lies at the bottom of a lot of Gregs position (correct me if wrong greg, not wanting to put words in your mouth) and others...well I sympathise completely. In many ways I find it far more hypocritical when that happens than when Greg (say) says 'I don't accept blind tests so I won't do them', at least he is being true to his beliefs.

But to accept any (possibly) poorly run 'blind test' simply because it is what you want and to try and find fault with those you don't want..well I see double standards at work there.

Yet greg, whilst agreeing with you in many ways, I cannot overlook the fact that you (and others) simply refuse to try a blind level matched test.

Look behind the screen, the wizard might not quite be what you expect (poor attempt at a movie reference...but as I don't watch movies it is probably very poor haha).

But equally, I fully understand that to some the magic and myth is what gives enjoyment.

Not brave? I accepted a challenge from a member of this forum to an ABX test to fly out to his place at my expense. He backed out. The attacks I have recieved on this thread alone is not for the feint of heart.

Another challenge to an ABX? It always comes down to that. What of someone like Mr. Fremer who has been able to pass an ABX test. Has it improved his position at all?

While I am a fan of the Wzard of OZ I prefer Tina Turner- "I don't need another hero." I don't need to know the way home." I'll just use my senses.
 
Well its comments like that which makes me wonder if you are overstepping your welcome here. We do strive to be a better forum but that's just my opinion.

A well-founded one from my point of view, Steve, and I think we all know my views are closer to Arny's than, say...well, Frank comes to mind. This place has balance; that is its strength. And while I wish all of you would just see the light and admit that I'm always right about everything, I guess that wouldn't leave much to discuss, now would it? Arny, relax. Nobody's picking on you. A few people are strenuously disagreeing with you is all. A couple of them are doing a pretty poor job of it. I think you can handle it.

Tim
 
No. the agenda is to come up with a test that proves something. As stated by Mr. Atkinson in your debate nothing gets proved.

Statistical probabilities, Greg. Nobody's trying to prove anything. That's not the mission, the objective, or the point. Is that why you think ABX is obsolete? Because it doesn't accomplish the impossible task it never set out to accomplish? Run enough trials and you can reduce the margin for error to something pretty meaningless, but proof? Nah. You can't prove anything to some Audiophiles, anyway, Greg. If you could, a statistical test like ABX wouldn't be necessary. I can measure whatever you imagine you hear with an instrument that is more sensitive than the best human ears, show you (if it's there at all) that it falls below the threshold or outside of the capacity of human hearing, and you will still "hear" it. The true believer doesn't need to deny statistics. He denies science.

Tim
 
Status
Not open for further replies.

About us

  • What’s Best Forum is THE forum for high end audio, product reviews, advice and sharing experiences on the best of everything else. This is THE place where audiophiles and audio companies discuss vintage, contemporary and new audio products, music servers, music streamers, computer audio, digital-to-analog converters, turntables, phono stages, cartridges, reel-to-reel tape machines, speakers, headphones and tube and solid-state amplification. Founded in 2010 What’s Best Forum invites intelligent and courteous people of all interests and backgrounds to describe and discuss the best of everything. From beginners to life-long hobbyists to industry professionals, we enjoy learning about new things and meeting new people, and participating in spirited debates.

Quick Navigation

User Menu