Is ABX finally Obsolete
- Thread starter Gregadd
- Start date
- Status
This is an interesting point, because the key research by Floyd/Sean/ and others done at both NRC and Harmen in the development of speakers did not use ABX from what I can see and have read fom the papers published;
Cheers
Tim, more commonly used across the various disciplines is actually same-different test methodology, and not ABX, which seems more tied in with audio for some reason and the same-different methodology is not used.
I can provide interesting research that fits exactly with our discussion of is there any difference and these all use same-different test methodology; these can be from neuroscience to relating more to product research, and critically psychology studies.
I still do not know why there is such a focus with ABX in audio, and not go further with validation by repeating the tests using same-different methodology.
Same-different relies upon solely AB, without the mechanism of forcing both to be compared to X as in ABX.
As I mentioned before in other threads on ABX here, some factors still are unknowns and we cannot rule out if they are having an effect or not, repeating the procedure with same-different methodology would go a long way to help.
But, I have never seen tests repeated or use this known standard way of testing.
A summary example, that I mention often is this;
It is proven that people unknowingly use anchoring and baseline references to assist with making comparison decisions.
What may be an issue (or not as it has to be proven either way) is that by using ABX we create a flipping scenario where the person's reference and possibly even their anchor bias are flipped creating perception confusion due to two perceived references instead of one and critically that are also randomised.
AB in the ABX is usually randomised for each sequence-iteration but this does not resolve the issue only balance it out across the sequences, IF (emphasised) anchoring or not having a constant and steady reference does cause issues then by randomising AB the effect is hidden within result and critically skews them by in effect making choice selection more random by the participant.
Other consideration is that the participant may anchor or tend to associate some reference to the only perceived constant and that being the button A or B, which again is a risk.
This is why it would be useful to repeat some of the ABX tests where results may seem a bit strange (clearly an engineering difference whether architecture design or subte performance) by validating results with same-different test methodology.
It would remove any possible concerns or unknowns with ABX if results match, and would be an ideal way for validation.
I could go on with some other aspects or more detail that I have mentioned in the past but I think it would become lost, as ABX discussions are here to stay and be repeated often
Cheers
Orb
Last edited:
Neither do I, Orb. I think most pros are using DBT, not just AB, but most of what I've done at home is just AB. I have an ABX comparator, and it's been handy for some things like codecs and hi-rez vs. redbook, but I've never felt compelled to set up the X for anything else. Of course I'm not trying to prove anything, and I'm really only trying to make a point to myself, for my own purposes, and that point is pretty simple: I'm relaxed, at home, listening to familiar material on my own system, trying to decide if a new component in the system is an improvement. If I can't differentiate between it and the old one without looking at them, any sonic differences that might exist are pretty insignificant. And, of course, under those circumstances, I also get to just listen, too -- long term and sighted if I like -- and if I begin to think I hear something, I can go back to blind AB. And if it wasn't there the first time, it always disappears again. That has never failed for me. Does that prove anything? It does to me. But I haven't bothered to set up an AB comparison on anything in my system in a couple of years. Why? I'm not in the market for anything but music. Why? Because the AB experiences I went through a few years ago taught me that most of this stuff Audiophiles talk about, if it is real at all, is insignificant compared to moving my listening chair a couple of feet. And that knowledge set me free.
Tim
Intelligibility is often an issue in live sound because the rooms tend to be large and reverberent. In general, listening rooms are usually too small and too well damped for intelligibilty to be a big issue.
However, I have seen some audiophile complaints about intelligibilty. They are more common with multichannel systems because of the need to balance the levels among the front channel properly, the variations in the mixing of the recordings, and the fact that putting more speakers into a room can tend to excite more resonances and echoes.
Last edited:
That is a false conclusion. Reasearchers at NRC were doing ABX tests for a number of different chores including developing speakers, at about the same time we were developing ABX for our own use. The Vanderkooy/Lipschitz papers about polarity were written as a consequence of work that we did together. My recollection is that I read about "A/B/X" tests that V&L were running and called Stan up, introduced myself, and told him about what we were doing. Since Waterloo is fairly close to Detroit, Stan and John volunteered to drive over and meet with us at one of our regularly scheduled club meetings. They had a simple A/B/X comparator that was a little less elaborate than ours, but still perfectly usable. The results of that meeting ended up providing the data for the V&L AES paper about the audiblity of polarity. Excluding polarity as a strong sound quality determining factor in the design of speakers headed off a number of trips down rabbit holes.
Looking at that article, it seems to be a nice compendium of audiophile myths:
Audiophiles often behave as if every amplifier sounds different. The buy amplifiers like many people buy socks. The science says that some do sound the same, and some don't. If a power amp passes a straight wire bypass test with a well-designed speaker, its no great shakes.
Audiophiles often behave as if every digital player sounds different. The buy music players like many people buy socks. The science says that some do sound the same, and some don't. It is more likely that music players sound the same than that amplfiiers would sound the same. Diigtal music players can generally be expected to pass a straight wire bypass test - it is no big deal and not technically hard to do, but many don't.
In fact the job of an interconnect is very simple, and while it is not all that usable, a collage of coat hangers can do a perfectly servicable job of connecting a line level output to a line level input. One little dirty little secret is that in many contexts, shielding isn't that much of an issue. Many audiophiles have been using high end cables that are unshielded for years. The impedances and voltage levels in a line level interconnect is such that conductivity isn't that much of an issue, either. Besides, heavy gauge steel wire may actually have better conductivity than a far smaller diameter copper wire.
A well-made MP3 can be indistinguishable CD, or not. Some musical program material can be a big challenge for even 320K bitrate MP3s.
If they sound different, its because the mastering was different, which is often the case. Remasternig can be a quick and dirty process. It's easy to make a recording sound different, but if it was prepared with any care at all, it can be very challenging to actually make it sound better.
Well, if moderate resolution PCM (CD) sounds the same as SACD (DSD under a different name) then this naturally follows. There is in fact no scientific/rational reason why DSD would sound different than PCM, all other things being equal.
Thanks for pointing out the site, the samples would be very useful for testing "transparency" of systems. Unfortunately, what is described there is good for distinguishing variations in audio with fixed gear, and I was talking about distinguishing variations in gear with fixed audio. The latter is what most people are worried about, which in turn brings in all the problems I was alluding to ...
Frank
Here is the thread I statrted about subjective v objective: http://www.whatsbestforum.com/showthread.php?1363-Harry-Pearson-at-RMAF-2009
Post #7,8,&9 give an example of a true DBT protocol for a new a medicine. It aso shows Harmon has gone well beyond ABX.
Post #7,8,&9 give an example of a true DBT protocol for a new a medicine. It aso shows Harmon has gone well beyond ABX.
...and never gone back to the useless foolishness of sighted listening tests that most Audiophiles still stand by as the only legitimate arbiter of sonic quality.
Tim
Arny it was not a conclusion and no need to shoot me
This was an observation when looking at various scientific studies and noticed I only mentioned speakers as there are many scientific papers available from Floyd and Sean and co, and relating to the comment of being involved in development of audio products.
Examples include:
Loudspeaker Measurements and Their Relationship to Listener Preferences: Part 1
Loudspeaker Measurements and Their Relationship to Listener Preferences: Part 2
The Modification of Timbre by Resonances: Perception and Measurement
The Detection of Reflections in Typical Rooms
Differences in Performance and Preference of Trained versus Untrained Listeners in Loudspeaker Tests: A Case Study
Then more recently we have the study of room correction products, amongst other studies that could be mentioned.
It is fair to say all of these scientific studies influenced product development, but none used ABX, however I appreciate there maybe scientific research papers by them that are ABX, might be worth listing some that you feel were used in product development as it would help.
Cheers
Orb
Last edited:
That's another thread Tim: Are sighted listening testis usless and foolish? Maybe you should statrt it.
Arny,
not sure because my statement stands that ABX seems to be limited to the domain of audio or used incredibly sparingly , while other disciplines used same-different and similar preference methodology both years ago and today, and in fact it is part of certain international standards and, reinforced by still being used in critical disciplines such as psychology that specialises in biases.
Maybe you mean when it comes to the subject of audio, however if it was a predecessor where is the validation papers showing comparisons of data-results between same-different and ABX?
But my earlier points still stand relating to the unknowns that I mentioned (and more so more in previous threads) cannot be resolved with current results from ABX studies that in theory may (need to be emphasised so no-one sees it as an actual known fact) skew them, and hopefully one day we will see modern same-different methodology used to remove any unknowns and validate ABX in the scenario of small or no difference between two similar products.
BTW what did you find insensitive about such a methodology and frustrating?
Anyway, it is is a surprise because ASTM International has found that it is useful due to its sensitivity, and has it as a recommendation-standard, where it has been discussed at length and its suitability with various panel-comittee experts.
Also has anyone ever seen ABX and same-different methodology combined together to validate, it may exist but I have only ever seen ABX on its own, or as mentioned earlier other DBT audio studies following more the preference or same-different methodology.
However the best source would be if there is any studies showing both ABX and same-different methodology when it comes to comparing near identical or similar sounding products, and as it has been done as you say hopefully there are papers somewhere showing the results and problems encountered using same-different methodology in this situation, which if anyone can link would be greatly appreciated.
Cheers
Orb
Last edited:
Nah. Not worth the digits. The truth is that sighted listening is a great joy; it's why we're all in the hobby in the sense that we all know what components are in our systems. But it "tests" nothing, so the premise of the thread would be flawed.
Tim
That's pretty inconsistent with what you have said elsewhere.
What is a "well-made" MP3?
I think it's pretty obvious ABX is obsolete!. Obsolete means it's outlived its usefulness not that it never had any according the Meriam Websters dictionary. At least it got us thinking about blind testing. And that some of our sacred beleifs might be wrong.
The problem of course is that it's voluntary. My observations show that people choose test methods that reinforce what they already beleive or at least hope is true. Genrally speaking if you think cables make a difference no need for blind testing. If you think cables are a l"oad of crap "you want an ABX test to prove it. Very few say, I don't kniow why don't we do a DBT and find out. Iroinically proponents of ABX actually do it as infrequently as the skeptics.
The problem of course is that it's voluntary. My observations show that people choose test methods that reinforce what they already beleive or at least hope is true. Genrally speaking if you think cables make a difference no need for blind testing. If you think cables are a l"oad of crap "you want an ABX test to prove it. Very few say, I don't kniow why don't we do a DBT and find out. Iroinically proponents of ABX actually do it as infrequently as the skeptics.
Please explain further what you mean by distinguishing variations in gear with fixed audio and distinguishing variations in audio with fixed gear.
I keep forgetting that you have these intentionally opaque blinders on, and so it seems like you will only consider evidence that is peer-reviewed and published in first-tier journals. It truely is a case of blinders because a great deal of product development never gets any formal papers written about it. Many of the methodologies used to develop products are proprietary and considered by its owners to be trade secrets. If the curtains are pushed aside momentarily and we see something interesting, you have forbiden that it be discussed.
I would like you to cite a peer-reviewed paper that was written in the past 10 years and advises the use of sighted evaluations during the development of SOTA audio products.
Please explain, based on true quotes prestented within the context they were origionally posted.
amir said:
One where good effective choices were made with resepect to the encoder and its parameters.
- Status
Similar threads
