Is ABX finally Obsolete

[Phelonious Ponk]

I think that what you just said is that you want to hear the differences that you want to hear and you don't want to hear the differences that you don't want to hear.

Unfortunately, the real world isn't that selective. If you seek for truth, you will no doubt find a mixture of what you want to know, and some things that you really didn't want to know.

There is a lot of that going around, Arny, no question about it. But I also think it is absolutely legitimate, and not just dodging the truth, to draw a line and say there are some things you won't bother to train yourself to hear. An example -- people talk about truncated reverb trails and softened attack on redbook files vs. hi-rez. I have no idea if that's real or not; Meyer and Moran seems to indicate that it's not real for most of us, anyway. But I've seen some pretty respectable audio pros say it is real, and I suppose I'm glad theycan hear it, but I live in a world where the overwhelming majority of the music I want to hear is only available at 16/44.1. 16/44.1, on my system, in my room, playing my best recordings sounds absolutely gorgeous to me. The last thing I want to focus on, and train myself to pull out of the listening experience is what's wrong with it.

Now, with that said, I'm really not afraid. I can listen to my old jazz recordings that are, by today's standards, just plain bad, and still enjoy them, so I think I'd rise above it. But why would I invest in listening for it?

Tim

 

[Gregadd]

You've misunderstood me, Gregg. I believe there are differences -- differences in noise, distortion, frequency response, transient response. What I don't believe in is what audiophiles hear. I can move your chair forward, toe in your speakers and, assuming said speakers are decent, immediately improve your imaging. I can turn them parallel to the wall and expand your sound stage (and smear the imaging a bit in the process). I can tilt them back and heighten your sound stage. I can push back your chair, pull the speakers further apart and, if your room is big enough, expand your soundstage, even with relatively small speakers. I can make a night and day difference.

But I can accomplish absolutely none of the above with a change of cable, tubes, preamp, amplifier or DAC. And I can't make it all collapse into a thin phantom by playing a digital file instead of a record or tape.

***

Tim

I can and so have countless others.

 

[Phelonious Ponk]

I can and so have countless others.

I know many hear these things. I do think they can be counted, though, the high-end audiophile market is small enough, and shrinking.

Tim

 

[microstrip]

(...) Now, with that said, I'm really not afraid. I can listen to my old jazz recordings that are, by today's standards, just plain bad, and still enjoy them, so I think I'd rise above it. But why would I invest in listening for it?
Tim

Tim,
Can you list what you consider your five best and five worst old jazz recordings? May be we can try to get them and debate how they sound in different systems.
One of my great audiophile experiences was listening to Miles Davis "Saeta" from Sketches of Spain in a great system using the LP, decades ago. Later I listened to it in my humble system and my reaction was that the recording was poor.

 

[arnyk]

I can and so have countless others.

I don't hestate to check with other pros when it comes to the usuability of certain mics or consoles, but as far as reliable listening test protocols go, not so much.

Pro audio is all about differences that are readily heard. They abound. They are the bread and butter of audio production.

 

[Phelonious Ponk]

I don't hestate to check with other pros when it comes to the usuability of certain mics or consoles, but as far as reliable listening test protocols go, not so much.

Pro audio is all about differences that are readily heard. They abound. They are the bread and butter of audio production.

I don't think these are the things we're talking about, though, Arny. We (or at least I) are talking about situations in which the recording is not changed, the speakers are not changed, the placement is not moved in the slightest, the room is left untouched...an interconnect, a preamp or a DAC is substituted for another that was perfectly competent at its job and the lover, excuse me, listener, suddenly hears an obvious, nay glorious expansion of the depth, breadth and height of sound stage. Unless the component being replaced was just horrible, such bad crosstalk and channel separation that it was delivering mono at best, I don't see how this effect is even possible with the change of such a component. And no one who hears it has ever been able to explain how it is possible.

Tim

 

[Phelonious Ponk]

Tim,
Can you list what you consider your five best and five worst old jazz recordings? May be we can try to get them and debate how they sound in different systems.
One of my great audiophile experiences was listening to Miles Davis "Saeta" from Sketches of Spain in a great system using the LP, decades ago. Later I listened to it in my humble system and my reaction was that the recording was poor.

I wouldn't call that a debate, micro, I'd call that a wonderful, enlightening conversation I'd love to have. The five worst would be relatively easy: Pick anything from prior to about 1955, certainly 1950, before the big technological leap in tape, whatever that was, happened that ushered in the golden era of analog. I love some of these recordings - from original delta blues artists to most of my Charlie Parker recordings. The five best? That's going to take some serious thought, though any 5 I pick will just be the ones I pick. There will be many more that are equal. But here are a few. For mono, I love Mile's "'Round About Midnight." Early stereo? Sonny Rollins' Saxaphone Collosus, Monk's "Brilliant Corners," Bill Evans' "Explorations in Jazz." Live? "Waltz for Debby," Dianna Krall's "Live in Paris." Modern digital studio album? Herbie Hancock's "Gershwin's World," Madeleine Peroux's "Bare Bones," Krall's "The Girl In The Other Room," Joni Mitchell's "Travelogue." And I don't really know if all of those were recorded digitally, nor do I care.

I'd really have to listen before I would commit to those choices because, frankly, I'm sure that I'm more influenced by the music than I am the production values. The best? Who knows. Very, very good, at least.

Tim

 

[arnyk]

I don't think these are the things we're talking about, though, Arny. We (or at least I) are talking about situations in which the recording is not changed, the speakers are not changed, the placement is not moved in the slightest, the room is left untouched...an interconnect, a preamp or a DAC is substituted for another that was perfectly competent at its job and the lover, excuse me, listener, suddenly hears an obvious, nay glorious expansion of the depth, breadth and height of sound stage.

That's not how things are in pro audio. There is nothing like having the actual live performance right there before you and having the full function of a full complements of mics, mixer and EFX at your disposal.

 

[arnyk]

OK it still seems more anecdotal and investigative than scientifically conclusive to me.
The 1st test mentioned was a bit relaxed for several reasons; primary one that the switch box listeners were given a 45 minute training session 1st and importantly a reference point/anchor, while the extended listeners were told to use whatever technique they wanted to, and as Tom mentions that group were not controlled.
Also critically the switch box users had an anchor with that training in 1st listening to the worst level of distortion of 13%.
In the 2nd test done by Tom, we have the anchor type training situation again where a previously failed listener of the extended listening is used for the quick switch and 1st listens again to 13%, but those given a disc for lengthy listening had no baseline/anchor/or training it seems (can only go by whats in the article) on the distortion signal.

The issue for the home listeners anyway is simple; they had no reference point and no training with the worst distortion, and critically no repeated trial runs involving both the clean and dirty CD that would be important for several reasons.
So while it is an interesting article currently to me it is more about being investigative rather than scientifically evident or conclusive IMO, and I appreciate not everyone will feel the same way.
But I can only go what is in the article, and maybe more has been done as a follow-up to it.
I just want to stress though that I am not being critical of the article because doing extended/long term listening as a controlled variable while also managing listening training and having usable test data at the end would be a nightmare and freakingly expensive.

With all that said even if one does feel different about the test it does not help with the factors I have raised in other posts.

Now in the interest of fairness, please apply the same criteria to sighted listening evaluations performed by:

Stereophile, TAS, etc.
Your typical audiophile posting to a web site, such as this one.

 

[Orb]

Arny no disagreement, as I say one has to consider again bias-heuristic-cognition behaviour even for this, and I gave even a simple example of how we skew limited blind test results (say if only some of the components are blind) if not careful by not even making speakers' position blind to the listener, which can give a different preference result value to seen or truly blind and unknown position.
So sighted has full list of bias-heuristic challenges for the listener and the reviewer, sadly I agree most seem to be very poor at understanding this while a rare few seem very good.

But this also has to then consider debiasing processes that can be applied albeit it takes a very analytical/methodical approach and a disciplined framework to get any benefits, and this may not work with biases that are actual chemical/brain related triggers but again this needs further investigation and several different studies, however currently debiasing investigations is being done outside of audio but enthusiasm for this is low in general.
It is unfortunate some of those factors I raised are not known in greater detail, because they would be of interest also on what you raise relating to reviewing and how they may interact with the reviewer (they could be beneficial or they could be detrimental but we do not know for sure and rely upon additional anecdotal evidence).

Anyway it would be great if all of this could be applied against unknowns of extended listening preferences that anecdotally may suggest our initial biases or some are then possibly overcome; examples include those who eventually feel something is not right-listening fatigue-notice their listening behaviour has changed to be more restless and flicking-etc.
Emphasis here is anecdotally until we get some studies investigating those factors mentioned.
My point was not about defending a specific approach, but highlight something that would be great further studies, which sadly has only happened in other subjects and not audio, but then none of what we are discussing is going to be a gamechanger.

One bias related test that could had been repeated and would be interesting is looking to see how certain biases behaved when dealt with analytically by the participant or participants who work with those biases in the same sector and critically also debiasing techniques/approach;
a good example is the mOFC study of wine tasting and looking at their enjoyment vs cost price that showed a good link between enjoyment triggers and expectation against price, it would had been very interesting if the experiment was repeated utilising debiasing process and another test with participants who understand and work with the mechanisms related to bias in that sector (this example would be product leveraging/related marketing/etc).
Just thoughts anyway and wishful thinking on my part

Cheers
Orb

 

[Gregadd]

As the original poster I think we are getting lost in the results. It's the process we should consider.

I think someone else raised the quesion-How doies the ordinary listener match levels?

 

[Gregadd]

http://www.moultonlabs.com/more/what_is_the_sound_of_one_amp_clipping/P4/

I would recommend matching levels at sample frequencies across the spectrum 100,500, 1000, 5000,12,000hz.

 

[fas42]

ABX was invented by people who were so interested and involved in the process of doing listening tests and obtaining sensitive and reliable results from them, that they invented ABX at a very signficiant personal cost in terms of time and effort.

You misunderstood my point. I was referring to the people who were the subjects of the test, not those conducting it.

Which brings me to another point, being that the whole environment of the ABX test arena is a system, complete in itself, which causes a certain level of sound quality to be experienced by those in earshot. What does mean? Well, many people here love to point out that the listening environment is terribly, terribly important, almost saying that it is the most crucial thing in the whole shebang, and guess what, that's also the case in an ABX test.

Scientists love the concept of sticking the subject under scrutiny in a box, being able to stand outside it, and saying that he, the experimenter, is completely divorced from the test object, what he's all about and how he behaves is completely irrelevant to the results. Well, the history of science is littered with failures of this process, often because of sloppiness, but many times also because there is an agenda in the mix being pursued by the supposed dispassionate scientists ...

ABX will work fine in audio at the macro level, dealing with fairly obvious discrepancies between different scenarios; once you intend to distingush far finer points of differentiation, the experiment will have to be set up with exquisite care, by those with absolutely no barrows to push ...

Frank

 

[Orb]

As the original poster I think we are getting lost in the results. It's the process we should consider.

I think someone else raised the quesion-How doies the ordinary listener match levels?

The problem I feel though Gregadd is that the article does not cover anything that I have just been discussing with Arny or other factors but seems to focus on the usual and is not very helpful either way; for those who support or have reserved feelings towards ABX.
Even discussing matching levels could be lengthy as I would argue that its key use is removing cues or tells and critically ensure the factor is controlled to assist with usable end test data.
Others may put forward that there is a preference for louder but it can also be said that there is an ideal loudness for any track, which actually may be lower.
While we have the thresholds for loudness over FR, this does not necessarily help in concluding how two different volumes affect the listeners choice subjectively when they concentrate on soundstage,timbre,etc.
There is anecdotal evidence for both sides of this, but what is universal is that level matching is essential when considered as a controlled variable (to assist with test data at the end) and to remove cues/tells, of course the last point about cues/tells can be removed if the volume can be reset in a way that the listener cannot establish its position or volume knob turn to each AB.

What I am interested is in whether there is actual perception bias affecting soundstage/timbre/etc when the listener is trained in both loudness and notes-scales (pitch);
in those instances consider those who play in a symphony and how they must play note and nuance perfect for different loudness of passages and as solo or part of the section and whole symphony, or those who are master tuners of instruments by ear.
Both break the proposal that volume by default affects and changes a listener's sound quality-pitch-etc, and there are plenty of other examples.
Problem is this needs further studying so to me it seems we are currently having to make assumptions based on what studies have been done, but this means we need to be careful how we suggest loudness affects a listener and cannot generalise beyond my points above on why we need or when to level match.
That said I do wonder (again not tested) whether a bias is in effect for soundstage/timbre once the volume increase is below a listener's accurate perception of volume differences, so someone may accurately tell there is a 0.3db to 0.5db volume difference and as far as they are concerned sound quality is identical, but what happens at say 0.2db; nothing or does perception become affected (this would be marginal difference) for even trained listeners.

Another aspect to consider that shows how level matching fluctuates a lot in reality;
Output impedance of the amp against the loading by the speaker (average amp will fluctuate between 0.3db and 0.7db with tubes being greater), speakers do not have uniform sensitivity although manufacturers suggest they do and critically behave much worse with music and this can vary by up to 6db peak to dip, one could then expand this to look at Class D (beyond Hypex and Primare where both have exceptionally low output impedance across frequency), frequency suckout in speakers,etc.
However the consideration here and is important; these are affected by part of the frequency range so a fluctuation within the music, instead of being flat across the whole frequency and so more linear.

Cheers

Orb

 

[arnyk]

As the original poster I think we are getting lost in the results. It's the process we should consider.

I think someone else raised the quesion-How doies the ordinary listener match levels?

Best done electrically, not acoustically. Use a steady signal source, such as a CD with tones maybe a minute or mreo long at 20, 50, 100, 1000, 10K 15K and 20K. Measure the voltage at the speaker terminals as you change between the two options in your test. Use a DVM with at least 4 digit read out, and match levels within 1% if possible. Tolerances as great as 5% are tolerable at the frequency extremes.

 

[arnyk]

You misunderstood my point. I was referring to the people who were the subjects of the test, not those conducting it.

ABX was always designed to be a self-administered test. The first thing that the people who developed it did, was test themselves. The basic idea was a test that an amplifier desgner or speaker builder could use all by himself and still have assurance of good bias controls.

Which brings me to another point, being that the whole environment of the ABX test arena is a system, complete in itself, which causes a certain level of sound quality to be experienced by those in earshot. What does mean? Well, many people here love to point out that the listening environment is terribly, terribly important, almost saying that it is the most crucial thing in the whole shebang, and guess what, that's also the case in an ABX test.

To me just another case of people being hypercritical about ABX tests, and then marching right out and making far-reaching judgements about audio gear based on sighted, non-level-matched, unsynchronized evaluations with massive switchover times and corresponding loss of sensitivity.

Scientists love the concept of sticking the subject under scrutiny in a box, being able to stand outside it, and saying that he, the experimenter, is completely divorced from the test object, what he's all about and how he behaves is completely irrelevant to the results.

In the case of ABX, that may be a story that someone wants to tell themself, but reality is that the first ABX test used as its first test subject the guy who developed the test. The first group test was done using a listening panel composed of the people who stimulated the development of the test in the first place. I'm never going to test a number of people that compares with the vast number of times I've tested myself.

My last ABX test was administed yesterday by me to me, and related to the files that you can download from this web page:

http://home.comcast.net/~arnyk/pcabx/jitter_power/index.htm

The piano is well known for being very diagnositic for power line jitter.

The jitter in the -20 dB sample is huge, compared to real world jitter. Can you hear it?

Here is a video showing how to run an ABX DBT test on your own computer:

http://www.youtube.com/watch?v=jt7GyFW4hOI

The samples I provided above have so much jitter that its hard to imagine a modern computer with audio interface that is so bad that it masks it. The use of good headphones or earphones is advised.

Here is the foobar download site:

http://www.foobar2000.org/download

Here is the download source for the ABX add-on for foobar:

http://www.foobar2000.org/components

 

[microstrip]

Something that worries me (as much as audio can worry me ) about ABX is that I can not find valid reports of tests with positive results, such as using ABX to establish a firm threshold of audibility for something.

I find hundreds of tests proving that things are not audible in the particular conditions of the test, but nothing really useful to increase our audio knowledge. BTW, I am only addressing audio hardware, not software, codecs, audio files, recordings or similar.

 

[arnyk]

Something that worries me (as much as audio can worry me ) about ABX is that I can not find valid reports of tests with positive results, such as using ABX to establish a firm threshold of audibility for something.

Been there, done that. Got answers that were the same as the book answers or maybe a few dB better, but that was probably due to using a different choice of listeners. The probable reason that nobody published anything about thresholds with ABX is that there is little or no news. ABX finds pretty much the same thresholds of hearing as you read about in books, or maybe a bit better. I haven't tested enough different people to come up with a convincing argument that ABX is more sensitive than the traditional sources.

BTW the equipment that was used in the tests that origionally set these thresholds is generally pretty horrible by modern standards. Just shows that either the ear is very adaptable or that all this heavy breathing about the resolving power of modern equipment is just that - heavy breathing.

I find hundreds of tests proving that things are not audible in the particular conditions of the test, but nothing really useful to increase our audio knowledge. BTW, I am only addressing audio hardware, not software, codecs, audio files, recordings or similar.

You expect that people's ears are changing any way but worse?

 

[microstrip]

Been there, done that. Got answers that were the same as the book answers or maybe a few dB better, but that was probably due to using a different choice of listeners. The probable reason that nobody published anything about thresholds with ABX is that there is little or no news. ABX finds pretty much the same thresholds of hearing as you read about in books, or maybe a bit better. I haven't tested enough different people to come up with a convincing argument that ABX is more sensitive than the traditional sources.

BTW the equipment that was used in the tests that origionally set these thresholds is generally pretty horrible by modern standards. Just shows that either the ear is very adaptable or that all this heavy breathing about the resolving power of modern equipment is just that - heavy breathing.

You expect that people's ears are changing any way but worse?

You are just confirming the thread tittle in another sense - it seems ABX has not been of any use during the last 30 years for hardware development. Apparently its only practical use was to show there were no improvements in hardware during this period.

I have seen your interesting private experiments and tests but I am addressing tests with real consequences in audio hardware development.

 

[arnyk]

You are just confirming the thread tittle in another sense - it seems ABX has not been of any use during the last 30 years for hardware development. Apparently its only practical use was to show there were no improvements in hardware during this period.

A rather far-reaching conclusion based on exactly what?

Were there no hardware improvements during the past 35 years? Of course not! The past 35 years saw the completion of the transition to solid state, and the introduction and maturity of practical digital audio, as well as dramatic performance improvements in transducers. Measured performance, price performance, and convenience improved dramatically.

What didn't change much is the sensitivity of human hearing. The evolution of the human organism is well known to move relatitvely slowly compared to the rate of change of modern technology. Everybody who is surprised about that has my permission to sit in the nearest corner and wear the uniform of the day which includes a pointed hat! ;-)

Interesting that you completely ignored my comments to that end... Does that belie an underlying belief in the infinite sensitivity of human sensations which is well known to be a dream, and not real?

I have seen your interesting private experiments and tests but I am addressing tests with real consequences in audio hardware development.