Is ABX finally Obsolete

[microstrip]

A rather far-reaching conclusion based on exactly what?

Were there no hardware improvements during the past 35 years? Of course not! The past 35 years saw the completion of the transition to solid state, and the introduction and maturity of practical digital audio, as well as dramatic performance improvements in transducers. Measured performance, price performance, and convenience improved dramatically. (...)
Interesting that you completely ignored my comments to that end... Does that belie an underlying belief in the infinite sensitivity of human sensations which is well known to be a dream, and not real?

I come back to my main question - do you have traceable evidence that positive identification using ABX took part in the audio hardware developments of the last 30 years ?
BTW, I will deliberately ignore comments that are not relevant to my main point I am addressing. At no point I was interested "infinite sensitivity of human sensations" in this post.

 

[Robh3606]

I come back to my main question - do you have traceable evidence that positive identification using ABX took part in the audio hardware developments of the last 30 years ?

Have you read Tooles book?

Rob

 

[arnyk]

I come back to my main question - do you have traceable evidence that positive identification using ABX took part in the audio hardware developments of the last 30 years ?

http://www.sonicdesign.se/tooleinw.htm

1. Please tell us about when and how you came to see the importance of ABX tests in audio?

Before I joined Harman International, I was a research scientist, at the National Research Council in Canada. In the 1970's I realized that there was a need for some serious research into the psychoacoustics of loudspeakers and rooms, and I examined the literature that existed at the time. There were plenty of measurement techniques, to generate technical data - graphs and numbers - but there were no reliable rules for interpreting them in ways that related to human perceptions. At the same time, there were people saying that "we all hear differently", and that there can be no rules for what sounds good. I simply did not believe that, and decided to test it. Looking into the methods of experimental psychology, it was clear that blind,and double blind testing was necessary, and that it was also necessary to control other factors, such as loudness, if we were to have any hope of examining the true opinions of listeners. When we did listening tests with even very simple experimental controls, we found that most people, most of the time, liked and disliked the same loudspeakers. There were exceptions, and when we examined why, we found that hearing performance was a main factor. People with close to normal hearing thresholds, and who had some experience in critical listening, all agreed very closely in their preferences. So, it is true that we do not all agree on what is good, but those persons with normal hearing, and some skill in making judgments about sound quality, show remarkably similar opinions. When people differ from this group they differ in random ways, so it is not possible to rely on such listeners for guidance or advice. All of this is described in great detail in:

1. F.E. Toole, “Listening Tests, Turning Opinion Into Fact”, J. Audio Eng. Soc., vol. 30, pp. 431-445 (1982 June).

2 . F.E. Toole, “Subjective Measurements of Loudspeaker Sound Quality and Listener Performance”, J. Audio Eng. Soc., vol 33, pp. 2-32 (1985 January/February)

A more recent description can be found in a chapter in the Focal Press book, "Loudspeaker and Headphone Handbook" edited by John Borwick, a new edition (the 3rd) will soon be released. ABX testing is just one of many techniques for evaluating sound quality. It is very useful for settling "is there an audible difference" kinds of tests - e.g. wires, CD players, amplifiers, perceptual encoders. The results of such tests are, ideally, yes or no. For loudspeakers, the differences are clearly audible, and the question is more one of preference and why there is a preference, so we use multiple comparison techniques, which give listeners a better "context" within which to form what is a very complicated opinion.

 

[microstrip]

F. Toole is known for his position of stating that all competently designed electronics sounds the same - it is not an unbiased view. Nothing he says answers to my request - direct traceable evidence of results connected to audio development.

 

[microstrip]

Have you read Tooles book?

Rob

Yes, I have it. But surely I skipped some parts, as it is fascinating but long book. This does not mean we have to agree with everything stated in it.

 

[Phelonious Ponk]

Micro, there is little doubt that most of the ABX testing being done in audio comes to the conclusion that the differences between A and B were not audible in the test. I think, however, that is to be expected. We don't ABX test the audibility of stuff that is clearly audible. We ABX test the audibility of stuff that is, at least, very arguable (from one side of the argument, one might say "questionable"). On top of that, I think a lot of audible - in the strictest sense of the word - stuff has been tested via ABX and not heard in the tests because to hear it you need to know exactly what you're listening for and exactly where to listen for it, ie: you have to be trained to hear it. A couple of things I think may fall into this category are artifacts from high bitrate codec compression and the differences between redbook and true high-rez files. I believe there are differences there. I believe if I were trained, like Amir is, to hear them, that I would - especialy in (only in?) a rapid-switching blind comparison. But they aren't likely to be indentified in blind listening by civilians or even audiophiles.

What that makes them, IMO, is not inaudible, but irrelevant.

Tim

 

[Phelonious Ponk]

Yes, I have it. But surely I skipped some parts, as it is fascinating but long book. This does not mean we have to agree with everything stated in it.

I don't think he's asking you to agree with everything in the book but offering it up as evidence that ABX is being used in audio product development. Is it being used a lot? I don't know, but I doubt it is being used much in the high end. A big chunk of that part of the business rejected it long before they gave it a shot.

Tim

 

[arnyk]

F. Toole is known for his position of stating that all electronics sounds the same - it is not an unbiased view. Nothing he says answers to my request - direct traceable evidence of results connected to audio development.

The above exception to Toole's viewpoint is stated as an excluded middle argument. If taken at face value it appears to be just another transparent ploy by a highly biased unknown individual who hides behind an alias to discredit a great man, an industry leader, and a great scientist.

Microstrip or whatever your real name is, if you can't provide reliable evidence that Toole's view is indeed biased, then your statement discredits itself.

So, where is your reliable scientific proof that all electronics sounds different?

 

[Phelonious Ponk]

The above exception to Toole's viewpoint is stated as an excluded middle argument. If taken at face value it appears to be just another transparent ploy by a highly biased unknown individual who hides behind an alias to discredit a great man, an industry leader, and a great scientist.

Microstrip or whatever your real name is, if you can't provide reliable evidence that Toole's view is indeed biased, then your statement discredits itself.

So, where is your reliable scientific proof that all electronics sounds different?

Micro you don't need O'Toole's book. If you want evidence that blind listening testing is being used in product development (was that really your question?) you only need to read Sean's blog. Harman does it all the time.

Tim

 

[microstrip]

The above exception to Toole's viewpoint is stated as an excluded middle argument. If taken at face value it appears to be just another transparent ploy by a highly biased unknown individual who hides behind an alias to discredit a great man, an industry leader, and a great scientist.

Microstrip or whatever your real name is, if you can't provide reliable evidence that Toole's view is indeed biased, then your statement discredits itself.

So, where is your reliable scientific proof that all electronics sounds different?

Perhaps I used the words biased in the wrong way - I wanted to mean that he does not support ABX with the evidence I was asking in this interview, and he was just stating that differences between electronics are very small.
Anyway , once gain I am out of debates with you - you are a master in exploiting others linguistics faults. Happy victory!

 

[Ron Party]

F. Toole is known for his position of stating that all electronics sounds the same - it is not an unbiased view.

Can you supply our readers (and me) with a reference for this assertion? I (and I suspect anyone being intellectually honest) find it difficult to believe anyone, and I mean anyone, believes all electronics sound the same.

While we're at it, perhaps we also need a definition here as to the term "bias". Is one *biased* for believing the perceptual world can be studied using the scientific method, or, the corollary, is one *biased* for believing the perceptual world cannot be studied using the scientific method?

 

[microstrip]

Micro you don't need O'Toole's book. If you want evidence that blind listening testing is being used in product development (was that really your question?) you only need to read Sean's blog. Harman does it all the time.

Tim

Tim,
My point was (and is finished now... ) about ABX not DBT.

 

[microstrip]

Can you supply our readers (and me) with a reference for this assertion? I (and I suspect anyone being intellectually honest) find it difficult to believe anyone, and I mean anyone, believes all electronics sound the same.

While we're at it, perhaps we also need a definition here as to the term "bias". Is one *biased* for believing the perceptual world can be studied using the scientific method, or, the corollary, is one *biased* for believing the perceptual world cannot be studied using the scientific method?

Ok, I hope everyone of you will be pleased "Every competently designed electronics will sound the same", "differences between correctly designed electronics are small" . I missed the "competently designed" when I typed the post with the quote from the Toole book.

 

[arnyk]

Perhaps I used the words biased in the wrong way

Now you are saying that your words were biased against you?

I wanted to mean that he does not support ABX with the evidence I was asking in this interview, and he was just stating that differences between electronics are very small.

I'm going to reproduce what he said and ask you to please show where he said anything at all about differences between electronics

http://www.sonicdesign.se/tooleinw.htm

1. Please tell us about when and how you came to see the importance of ABX tests in audio?

Before I joined Harman International, I was a research scientist, at the National Research Council in Canada. In the 1970's I realized that there was a need for some serious research into the psychoacoustics of loudspeakers and rooms, and I examined the literature that existed at the time. There were plenty of measurement techniques, to generate technical data - graphs and numbers - but there were no reliable rules for interpreting them in ways that related to human perceptions. At the same time, there were people saying that "we all hear differently", and that there can be no rules for what sounds good. I simply did not believe that, and decided to test it. Looking into the methods of experimental psychology, it was clear that blind,and double blind testing was necessary, and that it was also necessary to control other factors, such as loudness, if we were to have any hope of examining the true opinions of listeners. When we did listening tests with even very simple experimental controls, we found that most people, most of the time, liked and disliked the same loudspeakers. There were exceptions, and when we examined why, we found that hearing performance was a main factor. People with close to normal hearing thresholds, and who had some experience in critical listening, all agreed very closely in their preferences. So, it is true that we do not all agree on what is good, but those persons with normal hearing, and some skill in making judgments about sound quality, show remarkably similar opinions. When people differ from this group they differ in random ways, so it is not possible to rely on such listeners for guidance or advice. All of this is described in great detail in:

1. F.E. Toole, “Listening Tests, Turning Opinion Into Fact”, J. Audio Eng. Soc., vol. 30, pp. 431-445 (1982 June).

2 . F.E. Toole, “Subjective Measurements of Loudspeaker Sound Quality and Listener Performance”, J. Audio Eng. Soc., vol 33, pp. 2-32 (1985 January/February)

A more recent description can be found in a chapter in the Focal Press book, "Loudspeaker and Headphone Handbook" edited by John Borwick, a new edition (the 3rd) will soon be released. ABX testing is just one of many techniques for evaluating sound quality. It is very useful for settling "is there an audible difference" kinds of tests - e.g. wires, CD players, amplifiers, perceptual encoders. The results of such tests are, ideally, yes or no. For loudspeakers, the differences are clearly audible, and the question is more one of preference and why there is a preference, so we use multiple comparison techniques, which give listeners a better "context" within which to form what is a very complicated opinion.

For example he said:

ABX testing is just one of many techniques for evaluating sound quality. It is very useful for settling "is there an audible difference" kinds of tests - e.g. wires, CD players, amplifiers, perceptual encoders. The results of such tests are, ideally, yes or no. For loudspeakers, the differences are clearly audible, and the question is more one of preference and why there is a preference, so we use multiple comparison techniques, which give listeners a better "context" within which to form what is a very complicated opinion.

I see no place above where Floyd Toole said anything that could be logically construed as you claim, that the differnces between electronics are very small. Please help me out here.

Where in the above comments from Floyd Toole above was "he was just stating that differences between electronics are very small."?

 

[Phelonious Ponk]

Tim,
My point was (and is finished now... ) about ABX not DBT.

It's a pretty fine line, micro...

BT - blind test
ABX - blind test with a control -- "is X A or B"
DBT - double blind test - even the person conducting the test doesn't know which choice you're listening to.

Where there is a significant difference is in use. ABX is most commonly used, obviously, to determine if there is a perceptible difference at all (audible in our case). Thus the X. And while that is probably not as common in audio product development as testing for preference (BT and DBT), it wouldn't surprise me at all to hear it is being used by science-oriented companies like Harmon, who understand the power of human psychology, and know that if they are testing for preferece between subtle differences, people are likely to perceive a preference, and state one, even when they can't actually hear a difference.

Boy could that booger some test results.

Tim

 

[Gregadd]

To Properly CYA "Every competently designed electronics will sound the same", "differences between correctly designed electronics are small" . you should add, working within its range.

Ron I think our biases are usually the result of our collective experience. I stated audio firmly in the measurement camp. If we have no experience we adopt the opinion of others.

Ideally we would test first and then form an opinion. All too often we form an opinion and seek confirmation by whatever means available.

 

[Ron Party]

Ideally we would test first and then form an opinion. All too often we form an opinion and seek confirmation by whatever means available.

So true. Of course, the elephant in the room is how do we go about testing.

 

[arnyk]

On top of that, I think a lot of audible - in the strictest sense of the word - stuff has been tested via ABX and not heard in the tests because to hear it you need to know exactly what you're listening for and exactly where to listen for it, ie: you have to be trained to hear it.

This raises the question of why people have been doing so many sighted evaluations for so long without feeling an acute need for listener training?

One answer is that people seem to test until they hear a difference, and since sighted evaluations are rife with false positives, they thought they were hearing a difference right from the start when in fact the differences that they thought they were hearing were just false positives.

 

[arnyk]

Tim,
My point was (and is finished now... ) about ABX not DBT.

Problem is that all of the DBT protocols that people have tried pretty much give the same results. For example, ABC/hr is a testing methodology that is significantly different from ABX. Yet when you try to do ABC/hr testing when A and B are very similar, your different and far more complex mathematical analysis produces results that turn out to tell the same story - the listener was guessing randomly.

The actual comparision is between all of the different flavors of sighted evaluations versus all of the different flavors of bias-controlled tests.

Furthermore, it seems to be common for audiophiles to perceive all DBTs as being the same basic thing, and call that thing by the name they are most familiar with - ABX.

 

[Phelonious Ponk]

This raises the question of why people have been doing so many sighted evaluations for so long without feeling an acute need for listener training?

One answer is that people seem to test until they hear a difference, and since sighted evaluations are rife with false positives, they thought they were hearing a difference right from the start when in fact the differences that they thought they were hearing were just false positives.

And as long as what you're hearing supports your beliefs and reinforces your purchasing decisions, your hearing is brilliant. No training necessary.

Tim