Is ABX finally Obsolete

[Phelonious Ponk]

Bias produces prejudice. Prejudice produces a false consistency. Look at Stereophiles' recommended components lists. I just don't see that type of consistency there.

Greg we will probably disagree, and that's ok, but I don't think it is bias that produces a false consistency in blind testing. I'm not at all sure there is a false consistency in blind testing. That conclusion is a result of the very high number of negatives resulting from blind testing, and while there are a lot of crappy tests being done and I'd never argue that there are no false results, I would expect nothing less than a lot of true negatives. Why? Because we don't blind test Revel Salons vs. Klipschorns or vinyl vs digital. These differences are easy to spot. What we test is the stuff that, at best, is on the edge of audibility. When we test for differentiation, we test the differences that are very subtle, the things that are particularly hard to tell apart. We often find, and it should be no surprise, that when sighted knowledge is taken away, those subtleties are very hard to detect, and that they may vanish entirely. That's not shocking to anyone but the guy who thought he heard a difference before he closed his eyes. On the other hand, blind listening isn't just for testing audible differences. It can also be used to test preference without the bias of sight, and the folks at Harman do this often. Do they get a lot of false negatives? No. They get true positives. Their listeners hear differences between things that clearly sound different, and choose their preferences among them.

Tim

 

[Gregadd]

I have some serious problems with Harmon training people what to look for and then designing products that excel in those categories. Anyway all my opinions are free! I don't set standards or sell anything.

 

[Phelonious Ponk]

I have some serious problems with Harmon training people what to look for and then designing products that excel in those categories. Anyway all my opinions are free! I don't set standards or sell anything.

Then you misunderstand what Harmon is training people to look for.

Tim

 

[Gregadd]

Harmon is trying to sell product. Like any business. They embrace a standard that believe will endorse that effort. In that they are no different than the subjectivists.

 

[Phelonious Ponk]

Harmon is trying to sell product. Like any business. They embrace a standard that believe will endorse that effort. In that they are no different than the subjectivists.

Well, I suppose that's true enough, though the "subjective" standard they've chosen to embrace is accuracy. It's an evasive standard but, evidently, a good one, because they not only test with trained listeners to listen for specific things, they test with untrained listeners to determine preference, and if I understand what I've read in Sean's blog, they've found that when comparing, blind, the thing that most people, trained and untrained, experienced and inexperienced seem to prefer is, well...accuracy.

Tim

 

[Gregadd]

On the other hand there is a whole industry that takes a different approach. It seems to be fairly successful. The inescapable truth is that we tend to chose the path that takes us where want to go. Those who criticize the path we've chosen usually have more disagreement with the destination we reach, rather than the direction we travel.

If I express an opinion that I can hear the effects of demagnetizing vinyl based on sighted tests you are not angry that I did not perform ABX/ DBT. You are angry about my conclusion. If I use your method and still reach the same conclusion, I must have perverted the methodology or must be some kind of freak.

 

[Phelonious Ponk]

On the other hand there is a whole industry that takes a different approach. It seems to be fairly successful. The inescapable truth is that we tend to chose the path that takes us where want to go. Those who criticize the path we've chosen usually have more disagreement with the destination we reach, rather than the direction we travel.

Fair enough, but I wasn't questioning your path, I was question your implication that blind listening tests turn up a lot of negatives because there is something wrong with the methodology. I was proposing, instead, that blind differentiation tests show a lot of negatives because of the nature of what they test. And that the lack of negatives in blind preference tests makes that pretty clear.

If I express an opinion that I can hear the effects of demagnetizing vinyl based on sighted tests you are not angry that I did not perform ABX/ DBT. You are angry about my conclusion. If I use your method and still reach the same conclusion, I must have perverted the methodology or must be some kind of freak.

I'm not angry at all, Greg. And if you agree to publicly take an ABX test to identify the demagnetization of vinyl, I'll be absolutely delighted.

Tim

 

[Gregadd]

HMM...Yet another ABX challenge.

You don't question my path? Then why my must my demag test be blind?

 

[Gregadd]

You might recall that the Furetech Demag debate was already had, complete with blind test courtesy of Gary . A difference was heard.http://www.whatsbestforum.com/showthread.php?2701-Reviewing-the-Furutec-Demag

 

[Phelonious Ponk]

HMM...Yet another ABX challenge.

You don't question my path? Then why my must my demag test be blind?

Oh I do question your path, and your logic, but that has nothing to do with the post in question regarding Harman's successful use of blind listening to test preference and what it says about the efficacy of blind testing and the number of so-called "false negatives" when testing for differentiation.

Tim

 

[Gregadd]

Oh I do question your path, and your logic, but that has nothing to do with the post in question regarding Harman's successful use of blind listening to test preference and what it says about the efficacy of blind testing and the number of so-called "false negatives" when testing for differentiation.

Tim

My point is it can and has been done sighted. Objectivists claim they don't like' the method but in reality its the results they don't care for.

 

[Phelonious Ponk]

My point is it can and has been done sighted. Objectivists claim they don't like' the method but in reality its the results they don't care for.

Your point is that what can be done sighted? I feel like we're having two different conversations together.

Are we still talking about the high number of negatives in blind listening and whether or not that indicates that they must be false?

What is it that can be done sighted?

Which result don't objectivists care for?

Tim

 

[microstrip]

I have some serious problems with Harmon training people what to look for and then designing products that excel in those categories. Anyway all my opinions are free! I don't set standards or sell anything.

Greg,
I find curious that some people can consider that Harman training people is a completely inoffensive technique of general speaker development. I went briefly through the few articles that Harman and Sean Olive kindly share in the net and could see that they succeeded in creating a well engineered and consistent system for developing goods for a defined target market. But for me it was also clear that the similarity of the results between trained and non-trained listeners was valid only in a certain and restrained environment.

 

[Gregadd]

your point is that what can be done sighted? I feel like we're having two different conversations together.

Are we still talking about the high number of negatives in blind listening and whether or not that indicates that they must be false?

What is it that can be done sighted?

Which result don't objectivists care for?

Tim

preference tests can be done sighted.

 

[Phelonious Ponk]

preference tests can be done sighted.

Well, I guess one answer to 3 questions will have to do. Of course preference tests can be done sighted. When you do them that way you just have to understand that, along with sound, you're also testing preference of the size of the components, the look of the components, the participants' impressions of the brand of the components being tested, and throwing into the mix their impressions of/beliefs regarding all the other components in the system that don't mean to test. If you want to only test preference for the sound of the component being tested, no, it cannot be done sighted. But sure, preference tests can be done sighted, and I may or may not care for the results; but I know they can't be trusted either way.

Tim

 

[Robh3606]

I have some serious problems with Harmon training people what to look for and then designing products that excel in those categories.

Hello Gregadd

Have you listened to any of their recent speakers that were developed using that methodology?? If you have not why don't you give some a try, that is if you think you can be unbiased enough to not let their methods get in the way of what you hear.

To name a few as I am a JBL head you have the Arrays, 9800, 9900 and the Everest's all of which have had done well with the reviewers and most people who hear them. You also have the Revels. There just might be something to it.

Rob

 

[microstrip]

preference tests can be done sighted.

Everything in life is full of compromises, listening tests to choose our equipment also have compromises.

I would prefer to choose my equipment not sighted, however the logistics to do it are not compatible with the way I carry them. I need a long time to choose equipment, and need to listen to many recordings to get an overview of the piece of equipment I am considering. Most of the time my opinion is formed along weeks - listening to music for pleasure, not for testing. But slowly, I am becoming acquainted with the equipment and consider that the long time and large experience that I have got with it can overcome my expectation bias.

The few blind listening tests that I tried to organize were always brief ones and the results depended more on the choice of recordings than on the equipment, as I could not cover a significant sampling of my collection. BTW, swapping a single item in a system is usually not the proper way to evaluate it - you have to swap several others to compare the optimized systems.

Considering the great probability of making a bad choice due to the intrinsic limitations of brief blind tests and the constraints imposed by the logistics of blind tests I prefer to rely on sighted tests. Also, life is too short to spend 99% of my listening time watching TV while waiting that some swaps my equipment and hides it!

 

[Gregadd]

I think the important thing is that we establish a reference more commonly referred to as the control group. Do we hear something unique? Let's take articulation. So if HP claims he finally is able to decipher a word that the could not with his reference amp. That is to say a policeman must have an "articulable suspicion." He must be able to put his hunch into words. The policeman is not held to ordinary standards. His training and experience counts. He may act on his "bad feeling." Before he intrudes on the citizen he must suspect some "illegality" that he can put into words." If I claim that A is more articulate than B then I must be able to compare that source material with the standard or control. Can you decipher that word. Then play B and say, can you decipher that word? If it's obvious then fine. If not then go "blind."

Asking some citizen or audiophile to make a comparison based on equipment he has never heard with source material he never heard just does not cut it. The listener must clearly identify and articulate difference. So get that record out where you can hear the singer catch her breath on the new amp and demonstrate it was inaudible on the old one. That's what the best reviewers do. Then let the engineers figure out why.

 

[Gregadd]

Harry Pearson on the Maggie 3.7:

I loaded up the Mercury CD of The Composer and His Orchestra and we listened through the introductory cut, where Hanson introduces the instruments and instrumental sections of the Eastman-Rochester orchestra. Hanson is placed dead center, in the empty hall, while the sections in the orchestra, recorded separately, were, placed as they would be in concert, arrayed around and behind the podium. The ensembles and individual instrumentalists were recorded with almost no compression, so the scaling in space and dynamics was close to what you might hear in person, thus justifying the term “living presence.”

I immediately heard the acoustic behind Hanson, which I had not before. It was as if I could hear the distance to the back wall. And the space of that distance. It was as if the speakers had retrieved a third-dimensional space behind the conductor himself. Thus the 3.7s were delineating a virtual sonic portrait of the hall acoustic itself. All Magnepan speakers are dipolar in operation (by design), but no dipolar has captured this space in the same way before.

It is in fact difficult to suggest that description is the result of "bias." It is in fact "repeatable."

 

[Gregadd]

Jeff Fritz on the Q3

Tonally, the Q3s sounded dead neutral, as I would expect from seeing their pancake-flat frequency-response graphs -- there’s little of interest to report in that respect. But just because the signal was flat doesn’t mean the music sounded flat. In the Music Vault, the Q3s’ soundstaging and imaging set new standards of three-dimensionality in real space. Several years ago, YG Acoustics’ original Kipod, though flawed in other areas, set a supreme standard for pinpoint imaging in my room that has stood unchallenged -- until now. The Q3s were even more precise. The instrumental separation from good recordings bordered on sonic holography. This precision made "The Girl from Ipanema," from Stan Getz and João Gilberto’s Getz/Gilberto (24/96 AIFF, Verve/HDtracks.com), sound staggering -- it was as if I were hearing everything laid down in those March 1963 recording sessions for the first time, even though I’ve heard this track hundreds of times. I could tell -- to the inch -- just how far to the right singer Astrud Gilberto was standing from Getz’s tenor sax. Her form was right there, just to the outside of the right speaker. This wasn’t the first time I’d heard images that spanned my room from wall to wall, but never has beyond-the-speakers imaging been rendered as absolutely precisely as between-the-speakers imaging. And I’d never heard a recording of a tenor sax sound so natural. Q3

Again, articulate and repeatable.