Logical assumptions - Predetermined biases

[rce]

Is there 'anyone out there' tired/fatigued/bored with reviewers learned/cultural - ingrained fixed mindset?
Well, I for one am...
Reviewer speak: Cable/dac/cd etc. exhibit A 'measured' with a slightly (inaudible?) lower noise floor than exhibit B. 'Exhibit A was able to unravel a level of detail that exhibit B just could not match, owing of course to the lower noise floor of exhibit A'. Just one simple example of dozens of well worn/tiresome clichés/learned responces.
Now, did Mr reviewer really here a difference? Or was he/she simply following a logical assumption/predetermined bias? Was he/she a victim of one of the oldest human nature traits in the book - 'I desperately want to fit in here'. Nothing wrong with that! After all, we above all else are social creatures with a hunger/need for community. That's why you & I are together as members on this forum - and that's a good thing!
But, does this natural 'hunger' give us an insight to the not so good side of the human condition - 'I just wanna/must fit in'.
This forum is not about philosophy or the human condition - we are here because we love music! However, weather we like it or not - our humanness is closely akin with our passions. SS/tube - digital/vinyl debate anyone?
The question I pose is this: Ought we trust Mr reviewer in light of what we've just skimmed on? (of course human community isn't the only hurdle, there will always be commercial pressures being bought to bare, personnel biases/preferences etc).
The statement I make is this: We must learn to trust ourselves - our ears/our preferences/our budgets/our priorities/our listening space etc.
Listen with your ears not ya wallet

 

[microstrip]

(...) Listen with your ears not ya wallet

Unhappily if we do it probably our wallets would suffer a lot ...

 

[Shaffer]

Is there 'anyone out there' tired/fatigued/bored with reviewers learned/cultural - ingrained fixed mindset?
Well, I for one am...
Reviewer speak: Cable/dac/cd etc. exhibit A 'measured' with a slightly (inaudible?) lower noise floor than exhibit B. 'Exhibit A was able to unravel a level of detail that exhibit B just could not match, owing of course to the lower noise floor of exhibit A'. Just one simple example of dozens of well worn/tiresome clichés/learned responces.
Now, did Mr reviewer really here a difference? Or was he/she simply following a logical assumption/predetermined bias? Was he/she a victim of one of the oldest human nature traits in the book - 'I desperately want to fit in here'. Nothing wrong with that! After all, we above all else are social creatures with a hunger/need for community. That's why you & I are together as members on this forum - and that's a good thing!

I sold High-End in the late-90s. That was the time, I'm sure many will recall, when Sterophile had a DAC-a-month review schedule. Each DAC sounded distinctly different, in print, some going as far as redefining the genre. Being in a position where I could take these products home for a run in my own system, I took advantage of the option at every opportunity. Did I hear these gross differences? Absolutely not. Not even close. Did any of the DACs redefine performance, as per the reviews? Not at all. Have I ever purchased a DAC with my own money. No, not even with accommodation pricing.

But, does this natural 'hunger' give us an insight to the not so good side of the human condition - 'I just wanna/must fit in'.

Sad to say, this approach is common in most fields. Few are willing to put themselves, and their credibility, on the line.

 

[GaryProtein]

Is there 'anyone out there' tired/fatigued/bored with reviewers learned/cultural - ingrained fixed mindset?
Well, I for one am...

Reviewer speak: . . . . . blah, blah, blah . . . .

Listen with your ears not ya wallet

Don't get me started on this. I agree with you. I'm in love with blind tests.

There should be no expectations when listening to or reviewing equipment. Just sit back, relax and take it all in.

You're preaching to the choir here.

 

[A.wayne]

Meh .....

 

[Gregadd]

The purpose of magazines is to promote the industry.

 

[microstrip]

Don't get me started on this. I agree with you. I'm in love with blind tests. (...)

Gary,
My main problem with blind tests is implementation, not philosophy. Can you tell us how you manage to carry blind tests in your system?

 

[GaryProtein]

It isn't very difficult, it's just cumbersome and time consuming.

You need a second person to change whatever you are comparing with your control component. They have to change or not change what you are supposedly testing and then listen. The person changing the components gets to tell you they made the switch when they are ready, and they can lie to you about what you are listening to. It's up to the listener to detect any change.

If you can correctly tell a difference, that's great.
If you can't tell a difference, you can't tell the difference. It's that simple.

 

[microstrip]

It isn't very difficult, it's just cumbersome and time consuming.

You need a second person to change whatever you are comparing with your control component. They have to change or not change what you are supposedly testing and then listen. The person changing the components gets to tell you they made the switch when they are ready, and they can lie to you about what you are listening to. It's up to the listener to detect any change.

If you can correctly tell a difference, that's great.
If you can't tell a difference, you can't tell the difference. It's that simple.

Unless you have a permanent audio engineer slave and just want to carry it with a single component IMHO it is not possible to do it practice. I doubt that you will ever get a statistically valid result. And again IMHO systems must be tested as an whole. Most probably changing individually components will not result in easily perceptible differences, but a particular combination of them will be easily noticed and will result in improvement.

Just for my statistics can you tell us an estimation of the mean number of hours you spend per year in blind tests? Please do not solve it as a Fermi problem!

BTW, some people will consider that since your manipulator speaks and can lie, the test becomes immediately invalid - it is not an audio test anymore, it is a game between you and him!

 

[rockitman]

Unless you have a permanent audio engineer slave and just want to carry it with a single component IMHO it is not possible to do it practice. I doubt that you will ever get a statistically valid result. And again IMHO systems must be tested as an whole.

Whole is greater than the sum of its parts

 

[GaryProtein]

Unless you have a permanent audio engineer slave and just want to carry it with a single component IMHO it is not possible to do it practice. I doubt that you will ever get a statistically valid result. And again IMHO systems must be tested as an whole. Most probably changing individually components will not result in easily perceptible differences, but a particular combination of them will be easily noticed and will result in improvement.

Just for my statistics can you tell us an estimation of the mean number of hours you spend per year in blind tests? Please do not solve it as a Fermi problem!

BTW, some people will consider that since your manipulator speaks and can lie, the test becomes immediately invalid - it is not an audio test anymore, it is a game between you and him!

1---You should absolutely test one component at a time. That's what a reviewer is doing when he tests an amp, preamp or disk player for a review. It is NOT necessary to test an entire system as a whole unless you want to see if Mr. X's system is better than Mr. Y's system.

2---I am not in the audio business so I don't spend time testing. I spend time listening. I only do testing when I am in the market for something, and I have been very happy with my system, so I haven't been in the market lately.

3---The person setting up your test should have the option to switch or not switch what you are listening to when he asks for an evaluation. That's what makes the test NOT a game or fraud. Not knowing what you're listening to is what makes the test and review HONEST.

Finally, YMMV and I have donned my flame-proof suit.

 

[JackD201]

CAn anyone give an example of a bias that isn't predetermined? I can't think of any at the moment.

 

[microstrip]

1---You should absolutely test one component at a time. That's what a reviewer is doing when he tests an amp, preamp or disk player for a review. It is NOT necessary to test an entire system as a whole unless you want to see if Mr. X's system is better than Mr. Y's system.

It is here we politely disagree. When you optimize a system using a component as we always try and just insert a new one for reviewing, keeping all others equal you are not creating a fair environment to review the new component. You are just evaluating if by chance it matches your system better than the existing one. IMHO in order to write a review you must do it in an adequate system. YMMV.

 

[FrantzM]

So there is no way to test a component? only a system?ooooooookay ... I think I got your point ... Synergy and al that ? Right?

 

[microstrip]

So there is no way to test a component? only a system?ooooooookay ... I think I got your point ... Synergy and al that ? Right?

Big YES!

Long time ago I read in an Hong Kong magazine how they tested components: they had about one dozen reviewers and they handled one of them the piece. If he was not pleased with its performance they would handle it to the next reviewer in the line. If he still was not happy the sequence would go on until they get a reviewer who liked the equipment and wrote the respective the review. Otherwise, if no one liked it, it was politely send back to the manufacturer with a letter stating something like "We are sorry to have to decline reviewing piece X. We thank you for your kindness to submit it, but none of our reviewers was able to provide an adequate system to test your equipment, and get a good and enjoyable performance. We apologize for not being able to find an appropriate system to carry this review, we are sure that your equipment has many good qualities, but we were not able to find them" .

 

[treitz3]

"We are sorry to have to decline reviewing piece X. We thank you for your kindness to submit it, but none of our reviewers was able to provide an adequate system to test your equipment, and get a good and enjoyable performance. We apologize for not being able to find an appropriate system to carry this review, we are sure that your equipment has many good qualities, but we were not able to find them" .

Ouch! Can you imagine what was going through the manufacturers minds when they got that letter? That's gotta sting.

Tom

 

[rce]

Unhappily if we do it probably our wallets would suffer a lot ...

agreed...though there is relative 'value' to be had even in the hi-end

 

[rce]

I sold High-End in the late-90s. That was the time, I'm sure many will recall, when Sterophile had a DAC-a-month review schedule. Each DAC sounded distinctly different, in print, some going as far as redefining the genre. Being in a position where I could take these products home for a run in my own system, I took advantage of the option at every opportunity. Did I hear these gross differences? Absolutely not. Not even close. Did any of the DACs redefine performance, as per the reviews? Not at all. Have I ever purchased a DAC with my own money. No, not even with accommodation pricing.



Sad to say, this approach is common in most fields. Few are willing to put themselves, and their credibility, on the line.

Sad, that this wonderful hobby/passion has been so corrupted for so long. But thankfully, good performance at real world pricing is gaining momentum...long may it continue

 

[Shaffer]

Sad, that this wonderful hobby/passion has been so corrupted for so long. But thankfully, good performance at real world pricing is gaining momentum...long may it continue

I think it ultimately comes down to what one values on an individual level. Using myself as an example, to my ears Redbook sucks to a degree where upgrades make little sense. Someone else, however, may value Redbook differently and the small changes brought about by a DAC may indeed serve as a worthwhile improvement. Same for cost. I may see a, say, $5000 DAC as a ridiculous expenditure - and it would be for my system and my preference structure - but the same can become a rational investment for others.

My problem with reviews, and the market as a whole, stems from a lack of differentiation relative to orders of magnitude. A change in speakers exists on one order of magnitude, while a change in DACs exists on an entirely different sonic plateau. Yet, the sonic goodness, if you will, brought about by the upgrade is seemly given a parity value.

 

[edorr]

I sold High-End in the late-90s. That was the time, I'm sure many will recall, when Sterophile had a DAC-a-month review schedule. Each DAC sounded distinctly different, in print, some going as far as redefining the genre. Being in a position where I could take these products home for a run in my own system, I took advantage of the option at every opportunity. Did I hear these gross differences? Absolutely not. Not even close. Did any of the DACs redefine performance, as per the reviews? Not at all. Have I ever purchased a DAC with my own money. No, not even with accommodation pricing.

I beg to differ. I have bought mutiple DACs that transformed my system, starting with hooking up my old Sony CD player to an audio alchemy DAC in the 90s and culminating in buying a 5 figure EMM Labs DAC2X very recently. To my ears, they have provided huge bang for the buck upgrades, in many cases comparable with speakers.