Do you really trust your ears?

[Gregadd]

... You are, however, right as rain with the scientific and statistical community which claims, and offers good evidence that differences are more accurately identified when the A/B switching is fast...

Any slight of hand requires speed (smile).

I for one would never form an impression on any equipment without listening for a significant period with a wide variety of music.
Listening fatique.
Choice of music.
My own mood.
Even weather has an effect.
Is that really what a piano sounds like?

How we learn is through long term exposure. That is the brain prioritizes the things we do the most.

The effects of say harsh high frequencies, boomy bass, or noise have a cumulative effect not present from short tests. Even a good system could contribute to fatiqe in an A/B test.

 

[Gregadd]

There is also plenty of data which indicates that what the brain interprets is pretty consistent from brain to brain, and that, in fact, the brain compensates for minor hearing deficiencies to bring us closer to the same "truth," not further apart. So while I agree that the ears are the final arbiter, they are only the arbiter of what I like; no more, no less.

Tim

If it only reveals what "I" like then no listening test would have any validity. Surely even a blind test advocate would not argue that. Double blind test does not attempt to eliminate everything but the "ear". They only attempt to eliminate bias.

 

[Phelonious Ponk]

If it only reveals what "I" like then no listening test would have any validity. Surely even a blind test advocate would not argue that. Double blind test does not attempt to eliminate everything but the "ear". They only attempt to eliminate bias.

Sorry, I wasn't clear. My ears are only an arbiter of what I like. A bunch of ears in a well-controlled study could surely give a manufacturer insight into what people in general find to be more pleasant, more natural, choose your adjective. But it still has to be blind, whether it is AB/X testing for differences or A/B/C, etc testing for preference. And while I agree that it takes more than quick switching between options to determine if a system is going to cause listening fatigue, I think the way most audiophiles do that -- sighted, without a reference in the system to compare with -- is self-defeating unless you're immune to bias. I know I'm not, because I've tried this method and, at least once, when I put the original component back in after a couple of weeks of believing the shiny new one was making a very audible positive difference, that difference immediately shifted into the more typical "I think I might hear..." category. Blind? I couldn't identify which was which.

YMMV.

Tim

 

[JackD201]

Blind Tests are listening tests. There's no getting around that.

 

[rblnr]

sighted, without a reference in the system to compare with -- is self-defeating unless you're immune to bias.

I think a key to switching tests is to have a, b, (or c) be your reference. Having that 'backbone' is useful.

And delusional as some might say it is, I actually believe I'm immune enough to bias to take it out of the equation. (I could elaborate on the psychological reasons why I think this is so, but I'll spare forum members) Part of it is not feeling the need to insist on differences when I don't hear any.

 

[Phelonious Ponk]

I think a key to switching tests is to have a, b, (or c) be your reference. Having that 'backbone' is useful.

And delusional as some might say it is, I actually believe I'm immune enough to bias to take it out of the equation. (I could elaborate on the psychological reasons why I think this is so, but I'll spare forum members) Part of it is not feeling the need to insist on differences when I don't hear any.

I'm immune enough when it's somebody else's research, decision and investment. My own? Probably not so much.

Tim

 

[caesar]

When writing a check that large, it had better do more than satisfy me intellectually. I mean it has got to induce some seriously involuntary reactions, like goosebumps, alter my respiratory and heart rate, dilate my pupils.

I'm serious.

My ears, I do not trust that much. My goosebumps and vital signs, I trust much more.

Right on!!!!!!!!!!!!!!!!!

 

[caesar]

And Michael Fremer would be correct. Jonathan Valin is a joke as an audio reviewer.

I think Valin's problem is that he doesn't listen to a broad enough range of music before calling something THE BEST. It's kind of like someone saying that the 2 seat Porsche Boxster is the best car there is because it's plenty fast and it handles better than pretty much any car out there (due to the mid-body engine configuration). Of course if you are transporting a family, you run into problems. As a reader, you better know what he likes before writing the big check...but most people just see "THE BEST" and write the check.

Another problem with Valin is that he is the best marketing rep a company like Magico or ARC can have. Or any audio company can have, for that matter. No wonder he gets any gear he wants from them, for as long as he wants it, at a snap of a finger.

 

[ack]

Or perhaps Valin's problem is that he's most likely a full time employee at TAS with vested interests...

 

[caesar]

Over time, I have accepted that I hear differently from time to time and that my hearing is significantly influenced by my moods.

To me, a piece of equipment not only has to sound right but should also feel right. Still this is no guarantee that the equipment will always sound right and/or desirable through time. For me, there always comes a time when the equipment will be given a rest due to a change in my preference.

The dealer is definitely a very important part of the equation for me. I want to buy from someone I trust. I give the dealer more weight than the brand and only consider the brands that my trusted dealer carries.

I do not validate what I hear with publications or other audiophiles. I do not ask people who visit me for feedback. To me, it is mine and I am the only person it has to satisfy. If my visitor enjoyed the sound, then I am glad they did. If they did not, then I feel sorry for them that they have wasted their time. Come to think of it, friends have visit me to hear how a piece of equipment sounds like or how a chain of equipment matches. At the end of their visit, they usually describe objectively what they heard but hardly ever tells me their subjective verdict.

Mullard88,

I think you make great points. Most of us do not have dealers that carry stuff we like. In many cases, one has to travel far to a dealer and only briefly listen to gear that we are not familiar with, and in an unfamiliar room and system. And after a long day of work and travel (plane, trane, auto on an expressway), as well as establishing rapport with a dealer, there are certainly elements that would question my hearing, my mood, and my judgment.

 

[garylkoh]

There's a reviewer who has been coming to the Genesis demo room at CES/RMAF/etc. for years. He's never written anything about his visits. Last time he was in the room, he asked for the room to be cleared out. I refused because there were paying customers. But I could see that the people in the room were impressed that he visited. He put on the most horribly esoteric modern classical recording I have ever heard. It wasn't bad music, but to my ears quite unapproachable. And he did manage to clear out the room. It was LOUD, strident, had huge tympanis.

At the end of the piece, his only comment was "Hmmmmm...... it didn't break...." and he walked out. I think that he was trying to break the speakers, since the opening was quiet, and he kept asking for the volume to be increased.

One of the many reasons why we have so few reviews - they think that I don't give them the respect they deserve. That's why I only sell to dealers and customers who trust their own ears.

 

[caesar]

Or perhaps Valin's problem is that he's most likely a full time employee at TAS with vested interests...

Reminds me of a takeaway from a Rhetoric class professor a long , long time ago - "forget the substance of the point being made. The success of the argument is relative to one's ability to smoothly communicate." And there's a reason why Valin gets more covers than anyone, while living with the gear of many folks' dreams

 

[caesar]

There's a reviewer who has been coming to the Genesis demo room at CES/RMAF/etc. for years. He's never written anything about his visits. Last time he was in the room, he asked for the room to be cleared out. I refused because there were paying customers. But I could see that the people in the room were impressed that he visited. He put on the most horribly esoteric modern classical recording I have ever heard. It wasn't bad music, but to my ears quite unapproachable. And he did manage to clear out the room. It was LOUD, strident, had huge tympanis.

At the end of the piece, his only comment was "Hmmmmm...... it didn't break...." and he walked out. I think that he was trying to break the speakers, since the opening was quiet, and he kept asking for the volume to be increased.

I think that's one of the tricks Valin uses to beat on YG, Magico's other upstart competitor. And he calls out yg on this in his show reports.

 

[caesar]

Personally I trust my ears completely. If I hear something I don't like, well that's all it takes to disqualify that piece for me. Reviews are nice to read, particularly if the reviewer is in agreement with what I hear, but if that is not the case, then my ears take precedence. Which is why I will not buy any piece of gear unless I have heard what it can do for myself. On a A'gon thread a while back, I was taken to task for liking my SF GH's far better than the newer GM's. Many of the flamers were GM owners who never heard the GH's. As time has gone by, the a'phile community has began to accept the fact that the GH's are superior to the GM's. I knew this from the start by listening to both speakers and hearing for myself the differences. The problem that I think is interesting is that people many times do not trust their ears and are sold on something based on price and age/status. The prevailing thinking being the higher the price and the more recent the piece the better. Sometimes true, but in my experience many times not.

I think it is a really fine line between listening to someone who likes what you like or reading someone who is your "clone", and reading someone's opinion who does not think like you at all. The latter can really point out gaping holes in your argument, or just waste your time. Overall, I think it's worth it if someone challenges your assumptions and your perspective.

 

[caesar]

BTW - was the Magico Q5 review ever published in TAS?

I haven't seen it. He is trying to build up the suspense to his conclusion.

 

[caesar]

Bingo!

i do trust my ears. however; i don't trust them in a short audition....particularly in a group. yes, i'll have an opinion about what i'm hearing. but i always defer to how i percieve a piece of gear sounds in my system over a few days or a week ideally. i have my personal routine that i follow for listening tests which include recordings i have used over the years. i need to make sure i'm relaxed and under no time constraints and i need to make sure i'm feeling good and enjoying my listening.

if i find i'm stressing over trying to figure something out; i'll start reading a book or go do something else and come back fresh to the question. (which is why i put no validity into blind testing). i need to have three or four sessions with similar conclusions to form strong opinions.

the process of gear upgrades should be fun and a process of discovery, not a chore. also; any audition should include time with casual listening where it's just the music and not the sound. no agenda.

bottom line. i'm very happy with the result of my system synergy with my approach. my system does what i have always wanted it to be able to do.

Great post!

 

[caesar]

This is, of course, at odds with the audiophile conventional wisdom that says subtle qualities only reveal themselves through extended listening, and claims that quick-switching between components being tested is the foundation of the invalidity of AB/X testing. You are, however, right as rain with the scientific and statistical community which claims, and offers good evidence that differences are more accurately identified when the A/B switching is fast, and that listening over time only allows the time to muddy the waters with perceptual bias. They say it is even worse if you're judging preference instead of simply listening for differences, as humans are notorious for preferring the familiar. My experience is consistent with yours. I've had new components in on trial, given them a couple of weeks of good listening, completely convinced that I heard a difference, only to put my reference back in at the end of the trial to A/B, and been incapable of telling which was which. Trust. But verify.

Tim

Phelonius, I know we can start a whole new thread on judging experiences. But let's say you are lying on a beach of the Arkansas river, and it is sublime for a moment - and then quickly switch to a moment on the beach at the French Riviera. Those specific experiences may be comparable. But I would argue that if you look at your Arkansas vacation vs. your European vacation in whole, you may come to a different conclusion. I will start a thread on this in the sometime future when I catch up on my other reading here.

 

[caesar]

What we are talking about is hoiw the brain interpets what the ears hear. That subjects itself to infinite varaition.

So what I am after here is what else contributes to that variation. A smooth talking reviewer? A dealer who has been in business forever? Being a trend setter? Something else?

 

[tonmeister2008]

Any slight of hand requires speed (smile).

I for one would never form an impression on any equipment without listening for a significant period with a wide variety of music.
Listening fatique.
Choice of music.
My own mood.
Even weather has an effect.
Is that really what a piano sounds like?

How we learn is through long term exposure. That is the brain prioritizes the things we do the most.

The effects of say harsh high frequencies, boomy bass, or noise have a cumulative effect not present from short tests. Even a good system could contribute to fatiqe in an A/B test.

Long-term learning may certainly be a factor that influences your preference in the case of loudspeakers, for example. But I would argue it is less of a factor for a trained and experienced listener.

I'm not so sure long term exposure matters with AB/X tests where the audible differences are near or below absolute threshold, and all of the nuisance variables (e.g. listener training, program material, playback environment) are well controlled and sensitive measuring to the variable under test. Fatigue is only one of the nuisance variables, and it can be controlled as well. You can stop the test if the subject is fatigued, and let them resume the test when they feel rested. The research I've seen indicates that for loudspeaker evaluations using trained listeners, fatigue is not a factor until you get beyond 30-40 minutes.

The problem is that so few listening evaluations are properly controlled, so there is justification to doubt the meaning and validity of the results and conclusions.

 

[Gregadd]

There is a difference from just being tired of listening to music and system induced fatigue. While the former could be controlled the latter should not be. The latter should be a factor in the invaluation.

That's the problem with scientifc testing and real world evaluation. In science variables need to be controlled. In the real world everything is in play.

I am expereinced but not trained.