Blind Listening Comparison

[Al M.]

Do you want to share the details and raw results of such test? Was it a double blind test?

No, it was not double blind. I don't see how that would have made any difference, since my friend didn't let on even a little bit which was which. Again, to the point where I was seriously sweating it out.

***

In any case, also in sighted tests I have been able to trust myself. And yes, also when it came to experiencing no differences, or too small ones to bother -- I don't chase each "improvement". I am pretty confident that I have spent my money wisely on my system over the years, buying expensive components where it was warranted, and sticking to cheaper ones where that was the case as well.

 

[NC Lee]

What happened to NC Lee's test? I was intrigued!

I am having room treatment installed in a couple of weeks so am going to do that first.

 

[COF]

Allow me to reiterate. "Those who seek the chaperone of science need some science." You don't get to parade your anecdotal pseudo scientific evidence as fact.

This is a strawman. Nowhere did I parade "pseudo scientific evidence as fact."

I detailed my own experience and the outcome of blind testing some cables. And I was explicit with caveats, that what I was reporting was: "obviously not rigorous science,"

And I did not claim any universal negative had been demonstrated by my own meager tests.

So I was simply sharing my experience, explaining how I take them as my own data points in how I look at various claims, just as everyone else does here.

I doubt you leap on someone doing exactly that if instead they - like countless do here - reported hearing sonic differences.

Edit: Let us settle this. I am willing to accept the fact that you can't hear a difference as you claim. Are you willing to accept my claim that i can?

Differences between what in particular?

Between AC cables?

Likely not merely on your say-so.

And I think I have good reasons for this. I generally work on a sort of Extraordinary Claims Require Extraordinary Evidence heuristic. You tell me you just bought a 4K TV at Best Buy? I have no problem accepting that on your say-so. It's entirely plausible, people do it every day. You tell my you just bought a Perpetual Motion Machine at Best Buy? I'm going to want more than your say so or your receipt. ;-)

Likewise with audio products: If you tell me you auditioned two speakers and you described their sonic differences, I'd be happy to accept your report. Different speaker designs are widely acknowledged to sound different, with plenty of technical theory/practice and even scientifically-controlled studies to back that up. It's not technically controversial among people expert in the field.

But if you tell me you auditioned an expensive HDMI or Ethernet or AC cable against properly working cheap alternatives and heard an obvious sonic difference, I would prefer to wait for stronger evidence than an audiophile claiming he heard it. This is because I'm aware of the controversies over such claims. Insofar as I understand the technology involved, and having read lots of commentary on this issue by technically knowledgeable people who AREN'T in the business of trying to sell me those things, I find the arguments compelling for why these claims are dubious.

Another reason I'm cautious about such claims: AC cables, for instance, are often ascribed sonic qualities as if they were part of the cable itself "This cable has a wonderful combination of bass depth, smooth midrange and high end sparkle"...that kind of stuff. As if cables had some inherent sonic quality irrespective of the system you put them in - they just impart those sonic qualities somehow. I've yet to see a technical explanation that could make sense of that, let alone such claims with objective technical evidence, or passing blind tests controlling for bias. Or, for that matter, why two working AC cables should sound different in the first place.

And it is a red flag to me that the companies selling these types of products share a typical modus operandi:

Usually say they've identified some TECHNICAL problem that their technology "solves" to produce better sonics, yet they almost never produce the objective evidence - e.g. measurements - showing the technical problem and showing how their technology produced a change. Instead, the claims are made, and then it's just booted to marketing, and left to the subjectivity of audiophiles and reviewers to vet the claims. And of course, given our human bias, it is virtually guaranteed that some will "hear a difference," thus essentially "validating" the claim for the company. I can't ignore that this is essentially how virtually every pseudo-scientific, new age, alternative and snake oil product manages to get sold. It doesn't MEAN that every such product IS making bogus claims. But unless they are doing something to more rigorously distinguish their claims from the type I can find from pseudo-science products, I haven't a good reason to just presume they are any better. And, given how easily we succumb to sighted bias, I don't find that yet more sighted anecdotal reports is sufficient to confirm the claims. "I totally experienced a difference!" is again, available for literally every dubious claim anyone has ever confected.

So I believe I have some justified skepticism. I'd be happy to believe ANY of these things change the sound. I like tweaking my system as many audiophiles do. I just approach certain claims with more caution.

Back to your question: If you are claiming to hear differences between, say, AC cables, no I wouldn't just accept your claim. I have given my reasons for wanting stronger evidence.

But if you have that better evidence - e.g. measurements USING that audio equipment, showing a difference between the signal using a standard AC cable and your high end cable of choice...AND/OR...you can detail how you passed blind tests to identify between such cables...then I'm all ears.

I'm trying to keep my critical thinking cap on, as a consumer.

So you find any of that unreasonable?

 

[COF]

I am having room treatment installed in a couple of weeks so am going to do that first.

Cool. Thanks for getting back to us!

My view on blind testing: it's a tool that no audiophile has to use or bother with, but it's there IF someone is interested.

My own interest waxes and wanes. I found it helpful when I really wanted to have more confidence in some listening tests, and also sometimes just out of curiosity. I think that the type of blind listening tests we average audiophiles are likely to engage in are rarely going to meet scientific standards. But, if we do a good enough job with the protocol, I think the results can be "accepted provisionally" in terms of helping form our own views. No reported blind test result has to be accepted by another audiophile reading it on the internet. They weren't there, didn't supervise, something always could have been wrong with the test. On the other hand, if someone reads how a test was done and it seems to meet muster, it can be accepted as another point to consider in forming their viewpoint.

As someone familiar with the justifications for the scientific method, I can't pretend high end audio exists in some epistemic bubble, UN-influenced by the variables of human bias and where any claim can be verified as true purely by subjective experience, as if we have infallible ears and perception. But on the other hand I don't think ANY audiophile MUST care a single thing about science, or rigor, or blind tests, or measurements or any of that. We are all free to test as we want, buy what we want. If someone wants to only ever use the "Do I Hear It?" test...that's their prerogative.

But, also, folks who DON'T care about placing higher emphasis on objective evidence shouldn't be too quick in disparaging those who do. Having a skeptical opinion about a product isn't being a muckraker. It doesn't send jack booted thugs to knock down the doors of people "being insufficiently rigorous in their buying habits." It's just being an audiophile with another viewpoint. We can all have our own approaches, there's room for plenty of opinions and approaches.

 

[Gregadd]

Luckily what you said is written down for anyone to see. What you are saying is you did not "seek the chaperone of science.' Great. That makes your claims completely anecdotal and have no application to anyone except you. That is my point. That makes my sighted test as valid as your blind test. Both are mere individual anecdotal accounts.

 

[COF]

Luckily what you said is written down for anyone to see.

Agreed, given your proclivity to strawman my position.

What you are saying is you did not "seek the chaperone of science.' Great.

So...then why did you strawman what I claimed?

That makes your claims completely anecdotal and have no application to anyone except you.

Yes.

And...no.

Yes my claims are anecdotes. But not all anecdotes are equal in terms of plausibility or, in pragmatic terms, evidential quality. See my comments to NC Lee. And how I elaborated to you on the hueristic of "Extraordinary Claims Require Extraordinary Evidence." We accept anecdotes as evidential all the time. It's pragmatic. But that doesn't mean all anecdotes are equally plausible, or equally justified.

That is my point. That makes my sighted test as valid as your blind test. Both are mere individual anecdotal accounts.

Not necessarily. You can't just leap to the conclusion that your sighted listening is "just as valid" as someone else's blind test.

Most of the information we receive from science is in the context that we weren't there to vet the experiments. So what we do is try to have an understanding of the justifications a scientist is using for his method. If it makes sense, then that's a good start. Then we can look at the specific experiment to see if it seems they have followed that method. If we can't find any obvious flaws, and especially (though not necessarily) if it fits coherently within with a wider body of well established knowledge, we can provisionally accept the results. We can't all go around doing science on everything we believe.

Likewise, one can look at two different claims, like two different audiophiles testing out AC cables.

We can seek to understand if the claim we are looking at is a technically controversial one or not. If it is, that gives initial grounds for skepticism, or at least a requirement for carefully derived evidence, to start off with. And it would make sense to see who is being more rigorous in their approach to vetting the claim. And since we know there is the pronounced variable of sighted biases, it makes sense to see which audiophile is taking that most seriously in his method.

If audiophile A is using purely sighted listening, then we already know that method is vulnerable to misinterpretation and perceptual errors, especially when investigating what is already a technically suspicious claim.

If audiophile B is rightly cautious about the claim to begin with, and is employing methods to control for sighted bias (or is employing objective methods, e.g. measurements), then that audiophile method is already making more sense.

Then you can look at the specific method from audiophile B to see if it made some error LIKELY to invalidate the results.
If you can't spot such an error, then...just like accepting what seems to be good science...it at least puts audiophile B's claims as more rigorous and plausible. You don't have to accept it as providing the same level of justification as carefully controlled multiple studies. And, especially if the results actually were surprising in light of generally accepted theory, you'd want to run more tests. BUT...to say that both audiophiles have produced reports of EQUAL reliability or worth would be an unjustified conclusion. One audiophile didn't even get off the ground in terms of proper methodology.

So...yes we would both be producing anecdotal accounts.

But, no, that doesn't mean your methodology is equally valid. And it doesn't mean one report isn't on stronger footing than another, for that reason.

 

[microstrip]

No, it was not double blind. I don't see how that would have made any difference, since my friend didn't let on even a little bit which was which. Again, to the point where I was seriously sweating it out.

Well, we have different opinions on the importance of being double blind and the importance of being in presence of the person doing the switching and knowing the which was which. But the important part of my question were the details, methodology and raw data ...

***

In any case, also in sighted tests I have been able to trust myself. And yes, also when it came to experiencing no differences, or too small ones to bother -- I don't chase each "improvement". I am pretty confident that I have spent my money wisely on my system over the years, buying expensive components where it was warranted, and sticking to cheaper ones where that was the case as well.

We all are pretty sure on ourselves and are able to trust ourselves - it is part of the hobby, However IMO blind tests should not be a part time coffee talk - if we accept and trust them for small differences (differences that do not result in measured differences according to the usual methods and parameters) we should be able to successfully use them to all our decisions and be able to pass such "challenges" with success.

As I said before, still waiting to read about a proper audiophile blind test with current high end equipment.

 

[Al M.]

We all are pretty sure on ourselves and are able to trust ourselves - it is part of the hobby, However IMO blind tests should not be a part time coffee talk - if we accept and trust them for small differences (differences that do not result in measured differences according to the usual methods and parameters) we should be able to successfully use them to all our decisions and be able to pass such "challenges" with success.

Perhaps. Yet as I suggested in my first post on the thread, the stress factor in blind tests, possibly impairing judgment, should not be underestimated.

As a scientist I am highly skeptical when people want to sell something that by necessity involves fickle human psychology as "objective" or even "science".

There is nothing "scientific" about blind tests, even though of course when you pass one it can be viewed as extra confirmation -- it certainly can be viewed as vindication! But a psychological feelgood emotion no science makes...and it does not per se discredit sighted tests.

 

[Gregadd]

Agreed, given your proclivity to strawman my position.



So...then why did you strawman what I claimed?



Yes.

And...no.

Yes my claims are anecdotes. But not all anecdotes are equal in terms of plausibility or, in pragmatic terms, evidential quality. See my comments to NC Lee. And how I elaborated to you on the hueristic of "Extraordinary Claims Require Extraordinary Evidence." We accept anecdotes as evidential all the time. It's pragmatic. But that doesn't mean all anecdotes are equally plausible, or equally justified.



Not necessarily. You can't just leap to the conclusion that your sighted listening is "just as valid" as someone else's blind test.

Most of the information we receive from science is in the context that we weren't there to? vet the experiments. So what we do is try to have an understanding of the justifications a scientist is using for his method. If it makes sense, then that's a good start. Then we can look at the specific experiment to see if it seems they have followed that method. If we can't find any obvious flaws, and especially (though not necessarily) if it fits coherently within with a wider body of well established knowledge, we can provisionally accept the results. We can't all go around doing science on everything we believe.

Likewise, one can look at two different claims, like two different audiophiles testing out AC cables.

We can seek to understand if the claim we are looking at is a technically controversial one or not. If it is, that gives initial grounds for skepticism, or at least a requirement for carefully derived evidence, to start off with. And it would make sense to see who is being more rigorous in their approach to vetting the claim. And since we know there is the pronounced variable of sighted biases, it makes sense to see which audiophile is taking that most seriously in his method.

If audiophile A is using purely sighted listening, then we already know that method is vulnerable to misinterpretation and perceptual errors, especially when investigating what is already a technically suspicious claim.

If audiophile B is rightly cautious about the claim to begin with, and is employing methods to control for sighted bias (or is employing objective methods, e.g. measurements), then that audiophile method is already making more sense.

Then you can look at the specific method from audiophile B to see if it made some error LIKELY to invalidate the results.
If you can't spot such an error, then...just like accepting what seems to be good science...it at least puts audiophile B's claims as more rigorous and plausible. You don't have to accept it as providing the same level of justification as carefully controlled multiple studies. And, especially if the results actually were surprising in light of generally accepted theory, you'd want to run more tests. BUT...to say that both audiophiles have produced reports of EQUAL reliability or worth would be an unjustified conclusion. One audiophile didn't even get off the ground in terms of proper methodology.

So...yes we would both be producing anecdotal accounts.

But, no, that doesn't mean your methodology is equally valid. And it doesn't mean one report isn't on stronger footing than another, for that reason.

Leaning pretty heavily on those strawman arguments.
Remarkable how you I jump in and out of science to support your claims

 

[Gregadd]

When you rely on science e you don't get to cherry pick. You have to strive tly follow the protocol. You did not follow the protocol. The fact that you wore a mask of that or that your father in law is an engineer is irrelevant entirely. I .might add that the fact out that failed to follow the methodology seriously affects the validity of the results and how you can apply them.
Sorry if you don't like that. It is science.

 

[Bobvin]

The great thing for those who don’t hear a difference between cables is the hobby is much less expensive for them. And they probably get along better with their spouse too.

It leaves more money for fine wine (if you can taste a difference), or more music in your collection.

No doubt there is a lot of marketing vocabulary that is nonsense. I, for one, tend to run away when the “quantum” term comes out.

Leave people alone to buy what they want, they earned it to spend as they choose, using whatever logic helps them justify. We all are trying to make sense of the world in our own ways. Life is hard, fun is the best revenge.

 

[COF]

Leaning pretty heavy on that strawman th strawman arguments.

It's relevant that you did quite blatantly strawman my position "You don't get to parade your anecdotal pseudo scientific evidence as fact."

But you are using that to ignore that I've gone far beyond merely pointing out your strawman; I've provided lots of detail in terms of argument and reasoning for my position, most of which you continue to ignore.

Remar9kable how youa I 0ojump in and out of science to support your claims

I think it's plain by now that you saw a post that seemed skeptical about power cables, wanted to gripe, but are having trouble engaging with a nuanced position. You can't actually show any inconsistency in what I've written, or demonstrate it is unreasonable, because I've given the proper caveats.

 

[Ron Resnick]

The great thing for those who don’t hear a difference between cables is the hobby is much less expensive for them.

This assumes that those who do hear differences between cables don't subjectively prefer the less expensive cables.

 

[COF]

When you rely on science e you don't get to cherry pick.

I've already given the caveats - I am not a scientist, but I WAS at least attempting to account for sighted bias effects (that is: knowledge of what gear was being used). That already is a step beyond just ignoring the problem.

You have to strive tly follow the protocol.

Which protocol in particular? There are various protocols in science generally, blind tests (single or double etc) in particular.

you did not follow the protocol the fact that you wore a mask of that you were a .mask or that your father in law is an engineer is irrelevant entirely.

No it's not irrelevant that I wore a mask. It prevented me from seeing which cable was being swapped in (made it easier than just closing my eyes). Also, we accounted for another variable - the possibility that I could detect which cable was being used simply by listening to my helper's movement when swapping cables. We did a pre-test, no music, swapping cables, and I was unable beyond random guessing to detect which cable my helper had swapped.

Therefore: I had good grounds on which to infer I did not have knowledge, including cues I was not aware of from my helper, of which cables were which during the testing.

We also used a multi-meter to ensure levels weren't changing at the speaker terminals between AC cables, btw...helpful my father was an engineer with the suitable equipment...not that anyone can't get it themselves).

The switching pattern was decided by my helper flipping coins to randomize the switching pattern (writing it down on paper, hidden from me. I've generally used a randomized switching protocol, though these days my helper tends to use and on-line random number generator instead of flipping a coin).

During the test the only verbal communication I had with my helper was to say "switch" when I was ready to do so (and to say my guess after the switch). At which point the helper would pull out the cable - even if his switching pattern was to keep the same cable...this is so I couldn't decide it was the same cable just because I didn't hear him switch anything. So he always unplugged the cable and plugged one back in. Again our pre-test suggested this method was sufficient to not allow me to guess merely from the sound of the switching.

I would say which cable I thought I was hearing, with no response from my helper who would only write down my guess beside his switching pattern on the same paper, so we could later compare the two.

So...there's more detail.

Now, this wasn't a scientific study. But it WAS an attempt to control for sighted bias. Therefore mere implications that the test was worthless aren't sufficient to invalidate it.

Given what I have described: is it MORE likely the test was flawed, that I could know what cable I was hearing (and if it was, why were my guesses random?)...or MORE likely that it was sufficient to control for my knowing which cable was used?

 

[Al M.]

No doubt there is a lot of marketing vocabulary that is nonsense. I, for one, tend to run away when the “quantum” term comes out.

Beyond Just Technology: SuperUltraHyperTechnology

PYST cables are made from only the finest 6-nines Unobtanium™ alloy, molecularly assembled in our Alternate Universe™ reality-distortion tesseract field, using a secret geometry reverse-engineered from crashed UFOs, painstakingly smuggled out of Area 51 by deep-cover operatives. Performance is further enhanced by the use of a QuantConnect™ quantum-entangled pair of transmission interfaces, held at absolute zero by our exclusive Stasis Field™ technology. The cables are then wrapped in NanoAeroCap™, a nanotechnology-enabled aerogel anti-capacitance insulation system, featuring Fractal Interleaved Geometry™ to create negative inductance for maximum audio transmission quality.

Or, er, well . . . again, no. These are nice, high-quality cables, with solid, reliable connectors. That’s it. Hope you like them!

From:

Schiit Audio: Audio Products Made in USA

www.schiit.com

 

[COF]

This assumes that those who do hear differences between cables don't subjectively prefer the less expensive cables.

That's true in principle.

But how often, really, do we hear of people - those inclined to test out different AC cables - preferring cheap, supplied power cables vs more expensive high end cables?

I think, yes, we sometimes see people say "I actually preferred the less expensive cable," but from what I've seen, that's usually an audiophile comparing between a selection of high end cables, which are already significantly more money than just sticking with the supplied AC cable. So it still usually means more money was spent on a new AC cable than sticking with the supplied cable.

(Not that someone shouldn't spend more money if they want, of course).

 

[Ron Resnick]

That's true in principle.

But how often, really, do we hear of people - those inclined to test out different AC cables - preferring cheap, supplied power cables vs more expensive high end cables?

I think, yes, we sometimes see people say "I actually preferred the less expensive cable," but from what I've seen, that's usually an audiophile comparing between a selection of high end cables, which are already significantly more money than just sticking with the supplied AC cable. So it still usually means more money was spent on a new AC cable than sticking with the supplied cable.

(Not that someone shouldn't spend more money if they want, of course).

I hear ya.

But there's a bit of a false dichotomy here. I was not thinking of the supplied cable as the only inexpensive option.

 

[Gregadd]

I never said anyone's evaluation was invalid. I did say in order to extrapolate the result to the general public you must follow a strict criteria. Bo exceptions.. Trying to account for one variable does not cut it.
I can't be any clearer..

 

[henrich3]

I just use the AC cables that came with my equipment. If they were good enough (sufficient gauge) for the manufacturer to include, they're good enough for me. An AC cable is not a power conditioner. I have no beliefs that AC cable upgrades make any audible improvement.

My speaker cables are all generic 12 AWG. I've read enough blind cable comparison tests & measurement results over the years to feel confident that I'm not missing out on even a tidbit of sonic excellence by not upgrading to some esoteric $1000 per foot speaker or AC cables. I personally consider most of the manufacturer claims for these uber expensive cables to be snake oil. I prefer putting my money into things that actually do make significant sonic improvements, eg. acoustic treatments and a good room EQ solution. If folks want to buy expensive cables for "pride of ownership" or aesthetic reasons, that's all well & good, of course.

The OP set out to do blind testing. That's a worthy effort! Assuming that it's properly controlled, I would just suggest that there should be multiple listeners and a sufficient number of tests so that accidently guessing correctly even if there's no real audible difference won't influence the outcome. I'm no scientist, but it does seem that the OP's original plan of having one listener do six listening tests would not be sufficient to obtain a trustworthy result.

 

[sbo6]

I applaud your prospective efforts. I try to blind test as often as possible and find it a great way to add confidence to your possible purchases.

Also, I don't buy the "stress" that some say contributes or colors listening. I find I'm relaxed, open and curious to perform the test and learn of the results.

Also, I find the faster the switching, the easier differences are to detect.

Enjoy!