Does Everything Make a Difference?

[NekoAudio]

The current recommended test methodologies are defined in:
- BS.1534 : Method for the subjective assessment of intermediate quality level of audio systems
- BS.1116 : Methods for the subjective assessment of small impairments in audio systems

These are the test methodologies that are followed in rigorous double-blind audio testing, and in regular training of experts working in the field. (Just like anyone an athlete that trains regularly.)

 

[Mike Lavigne]

Interesting, but not relevant to our hypothetical audiophile under test who believes he can hear differences only over longer time frames.

that's not quite right. way too two dimensional. the issue is not that the subjectivist needs the longer time to hear things. differences, and better or worse, come in all degrees and types. however; part of judging is the emotive part. some differences need to be lived with to determine value. in different head spaces and in multiple sessions. and there are so many kinds of changes that you just cannot boil it down to something simple.

when i set up a cartridge and change VTA or VTF or other things i'm listening for some particular things. but if i'm judging an acoustical change or trying out a grounding product, or maybe my anti static products..........i'm listening for different types of things each case, and i'm pre-focusing to a degree into certain limited areas for efficiency determined by lots of practice with those items. but other times my sense is very general and less focused.

so there are so many approaches to listening, you just cannot nail it down. and being rigorous about blind testing or just quick A/B's are of limited effectiveness. you learn what works. no approach is 100% perfect. time is the proof. it helps a lot to have a system you trust, so you know certain things to begin with, which is work to achieve.

 

[Ron Resnick]

The current recommended test methodologies are defined in:
- BS.1534 : Method for the subjective assessment of intermediate quality level of audio systems
- BS.1116 : Methods for the subjective assessment of small impairments in audio systems

These are the test methodologies that are followed in rigorous double-blind audio testing, and in regular training of experts working in the field. (Just like anyone an athlete that trains regularly.)

Thank you, but these protocols do not help the matter. We are past these conventional protocols. Conventional protocols are not acceptable to subjectivists.

I see no point in banging the same objectivist blind A/B test with quick swaps drum.

I'm trying to come up with an idea which both sides can agree a priori is satisfactory in concept.

 

[Elliot G.]

Thank you, but these protocols do not help the matter. We are past these conventional protocols. Conventional protocols are not acceptable to subjectivists.

I see no point in banging the same objectivist blind A/B test with quick swaps drum.

I'm trying to come up with an idea which both sides can agree a priori is satisfactory in concept.

Best of luck with that!

 

[bonzo75]

On this issue usually where the rubber meets the road is the question of time frame. Objectivists want to assess quick A/B switching. Subjectivists reject that methodology and want to live with each of A/B for some time.

Ask the audiophile in question what time frame he/she needs to feel comfortable hearing the difference (this could very well be days or weeks), and then plan a "properly controlled blind listening test" around that exact time frame, and maybe we will have a conclusion we all can agree on for that particular audiophile assessing that particular possible difference.

this is an incorrect representation of the problem. For example, when the Soulution pre was placed in my friend’s Allnic Avalon system, it took me C 15 minutes to tell him he should buy it. He agreed with all my conclusions he heard them himself. After I left, it took him one month to decide. Why? 1) it was his money 2) my records left with me and on his records, you can’t always make the difference our that quick

That was the Soulution 720. When the 720 had to be sent for repairs, the next upgrade 725 was placed free in his system by the dealer. I told him this is good but not as good (IIRC Shakti found the same in his system the 720 to be preferable). Due to the pandemic and Soulution service, the 720 returned two years later. The 725 was being offered to him for only 6k GBP more. He declined, as he now after keeping both in his system heard the exact thing. I can cite more such examples

the difference here is more than A/B. You can A/B quickly, but then check if that difference is repeatable across records. During the A/B, you can look for artefacts which will make the difference boring in a few weeks (e.g. homogenising across records despite having a positive difference on the first record).

In order to A/B efficiently, you need to have good listening points (that is, the right type of records with the right type of music and points to listen to on the records). You should have a variety of records. I can’t use my friend’s records or your fields of gold to make meaningful assessment as quickly, or as correctly.

After that, meaningful differences in the same system stand out pretty quickly. In different systems due to different synergies and set ups they might differ. If the differences are marginal and you can’t make out easily, just go with the better deal instead of trying to find meaning in the minute difference. Or, a system is very coloured and does not tell some differences when they are meant to be meaningful.

 

[Zero000]

this is an incorrect representation of the problem. For example, when the Soulution pre was placed in my friend’s Allnic Avalon system, it took me C 15 minutes to tell him he should buy it. He agreed with all my conclusions he heard them himself. After I left, it took him one month to decide. Why? 1) it was his money 2) my records left with me and on his records, you can’t always make the difference our that quick

That was the Soulution 720. When the 720 had to be sent for repairs, the next upgrade 725 was placed free in his system by the dealer. I told him this is good but not as good (IIRC Shakti found the same in his system the 720 to be preferable). Due to the pandemic and Soulution service, the 720 returned two years later. The 725 was being offered to him for only 6k GBP more. He declined, as he now after keeping both in his system heard the exact thing. I can cite more such examples

the difference here is more than A/B. You can A/B quickly, but then check if that difference is repeatable across records. During the A/B, you can look for artefacts which will make the difference boring in a few weeks (e.g. homogenising across records despite having a positive difference on the first record).

In order to A/B efficiently, you need to have good listening points (that is, the right type of records with the right type of music and points to listen to on the records). You should have a variety of records. I can’t use my friend’s records or your fields of gold to make meaningful assessment as quickly, or as correctly.

After that, meaningful differences in the same system stand out pretty quickly. In different systems due to different synergies and set ups they might differ. If the differences are marginal and you can’t make out easily, just go with the better deal instead of trying to find meaning in the minute difference. Or, a system is very coloured and does not tell some differences when they are meant to be meaningful.

No. But where is the constant?

 

[PeterA]

Thank you, but these protocols do not help the matter. We are past these conventional protocols. Conventional protocols are not acceptable to subjectivists.

I see no point in banging the same objectivist blind A/B test with quick swaps drum.

I'm trying to come up with an idea which both sides can agree a priori is satisfactory in concept.

Ron, what exactly are you trying to come up with and for what reason? Why does it have to be satisfactory to both sides? What is this theory supposed to be about?

it seems to me that people with experience in this hobby have determined what works best for them and they use that approach when making judgments.

 

[Another Johnson]

@bonzo75 - the point seems to be that it takes skill and patience to discern whether there are 1000 angels dancing on the head of the pin, or perhaps only 677 … or perhaps none.

I think this is why so many have trouble suspending their disbelief in the efficacy of many of the mystical tweaks.

What’s obvious and important to one, is just not critical to the musical enjoyment of another. It is the classical demonstration of YMMV.

 

[bonzo75]

@bonzo75 - the point seems to be that it takes skill and patience to discern whether there are 1000 angels dancing on the head of the pin, or perhaps only 677 … or perhaps none.

Yes so they should build up the skill to know what your of records, music, and points to play instead of discuss blind testing Vs how many hours to listen, which kind of assumes if you blind test or listen 100 hours with the same bad record on amplified female vocals you will end up with a good conclusion.

 

[Robh3606]

I don't know. I came up with the idea. Implementation of a practical experiment is not my problem.*

This discussion has allowed us to advance a theory beyond the usual rock'em sock'em robots of blind A/B testing on quick A/B swaps versus leisurely long-term listening.

*This week I watched the Oppenheimer movie, and I saw clearly the bifurcation between theoreticians and experimentalists.

So who's who?? Theoreticians = Subjectivists and Experimentalist's = Objectivists. So an audio Oppenheimer is needed??

Well if you don't have an agreed method that allows for practical experimentation you simply are not there yet.

So OK you have a theory now what???

I also don't see where this has changed any ones mind. It's a civil discussion and that's a start.

Rob

 

[COF]

It is often said by many audiophiles that "everything makes a difference." Beyond different cables and different components making a difference, these folks mean that everything makes a difference:

-- the metallurgy of the duplex outlet wall cover plate

-- a block of wood or other material on the top of a component

-- a Shun Mook disc or Shakti Stone on the top of an amplifier

-- the power cord going into the power supply of a turntable motor

-- the DAC is plugged in when you are listening to vinyl

-- interconnects are elevated above the floor

-- the metallurgy of the wire carrying power from the electrical sub-panel servicing the listening room to the outlets into which components are lugged

-- different fuses in a component

-- power cables are not criss-crossed with signal cables

-- an unused amplifier is sitting on the floor of the listening room

-- the connector on the speaker cable is Rhodium plated or not

-- the unused jacks on the pre-amp are plugged up

-- a coffee table is in front of the listening chair

-- an extra piece of wire "dongle" is inserted somewhere

-- a block of wood or steel or granite underneath a component

-- the ASC TubeTrap is diffusion side out or absorption side out

-- a clock is plugged into a AC outlet in the listening room

-- the wire coming out of your Wi-Fi router

-- the USB extension bus ("switch") going to your streamer

-- etc.

Does everything really make a difference? Or do we just believe -- or do we just want to believe -- that everything makes a difference?

Do we want to believe that everything makes a different because we think we can hear a difference?

The audiophile belief that "everything makes a difference" makes no sense in terms of known engineering and science - we know plenty about the limits of human hearing, and there is audio gear that passes along the signal without rising to audible levels of distortion, and all sorts of "tweaks" that have no hope of altering the sound.

Instead "everything makes a difference" is exactly what you'd expect when you allow for imagination to play a role. Because then someone can always imagine a difference. That explains why there has virtually never been an audiophile tweak that someone has dreamed up, no matter how wildly implausible, that doesn't have some cohort of audiophiles imaging it makes a sonic difference.

 

[Al M.]

The audiophile belief that "everything makes a difference" makes no sense in terms of known engineering and science - we know plenty about the limits of human hearing, and there is audio gear that passes along the signal without rising to audible levels of distortion, and all sorts of "tweaks" that have no hope of altering the sound.

Instead "everything makes a difference" is exactly what you'd expect when you allow for imagination to play a role. Because then someone can always imagine a difference. That explains why there has virtually never been an audiophile tweak that someone has dreamed up, no matter how wildly implausible, that doesn't have some cohort of audiophiles imaging it makes a sonic difference.

Not everything makes a difference. Some tweaks make a difference in some systems, not in others. That's not imagination, it's experience. And of course, some tweaks may not make a difference at all, indeed.

As for "known engineering and science", I would be careful with that. I am a scientist myself (a biochemist), and as such I know that we don't know all there is to know and that our current measurements do not reveal all there is to know. Only cocky engineers lacking a scientific mindset think that we know how to measure all there is to measure. A little humility is advised.

 

[COF]

Not everything makes a difference. Some tweaks make a difference in some systems, not in others. That's not imagination, it's experience. And of course, some tweaks may not make difference at all, indeed.

Well, to be honest, it would depend on what tweaks you think make a difference. Since audiophiles can easily imagine differences, and I can find a testimony for any tweak out there, I hope you'll understand - especially as a scientist - that I'm not apt to just take your word for it

As for "known engineering and science", I would be careful with that. I am a scientist myself (a biochemist), and as such I know that we don't know all there is to know and that our current measurements do not reveal all there is to know.

Ok, but scientists are also supposed to have humility in regards to recognizing their narrow scope of knowledge. So the fact you are a scientist doesn't necessarily mean you know what is known, or not known, in the realm of audio, psychoacoustics etc.
I mention this because it's incredibly common for audiophiles who believe in various tweaks to say "We don't know X about audio" when what it really means is "I don't know X about audio." They are just not aware of what IS known. I'm not saying I know this to be true for you, but the "there's a lot science doesn't know" is a common red flag for mumbo jumbo and pseudo-science.

Only cocky engineers lacking a scientific mindset think that we know how to measure all there is to measure.

The engineers don't need to know everything possible there is to measure, to know enough that many audiophile claims are highly implausible, or flat out impossible. Again...it all depends on the claim.

Since you are a scientists you should recognize this basic issue: before assuming "we need to explain X" you should be able to demonstrate "X" exists. So in other words, if a homeopath says "science really needs to explain how homeopathy works!" they are begging the question, cart before the horse. They would first demonstrate homeopathy works - produces statistically relevant results - in a scientifically rigorous fashion, before science "needs" to do any such explaining.

Likewise, when audiophiles say "well, maybe one day science will explain why I hear what I hear when I swap my USB cables" that's also question-begging: a scientist would recognize that one should be able to demonstrate that phenomenon, with some scientific rigour, before assuming it actually exists, and "might be explained some day." (This pertains the more we are in to the realm of "extraordinary claims" - in the case of audio, technically implausible or un-demonstrated claims).

A little humility is advised.

I couldn't agree more. I try to stay in my lane in terms of technical expertise (I'm in pro sound, but I'm not an electrical engineer etc). I also try to stay humble regarding the perceptual liabilities I share with every other human being, and scale the confidence of my beliefs to the plausibility of a proposition. I realize I'm as prone as anyone to imagine differences that aren't there. (And blind testing numerous audio devices has been very educational).

Can you tell me: do you share the same humility regarding the tweaks you think make a difference? That is, if you seem to hear sonic differences in a tweak you are aware is controversial, do you take in to account you could be imagining it?

Cheers.

 

[treitz3]

One imagines winning the lottery. I would venture to say that most of us here at the WBF actually listen. There is no "belief" in the equation.

Tom

 

[Chop]

Here's my two penny worth...
Looking at Ron's original list on page one, I think I have heard about 80% of those making a difference. But here's my problem with trying to listen for the effect of a tweak: If I touch NOTHING on my reasonably resolving system it can sound different one day to the next anyway!!
So... Is this due to the variable quality of my electrical supply? Temperature or humidity? Ley lines? The wind blowing from the West that day? Me being in a better or worse mood that day? They are all things I can do nothing about with tweaks.

Whatever, it sounds different: The problem is that if we accept the fact that the quality of sound our system provides can be different from day to day, surely we have no chance with any form of short term analysis of whether a targeted change is making a sustainable improvement. Personally I can't rely on a quick analysis of a change.

When I'm not relentlessly fiddling with the system I listen to music for an emotional response, not how deep the bass is or how good the imaging is. The changing level of emotional involvement is the longer term goal.
The only thing I have found that works when considering a tweak is to split my listening into analytic and emotional sessions. I make a change, and immediately listen analytically, & I force myself to write down exactly what I did ('cos I'm dopey and I might forget). I then leave the change in the system and try to switch the analytic brain off. I just listen to the gestalt of a variety of music over a few days and see what it feels like. Sometimes it feels worse, sometimes better.

Of course, another conclusion would be that if the sound changes from day to day anyway why make tweaks at all, but I'm not going there - its just fun to fiddle with it, right?

 

[Al M.]

Well, to be honest, it would depend on what tweaks you think make a difference. Since audiophiles can easily imagine differences, and I can find a testimony for any tweak out there, I hope you'll understand - especially as a scientist - that I'm not apt to just take your word for it

Of course you shouldn't.

Ok, but scientists are also supposed to have humility in regards to recognizing their narrow scope of knowledge. So the fact you are a scientist doesn't necessarily mean you know what is known, or not known, in the realm of audio, psychoacoustics etc.

Agreed, i don't know everything about audio science. But I do know that psychoacoustics, for example, is much more complex than some audio engineers would admit. That right there is a red flag.

For example, THD measurements are only relevant to a point -- the ear reacts differently to different harmonics, and some higher uneven harmonics have nasty effects on aural perception at very low amounts, hiding under very low overall distortion numbers. Yet some engineers (and customers) treat THD numbers as a holy grail, without bothering about the underlying details. Preposterous, and indicative of a pseudoscientific mindset.

I'm not saying I know this to be true for you, but the "there's a lot science doesn't know" is a common red flag for mumbo jumbo and pseudo-science.

Agreed, that is a danger.

The engineers don't need to know everything possible there is to measure, to know enough that many audiophile claims are highly implausible, or flat out impossible. Again...it all depends on the claim.

Since you are a scientists you should recognize this basic issue: before assuming "we need to explain X" you should be able to demonstrate "X" exists. So in other words, if a homeopath says "science really needs to explain how homeopathy works!" they are begging the question, cart before the horse. They would first demonstrate homeopathy works - produces statistically relevant results - in a scientifically rigorous fashion, before science "needs" to do any such explaining.

Likewise, when audiophiles say "well, maybe one day science will explain why I hear what I hear when I swap my USB cables" that's also question-begging: a scientist would recognize that one should be able to demonstrate that phenomenon, with some scientific rigour, before assuming it actually exists, and "might be explained some day." (This pertains the more we are in to the realm of "extraordinary claims" - in the case of audio, technically implausible or un-demonstrated claims).

You bring up good points worth considering. Not that I agree on all thoughts completely, but I see your points.

I couldn't agree more. I try to stay in my lane in terms of technical expertise (I'm in pro sound, but I'm not an electrical engineer etc). I also try to stay humble regarding the perceptual liabilities I share with every other human being, and scale the confidence of my beliefs to the plausibility of a proposition. I realize I'm as prone as anyone to imagine differences that aren't there. (And blind testing numerous audio devices has been very educational).

Can you tell me: do you share the same humility regarding the tweaks you think make a difference? That is, if you seem to hear sonic differences in a tweak you are aware is controversial, do you take in to account you could be imagining it?

Cheers.

Absolutely. I am always skeptical with tweaks. With any change in a system I ask myself if this is real, and I go back and forth repeatedly (or ask the host to do so) to make sure that I (or the host or both) am not imagining things. Often it's just imagination after all. I have the humility not to trust myself a priori.

It is always funny to me to read a report of "we changed this thing and then we heard this difference", and that report touts things as fact. I mean, did you ever go back and verify?

 

[mtemur]

IMHO everything makes a difference, that’s for sure, but the meaning of “everything” depends on a couple of things;
- how good or how responsive your system is,
- how good your ears to detect small changes,
- how much you’re accustomed to the instruments.
When these things improves the definition of everything enlarges too.

P.S. I’m saying that I’m at the last leg of this improvement. No, probably I’m at the beginning but I’m certainly learning by keeping an open mind.

 

[mtemur]

One important factor I forgot to mention is treated rooms. IMHO/IME treated rooms full of absorbing materials have their very strong character. I heard smaller effects from tweaks in absorbed rooms.

 

[Tangram]

Not everything makes a difference. Some tweaks make a difference in some systems, not in others. That's not imagination, it's experience. And of course, some tweaks may not make a difference at all, indeed.

As for "known engineering and science", I would be careful with that. I am a scientist myself (a biochemist), and as such I know that we don't know all there is to know and that our current measurements do not reveal all there is to know. Only cocky engineers lacking a scientific mindset think that we know how to measure all there is to measure. A little humility is advised.

As a fellow scientist who, like yourself, trained in a field far removed from audio (in my case geoscience), I’m not surprised our beliefs are aligned. What DOES surprise me is that the scientific method - along with the null hypothesis - are parts of the high school science curriculum; at some point most people learn this, but it seems like a post-secondary reinforcement is required for it to stick.

Humility is an inherent aspect of good scientific research because without it, bias may impact results. “Oh, we don’t have to test for that because we know it isn’t important,” is a career-limiting mindset. In addition, scientists know that the outcome of experiments depends significantly on factors that lay people - who go straight to the punchline - don’t appreciate, whereas scientists study how the experiment was designed and controlled before opining on the results. In my view, this is the number one problem with many audiophiles - all they care about is the punchline.

I like trying tweaks, but only if they are free or “try before you buy.” Even though my scientist days are far behind me, I consider my stereo system a lab, and trying a tweak is like conducting a little experiment. My processes are crude and flawed compared with real science, but being humble, minimizing bias, having some understanding of psychoacoustics and trying to control variables as much as possible, those are all parts of the hobby’s fun. This has yielded some surprises for sure, ones that undoubtedly are caused by the limits of my hearing, system and room, but unlike “real” science, I am not in search of universal truth, just the truth that exists in my listening room.

 

[Mike Lavigne]

here is how i view 'everything'. 8 years ago, i had been in my dedicated room for 11 years. over those 11 years i had made various changes to the room, but even through those efforts all was not settled. then i happened to visit a friend with the same gear as me, and heard some things about his room that opened my eyes to my own room. which gave me a reference point in my mind.

which got me thinking. two months later i was ready to act.

looking at my ceiling i wondered if i was getting a reflection from it that was obscuring clarity. so i took some cloth i had, and thumb tacs, and attached it to that spot on each side. and listened. better. listened for two weeks. ok, this was definitely better. then i tried it in the center area on the ceiling, even though the geometry was not obvious to be solving a direct reflection. again better immediately, and also after another two weeks of listening. this then started a process that caused me to travel around my room doing this trial and error. cover the surface with cloth, then listen for 2 weeks. it was like finding a door to another level deeper of musical truth, opening the door, then finding another door behind it, opening it. then removing the wall, and finding another door.

i came to the center skyline diffuser on my wall behind my speakers, which had been there for 5-6 years and had helped calm my center image. i wondered if with the treatments i had done whether this was still helping? was this now a wall in my way? i removed it. better, listened for two weeks. still better. added cloth on that center round diffuser. better.

came to my two front windows. 5 years prior when i removed some bass trapping i had the contractor build window inserts. however; at that time in my room development i could not tell the difference between in and out musically, i liked the view, so left them out, upstairs in my attic.. but now with greater clarity in my room, would i hear a difference? would it be positive? i installed them, much better, more filled in seamless soundstage. covered them with cloth, even better.

both the skyline diffuser and window situation had worked in a previous reality, but until i solved other underlying issues, the benefit of fixing those was not possible.

i was working on my side walls and experimented with many things. eventually i realized i had a resonance between the large side surface of my bass towers and my side wall, and added some t-fusor diffusors to that area. but i could only localize the issue since i had added some much more clarity.

at the end of this process i discovered i could hear the reflectivity of the meter bridge's on my Studer tape decks since they were at ear level to my direct right as i listened. i covered them with a towel, better. that was something not audible 9 months prior. now everything was revealed to matter, since the other issues were no longer covering them.

i've not touched my room acoustically since then, 8 years ago. it was a lot of work, but i've loved it since and had zero thought of messing with it.

my whole point being that everything does matter when you start to get the room out of the way of the music. but many times in spite of much effort there are things in your way so you cannot really understand cause and effect. this same process works on the power grid, the signal path, listening position, speaker set-up, etc. etc.

does everything matter? how deep do you want to truly look? not everyone is willing to look that hard. but that does not mean it all does not matter. you might have a number of layers of stuff in your vision clouding whether things matter.

you do need a reference point, and you do need to believe your ears. otherwise, you will never even get started.