Good post!
I hold these two propositions:
1. The scientific method, and knowledge gained from it, as well as from engineering, is deeply relevant to claims made about audio gear. It is the most reliable method of vetting such claims.
however
2. No audiophile is required to use a scientific method, or engage in any such rigorous methods of inquiry, when practising this hobby. Every audiophile is, and should be, free to enjoy audio gear in any way he/she pleases.
The problem I've found in ever introducing the relevance of science/engineering to purely subjective audiophiles, is that they tend to see a defence of #1 as an attack on, or undermining of #2. Like "who are YOU to browbeat others in to how to do this hobby!?? You are only out to troll and ruin people's enjoyment!" When that is not what is happening at all.
As for #1, the scientists in this thread should appreciate why the scientific method arose in the first place, and why it has been so successful in gaining us reliable and predictive knowledge. We humans are basic empiricists, we have managed to survive by doing basic forms of testing and inferences (e.g. how to grow crops, raise animals, build homes, etc). However we have massive liabilities that mean we also produce a huge amount of error. It was the honing of the scientific method, where we really take seriously the ways we can go wrong (e.g. biases, sloppy inferences) in to the method itself, so you control for variables, including bias, and you have others trying to prove you wrong, checking your work. It is a FAR more rigorous method than everyday empiricism. Everyday empiricism of "try this claim and see if it works" is sloppy enough to have people believing in untold number of false, wacky, unscientific beliefs. "Skeptical" people tried astrology, New Age medicine, spirit readings, palmistry, you name it, "and found it worked!" And the reason was their sloppy method of "testing" and inference making and ignoring the skeptical scrutiny of others checking their work.
This is why "I tried it and it worked" doesn't reach anything like scientific confidence levels.
It's why it doesn't matter, for any claim, or anyone "experimenting with things at home," it doesn't matter if one is a scientist in their day job. What matters is the METHOD, the rigour, they are using. That's why it's their in their day job as a scientist!
That said...and speaking to #2 above: The fact is none of us are in in a position to scientifically vet our every decision. That's impossible. So we have to recognize we can relax our standards to allow rational inferences using "every day empirical insights."
And after all, engineers through time have often not been doing science per se, but a lot of "try and see what happens."
For me, the way I reconcile and balance these two issues - 1. science is the gold standard for testing and 2. We can not use this method for much of what we are doing in life - is to use a heuristic what we all tend to use: Extraordinary Claims require extraordinary evidence. Applied to audio, if I'm evaluating two loudspeakers like a B&W vs a Revel speaker, and detecting a rising high end in the B&W and it's sonic consequences in the recordings, it's possible there is a bias effect going on. But it's also an extremely plausible phenomenon! So I can accept what I perceive (with the appropriate background caveats and confidence levels). But if I'm 'hearing' differences between usb cables, or some AC cables or things that I know to be quite controversial among technical experts, then I'm much more cautious in my confidence levels. I'd prefer to see more rigorous evidence, both technically and in listening tests (which is why I have done blind testing).
But as I said, no audiophile is REQUIRED to even think like that. I love my tube amps and have done tube rolling, which seems to alter the sound. I also seemed to have heard some difference between the same amp design with different capacitors. I understand this is controversial among some technical experts, but I'm fine going with my impressions because I acknowledge I'm not practising science, just trying to have fun and please myself.
However, IF someone says "it's possible you think your tube amps are just bringing 'warmth' to the sound because you expect it, or because of how they look with the glowing tubes" etct, I am NOT going to rail at the skeptic saying "What do you know? You must have ears of cloth if you can't hear what I hear! Or your system must suck in terms of resolution!"
No, I will happily acknowledge they are right - it COULD be my imagination, and without a more rigorous test, I'm not in a position to claim otherwise to anyone else. (Which btw, is one reason I did blind test my tube pre-amp against my solid state preamp, and easily passed that blind test).
So I don't care how any audiophile enjoys tweaking his system. But if they are going to leverage that to making objective claims about the sound really changing, or proposing dubious technical stories about how that is happening, then my critical thinking cap goes on.
Thank you for this sober and nuanced essay. This is how I think of most of these topics as well.