If all conversations ultimately degrade to sex or farts, why is it on this forum they degrade to objective vs. subjective impressions of music/sound/equipment. 23 pages in, can we get to the sex or farts already!
Comparative Listening Tests
- Thread starter Ron Resnick
- Start date
- Status
I went around and around with Klaus in the past...not worth the effort to do so again. It is talk like this that gives non-scientists the impression that us scientists (yes, I am a scientist...and not a "hack" either) are close minded when actually it is the open mindedness of scientists that allows them to make the amazing discoveries of the last couple hundred years. Klaus talks more like an engineer than a scientists. Many engineers operate that because a technology does ostensibly what it is supposed to do that it is "worked out" and one can simply apply the right formulas to get the desired output. Science is about observation, hypothesis based on observation and then attempts to falsify that hypothesis. Many non-scientists stop at the first step, which is observation but do not or are not able to put those observations into a more general framework that allows for some measure of predictability based on the observations. They just have a collection of observations that they true to use intuitively rather than systematically.
Engineers (not all but a large %) are at the other end of the spectrum where the main aspects of the chemistry and physics are worked out by theory (hypotheses that have stood up to debunking) but don't realize that further observation is required to perhaps get the desired outcome...particularly when a human interface is required and an aestethical decision is the endpoint. This is where in audio there has been an epic fail over the last 70 years, IMO. Engineering without human factor consideration and turning off controlled observation and turning primarily to meter reading. There is a tendency for engineers to be reductionist in the parameters that are critical to the performance of a thing. Klaus wants you to believe that FR is paramount for speaker quality and that electronics and wires are minimal and possibly delusional minor contributors. Careful observation tells you that this viewpoint is obviously flawed. However, these reductionist attempts are simply a manifestation of a belief that it is "worked out"...after all audio is a "mature" technology. Preconceived belief KILLS observation. A scientist should enter every experiment as free of expectation as possible...like a little child and let the observation guide the analysis. Then that can be put into a framework later against previous data and/or hypotheses and if it fits a given paradigm or not. Sadly, experiments are usually gone into with the observations already anticipated...nevermind the interesting stuff is always what you least expected...
steve williams
Site Founder, Site Owner, Administrator
Excellent summary and I couldn't agree more as to your description of an engineer vs a scientist.
Need to annoy you once more.
Didn’t know that you’re the spokesperson for other WBF members!
In #189 I said: “We never had the ambition to do a serious test anyway, since my colleague, just like me, does not believe in cables sounding different.”
You can read, can you? What did I say? Does not BELIEVE, that’s what I said. Now you turn this into “Klaus claims that there are no differences between cables” as can be found in #412
Believing and claiming in my book and every dictionary of the English language is not quite the same thing, sure that you’re a native speaker?
You can’t even read correctly but you have the guts to state
Not only you should consult physics textbooks but also a dictionary of the English language from time to time to get at least the basics right, methinks.
stehno said:
Didn’t know that you’re the spokesperson for other WBF members!
In #189 I said: “We never had the ambition to do a serious test anyway, since my colleague, just like me, does not believe in cables sounding different.”
You can read, can you? What did I say? Does not BELIEVE, that’s what I said. Now you turn this into “Klaus claims that there are no differences between cables” as can be found in #412
stehno said:
Believing and claiming in my book and every dictionary of the English language is not quite the same thing, sure that you’re a native speaker?
You can’t even read correctly but you have the guts to state
Not only you should consult physics textbooks but also a dictionary of the English language from time to time to get at least the basics right, methinks.
Last edited:
morricab said:
By this you don’t mean sighted listening, or do you? Because this is what this whole thread is about:
Ron Resnick said:
If temperature affects you meter reading, you have to take that into account. If knowing the identity of the DUT affects the results of your listening test, you have to take that into account. Sighted listening cannot do that, blind listening can. Simple as that.
morricab said:
Since audio gear is supposed (as least that’s what I’ve always thought) to not add to or subtract from from what’s going in, of course FR is the very first, but not the only, parameter that should be looked at. If OTOH the goal is not to not add or subtract, then one wouldn’t care about FR, obviously.
Those who want to know the truth apply scientific methods, in this case the controlled listening test. Those who only care about whether or not the sound is pleasing to their ears do sighted listening. I cannot see why both approaches should not co-exist and be mutually accepted and tolerated.
I made no edits, i.e changes. I quoted Ron's request at the end, word for word. You simple added the background which does nothing to deter what Ron was saying.
Once again you avoided replying to what he is asking. Why? What concerns do you have to say whether you agree or disagree with Ron? I have given my answer. Peter was kind enough to do the same. The rest of you do everything in your power to change the topic. Why?
You rather bicker, discuss other members, than letting the message from Ron sink in. A dose of reality that is too painful. But it is reality. Presented by Ron with Al's help giving it credibility. Yet you let it slide off your feathers.
Sorry no. Nothing you all talk about is about new scientific discovery. It is about ignoring the science that has for decades proven to be correct. Science had demonstrated time and time again that sighted observations are wrong. That placebo exists. No scientist in audio or medical field will throw that out for the purposes of "falsifying hypothesis." There is no hypothesis here. You simply cannot use uncontrolled evaluations in audio and arrive at anything but wrong conclusions. This is what science tells us.
Science has also extensively studied how we hear, again both for audio enjoyment and medical. A lot of audio research is also done by the way for improvement learning in school, i.e. hearing the teacher better. A lot of that is encapsulated in such bibles:
Those of you who say you hear things that science doesn't understand, have read this? I suspect not.
We can't operate at the zero line domain where we always assume we know nothing about audio perception. THat doesn't happen in any other science domain we live in. Scientist as well as engineers build on current knowledge. They are happy to go and investigate more but you have to give a reason more than, "I did this faulty test and hence science needs to investigate it." Well, no. You have to follow science in your testing and then the entire scientific world will listen. Otherwise we will be chasing every theory every layman has.
Just to be clear on where we are here, in the OP Ron quoted tests done by Al and Peter showing that sighted outcome did not exist for unsighted. That the cable preferences in sighted did not transfer to when the knowledge of cables was removed.
I also presented a test that anyone can run: play the same file multiple times and see if your observation of its fidelity changes.
Neither one of these facts have been disputed. The logical conclusions is what Ron eloquently put. That there is a degree of error -- potentially large error -- in the type of evaluations audiophiles do. Honesty therefore requires one to accept that, and incorporate it into your future testing and reporting.
This is it really.
I also presented a test that anyone can run: play the same file multiple times and see if your observation of its fidelity changes.
Neither one of these facts have been disputed. The logical conclusions is what Ron eloquently put. That there is a degree of error -- potentially large error -- in the type of evaluations audiophiles do. Honesty therefore requires one to accept that, and incorporate it into your future testing and reporting.
This is it really.
Here's another book some people should look into:
Description
As audio and telecommunication technologies develop, there is an increasing need to evaluate the technical and perceptual performance of these innovations.
A growing number of new technologies (e.g. low bit-rate coding) are based on specific properties of the auditory system, which are often highly non-linear. This means that the auditory quality of such systems cannot be measured by traditional physical measures (such as distortion, frequency response etc.), but only by perceptual evaluations in the form of listening tests.
Perceptual Audio Evaluation provides a comprehensive guide to the many variables that need to be considered before, during and after experiments. Including the selection of the content of the programme material to be reproduced, technical aspects of the production of the programme material, the experimental set-up including calibration, and the statistical planning of the experiment and subsequent analysis of the data.
Perceptual Audio Evaluation:
Provides a complete and accessible guide to the motives, theory and practical application of perceptual evaluation of reproduced sound.
Discusses all the variables of perceptual evaluation, their control and their possible influence on the results.
Covers in detail all international standards on the topic.
Is illustrated throughout with tables, figures and worked solutions.
Perceptual Audio Evaluation will appeal to audio and speech engineers as well as researchers in audio and speech laboratories. Postgraduate students in engineering or acoustics and undergraduate students studying psychoacoustics, speech audio processing and signal processing will also find this an essential reference.
Description
As audio and telecommunication technologies develop, there is an increasing need to evaluate the technical and perceptual performance of these innovations.
A growing number of new technologies (e.g. low bit-rate coding) are based on specific properties of the auditory system, which are often highly non-linear. This means that the auditory quality of such systems cannot be measured by traditional physical measures (such as distortion, frequency response etc.), but only by perceptual evaluations in the form of listening tests.
Perceptual Audio Evaluation provides a comprehensive guide to the many variables that need to be considered before, during and after experiments. Including the selection of the content of the programme material to be reproduced, technical aspects of the production of the programme material, the experimental set-up including calibration, and the statistical planning of the experiment and subsequent analysis of the data.
Perceptual Audio Evaluation:
Provides a complete and accessible guide to the motives, theory and practical application of perceptual evaluation of reproduced sound.
Discusses all the variables of perceptual evaluation, their control and their possible influence on the results.
Covers in detail all international standards on the topic.
Is illustrated throughout with tables, figures and worked solutions.
Perceptual Audio Evaluation will appeal to audio and speech engineers as well as researchers in audio and speech laboratories. Postgraduate students in engineering or acoustics and undergraduate students studying psychoacoustics, speech audio processing and signal processing will also find this an essential reference.
steve williams
Site Founder, Site Owner, Administrator
Sadly you can't tell the trees for the forest. We all knew what 853guy was alluding to. Just one more way to shift things away from you as you think you do so well. Keep believing Amir. Just another version of The Emperors New Clothes. You can say what you want but we all agree with 853guy
Unless the speaker has controlled directivity, then it's moot. The room can be almost as small as you need the gear to fit in. The confusion comes when the mind can't understand what it hears compared to sees.
That is something not really understood by too many normies. Scientist study the unknown to collect data. Engineers use data. That is the very essence of the two. And yet we get blasted with "scientifically made" bullshit all the time. It drives me crazy how it's, apparently, impossible for people to understand that not everything is going to be a double blind study to be valid, and things can be made without any "science" that are engineered. I'm not just talking about audio products, by any means. It's just that "science" is a buzz word that's as bad as "epic" or whatever.
But what is certainly funny is what you bring up with engineers inability to help themselves so often. People tell them what they want, and they don't even hear them speak... they are unable to comply with concepts that either degrade their engineering, or make no sense to them. Yet there's people standing in a line waiting to give them money if they'll do it.
This was pretty well covered before... A lot of people don't subscribe to having the sound reproduced as accurately as possible. It might even be a majority here. They just want it to sound a particular way that makes them think it seems like jazz is happening in front of them. So.... FR.... ya....
And yet you only offer controlled listening test approaches that lead to null results unless someone has very extended experience with them. So let's see, it's back to this argument; A. the listening tests are flawed because as currently how ABX is done they don't account for how humans recall audio. B. everything sounds the same and all purchasing patterns, reviews, and externally oblivious (such as spouse) observations are just manifestations of people being so damn stupid that they placebo themselves into every single decision because of biases mostly generated from marketing....
Obviously I don't subscribe to either. I believe if all experiments point to a null except when a biased subjectee is thrown into the mix - and yet we have 60 years of conflicting data with purchasing patterns, etc - then there's a problem with the experiment. As much as we probably do placebo ourselves, I don't think it's reasonable to assume that everyone is just really dumb, so it's not significant enough of a factor. If anything our wilfulness in belief simply stimulates purchasing more of a specific type of sound, not gear that we want to believe has the sound; as in since we believe X should be good we begin to use it as a sonic benchmark irrespective to what we would choose if everything was in generic colored boxes without prices so we buy that sound.
Now if you please, give us the old "I have biases too, but I'm smart, so the answer isn't that everyone is dumb, it's that everyone is so biased they can't make decisions at all" defence... for the court, if you please.
Certainly very applicable to this topic. I only have their AES presentation. And of course boatload of papers from Søren Bech in my library. Unfortunately the book is not available in Kindle format so I have not purchased it. I see some ebook versions in Europe though. Do you have the hardcover or ebook?
How do you know we are not "dumb?" It this were a math problem, then we would give you a test where we know the answers. If you got it all wrong we would say you are dumb with respect to math.
What tests have people have done in audio where the same is true? That is the answer is objectively and verifiably is known and they are tested without knowledge to see if they get it? Answer is never. Because if it is anything else, amazing amount of humbleness would be shown in testing any and all audio tweaks, equipment, etc.
This thread is an exception in that Al gave a test to Peter where Al did "know the answer." Result was a failure to get the right answer on Peter's behalf. Do enough of these and soberness sets in as to how dumb we really are in this regard.
Heaven knows I can tell you story after story on how dumb I am in this regard.
After a while you realize that the best way to not be dumb, is to not act dumb and keep insisting that our hearing perception is so great that it trumps everything science and engineering has shown.
How do you know we are not "dumb?" It this were a math problem, then we would give you a test where we know the answers. If you got it all wrong we would say you are dumb with respect to math.
What tests have people have done in audio where the same is true? That is the answer is objectively and verifiably is known and they are tested without knowledge to see if they get it? Answer is never. Because if it is anything else, amazing amount of humbleness would be shown in testing any and all audio tweaks, equipment, etc.
This thread is an exception in that Al gave a test to Peter where Al did "know the answer." Result was a failure to get the right answer on Peter's behalf. Do enough of these and soberness sets in as to how dumb we really are in this regard.
Heaven knows I can tell you story after story on how dumb I am in this regard.
That is not quite true, amirm. Al gave me a four tests:
1. Identify differences if any and preference if any between two cables, A and B.
I completed this test and was able to identify a difference, a preference, and able to describe differences.
2. Listen for a third time to X and recall whether it is A or B.
I failed to correctly identify X as A or B.
3. Using different musical cuts, listen to both cables again in random order and identify differences and preferences between A and B.
I completed this test and was able to identify a difference, a preference, and able to describe the differences. I was also able to pick which cable matched the cables in test #1 in the correct order, not knowing the identify of the cables. This contradicts result of test #2.
4. Listen for a third time to X with the new musical cuts and recall whether it is A or B.
This was difficult as I thought that the third sample did not match either A or B, yet I thought those were the only two possibilities, so I gave up saying I was confused. Al then revealed that he actually changed the parameters by rotating the Tube Traps while I was out of the room between #3 and #4, in effect creating a C variable. So we concluded that this was a flawed or false test because an additional variable was introduced without me knowing it.
You are incorrect to describe the event as one test. You are also incorrectly describing how I failed and not acknowledging that I did complete and pass two of the three given tests.
I can not concluded based on this one evening with 3 or 4 tests that I have done enough of these kinds of tests for soberness to set it and realize how dumb I really am in this regard. The fact that I could listen again to the two cables with different music and identify which order Al hooked them up in relative to the original order, gives me some level of confidence, though it contradicts results of test #2. Perhaps it was increased familiarity with the differences between the cables. In test #2 the music was unfamiliar to me. In test #3, I requested familiar music, so that may have helped.
Perhaps you and Klause can identify all sorts of flaws with this test, and it may not be rigorous enough for your standards, yet it must be somewhat legitimate for you to use my result in test #2 to prove your point that I need to sober up and admit how dumb I am and that others should read this thread and conclude the same thing. My mind is still open to whether or not these kinds of tests are definitive. I think that evening was useful, if only to help Al select a cable and to lead Ron to ask the question in his OP. It was a fun evening and an interesting experience. I did learn that being aware that there are two cables, and that I listened to one and then to the other without being aware of which is which, that I could hear differences, describe those differences, and identify a preference. (It happened to disagree with Al's preference.)
I did listen with my eyes open, but because I could not see either cable, and I did not know which one was hooked up at any moment, I think this can be considered "unsighted" but I don't really know. The room was illuminated and my eyes were wide open. Having read this thread, they are now wider open.
I'm off to listen to some Deep Purple on vinyl to pump me up before I play in the Men's singles draw of a local indoor badminton tournament. Take care.
Who are going to educate our children? Who are we going to follow? Who can we trust?
Reading this thread, among many other threads of the Internet, I'm looking for a balance. It is certainly not easy when there is such discord.
I'll be honest, I am @ lost. If someone is wrong, does it make someone else right? ...And vice versa.
A balance of human dignity in our civilization is in need, IMO
______
WBF & ASR, only two forums among a vast divided ocean of opposing views.
I remember when I got my first transistor AM mono radio; it was perfect. It was a present from Grand Ma for my sixth birthday.
And the adventure begun ...
We're all like children playing with our toys in the same sandbox with no direction home.
We missed the track
We jumped too fast
Like we're on crack
Stampeded by our past - Bob (me Bob)
_____
? Hawking: Actually, We Have 100 Years to Escape Earth ? Stephen Hawking is a scientist, yes and should we weight what he's saying?
1,000 years from now, ...it's alarming to contemplate life on Earth, very.
• http://www.dailymail.co.uk/sciencet...ephen-Hawking-says-leave-Earth-100-years.html
? http://www.express.co.uk/news/science/799719/Professor-Stephen-Hawking-end-of-the-world-brian-cox
This is all fresh news; check that last link's interview ? ...from express.co.uk/news/science
Reading this thread, among many other threads of the Internet, I'm looking for a balance. It is certainly not easy when there is such discord.
I'll be honest, I am @ lost. If someone is wrong, does it make someone else right? ...And vice versa.
A balance of human dignity in our civilization is in need, IMO
______
WBF & ASR, only two forums among a vast divided ocean of opposing views.
I remember when I got my first transistor AM mono radio; it was perfect. It was a present from Grand Ma for my sixth birthday.
And the adventure begun ...
We're all like children playing with our toys in the same sandbox with no direction home.
We missed the track
We jumped too fast
Like we're on crack
Stampeded by our past - Bob (me Bob)
_____
? Hawking: Actually, We Have 100 Years to Escape Earth ? Stephen Hawking is a scientist, yes and should we weight what he's saying?
1,000 years from now, ...it's alarming to contemplate life on Earth, very.
• http://www.dailymail.co.uk/sciencet...ephen-Hawking-says-leave-Earth-100-years.html
? http://www.express.co.uk/news/science/799719/Professor-Stephen-Hawking-end-of-the-world-brian-cox
This is all fresh news; check that last link's interview ? ...from express.co.uk/news/science
Last edited:
Hello Peter,
The question I ask myself is: What type of world do I want to live in, and more importantly, what type of world do I want my children to inherit?
The answer is pretty clear: One in which all forms of abuse of power* and negation of the other are called out for what they are, irrespective of whether they’re ideologically intrinsic to a dictatorship, government, political party, socio-cultural group, gender, religion, or indeed, an intellectually dishonest and myopic form of zealotry masquerading as “science”.
Thanks for your post,
853guy
*Knowledge being a form of power that can be misused.
Hey Morricab,
Thank you very much for that. As someone who's posted many links to neurobiological research in the past, I greatly appreciate your perspective.
Be well,
853guy
Hi Amir,
Taking a passage of words and reducing it down without altering the content is generally considered "editing". But I'm sure you know that already.
Your quote - word for word, yes. But Ron’s request at the end was this: “We should attempt to do the best we can do, and to try to remain as intellectually honest as possible, but perhaps we should acknowledge that we may be fooling ourselves about some of our listening conclusions.” (Bolding, mine). And yet, despite claiming in your most recent post to Morricab (Post #459):
amirm said:
…I can mostly only think of one person in this thread who specifically mentioned they were thinking of performing a sighted evaluation under non-controlled conditions when they said:
Sound familiar? Should be, you said it (Post #274). Want to discuss Ron’s request? It’s for intellectual honesty. There is absolutely nothing intellectually honest, cognitively unbiased, experimentally robust, nor scientifically credible about your post, hence why we're having this thoroughly enjoyable discussion here.
amirm said:
Dude, are you kidding? I said only eight posts ago that I “fully agree with Ron’s statement, and that yes, we do wish to discuss your explicit bias and deficiency of intellectual honesty” (Post #450). You really need trying to read beyond the first line if we're going to go steady. As it is, I'm already thinking of breaking it off with you.
By the way, there’s no “rest of you”. There’s no minimal group paradigm here, Amir. You can’t be in the outgroup - it’s intellectual honesty we’re discussing, and that applies to all of us. You included. And why? Because intellectual honesty is the topic, I'm very deliberately pursuing it here in my exchanges with you, and - as irony would have it - the only one to continue to double down on their lack of it is… you.
amirm said:
Bicker? My perspective is that I’m one of the few challenging your repeated efforts to avoid taking responsibility for what you write via deflection, re-framing and selective exposure; your explicit confirmation bias inherent in testing an object you believe won’t do “anything for the sound”; your lack of experimental robustness in performing a sighted evaluation, having paid for it yourself, while specifically ensuring it is not a product that could negatively impact any personally-derived profit from your high-end installation business; and your attempts to obviate yourself from evaluating your own bias and intellectual dishonesty because, of course, it's only “subjectivists” that suffer from intellectual dishonesty, right? I guess if you want to call that “bickering”, go ahead. I've already decided I'm asking Jonny to the dance.
Discuss other members? Only the ones posting on a thread about intellectual honesty who continue to display a complete lack of it. That seems fair, right?
Letting the message sink in? I went and heard a system recently and wrote a long post about it. I very specifically couched every one of my words in personalised, subjectivized framing, making no absolute statements whatsoever outside of my own perception. Funnily enough, Ron, the OP of this thread, went and called it “introspective” (which, if the dictionary is right, would be the opposite of intellectual dishonesty, wouldn't it?).(2)
It’s actually amazing to me that in reading your comments, the only one who seems to be struggling with “letting the message from Ron sink in" because it comes with “(a) dose of reality that is too painful” is... well, I'm sure you're getting the picture.
Amir, you can keep up the deflection, re-framing, compartmentalisation, hypocrisy, domain-specific ethnocentrism, ferocious dogmatism and self-blindness that’s become a defining trait of your posts all you like. As long as we’re discussing intellectual honesty, and you continue to display a lack of it, I’ll keep pointing it out.
And by the way, I did cheat on you with Marvin. He was a much better kisser than you, too.
853guy
(1) Just saw this on your audiobiasreview.com site. You said "Then you would be willing to participate in a blind test conducted by me where you win $5,000 if you can do this but if you fail, I only get my expenses of $500. Yes?" Wow. Impressive use of experimenter bias, allegiance bias, non-random sampling, and incentivising the subject’s bias toward participation. You are truly a robust practitioner of scientific credibility. Of course, now I need a shower. Ugh.
(2) You remember this right? http://www.whatsbestforum.com/showt...ined-quot-ears&p=433337&viewfull=1#post433337 When I took the ABX for the Soundblaster 1 Generation test at your behest? You do? And that I mentioned I wrote a long subjective post on the sound of an audio system and contextualised every comment as simply an opinion from one person within the finitude of their limited perspective? Tell me, Amir, given those two variables, do you think I'm attempting to be intellectually honest, or are you going to insist I listen to some more of Ethan's files of his muzak I am sure the devil has downloaded specifically to play in the elevator descending to hell?
Last edited:
- Status
Similar threads
