Comparative Listening Tests

[microstrip]

(...) The whole exercise was to assist Al in his cable decision by offering additional listening impressions, biased, unbiased, sighted or unsighted, from a friend who knows his system pretty well. It was not a rigorous investigation, but a casual effort to learn and confirm impressions, flawed as it was. I think it helped Al, and I did learn something. It involved minimal effort and was accomplished with what he had on hand and available to him. It may not have been ideal, but it got him a bit further along in his quest to improve his system. In that sense, it is all good, as the younger generation is fond of saying.

Unfortunately this type of "flawed" test, reported in good faith as information to a forum audience, is being used to fuel anti-high end audiophile arguments in a noisy environment.

The main question the objectivist people raise is on the audibility of cable differences - it is why the discourse always ends in debating techniques used in scientific studies to research thresholds of audibility.

Audiophiles know and assume from experience that cables in stereo high-end systems can sound different and must accept that they will not find scientific proof of it the scientific literature, at best they will find just statements and opinions of respected people.

Once we accept that cables can sound different and differences are significant we will be debating techniques to establish preference, not audibility, and our findings on them , deeply mixed with our preferences. Reports on such tests should be considered as friendly and informative opinions. Having read since long about Al and Peter preferences and systems, I enjoyed the OP.

IMHO the high-end people has very little to learn considering subjective aspects from the people who believe everything sounds the same and all else is placebo effect and protect themselves behind the refereed and published audio science. I can not understand why we are cyclically fighting with them in WBF - they do not want to learn form us and they can not teach us anything, as we can not use adequately their methods. As always, IMHO, YMMV.

 

[amirm]

Amir, I don't understand this. I can think of at least these several possible outcomes:

1. After repeated listening, the file sounds the same to him, he reports that it sounds the same to him. He is correct and he is honest.
2. After repeated listening, the file sounds the same to him, he reports that it sounds the same to him. He is incorrect and he is honest.
3. After repeated listening, the file sounds the same to him, he reports that it sounds different to him. He is correct and he is dishonest.
4. After repeated listening, the file sounds the same to him, he reports that it sounds different to him. He is incorrect and he is dishonest.
5. After repeated listening, the file sounds different to him, he reports that it sounds different to him. He is correct and he is honest.
6. After repeated listening, the file sounds different to him, he reports that it sounds different to him. He is incorrect and he is honest.
7. After repeated listening, the file sounds different to him, he reports that it sounds the same to him. He is correct and he is dishonest.
8. After repeated listening, the file sounds different to him, he reports that it sounds the same to him. He is incorrect and he is dishonest.

Ah, no. Hmmm. I keep starting my posts that. Sorry .

Being incorrect doesn't make you dishonest any more than being correct makes you honest. What makes you dishonest is if you run the test of identical files, hear differences, but report here that they all sounded the same.

One might assume that because the file is the same, that it should always sound the same. Trained listeners like yourself, knowing that the file is the same ahead of time, may repeatedly hear it as the same, either because of their training or their bias caused by knowing that it is the same file.

Again, no. I ran the test prior to posting it and the outcome has been the same for me as always: I hear them differently in different runs. Indeed this is the explanation I get of why identical stimulus has sounded different to me. It simply can due to variability of our memory and perception. Training does not help you other than accepting the outcome as something that happens because we are human.

But an untrained listener may start to notice different things because his mind may wander, loose focus and concentration, and then may either hear it as the same, because he has been told it is the same, or as different because of his wandering sense of perception. And to complicate matters, he may report it honestly or dishonestly, because he is human. I don't see how you could know with certitude which is which and arrive at a conclusion about the listeners honesty?

The fact that you write that his reporting would tell you a lot about his honesty implies to me that you are inherently biased because you presume to know which of the possible outcomes I listed is the correct one.

I see where you are going. I was obtuse on purpose with John. I wanted him to run the test. If he ran it and reported that he is not subject to all the factors that you well list above, and that it always sounded the same, then in my book he would be dishonest. We can't deny what makes us human as you say.

 

[PeterA]

I didn't say you were given one test. I said one test was conducted where Al knew the answer with certainty and you did not. Remember we were talking about one taking an exam where your instructor knows objectively what the right answer is. This was the case with Al. He had you give one set of impressions sighted, then he gave you a test where the knowledge of which cable was which was hidden from you. You did not manage to give the same response as you did in sighted test.

Unfortunately for you all , we get to rely on that negative outcome and ignore any (single) positive outcome. For any positive outcome it needs to rise above someone guessing. Getting one answer right out of one trial means the probability that you knew the answer is just 50%. If you ran the test again, then it would rise up to 75% if you got the right answer again. Standard practice requires 95% confidence. To get there you need 7 or 8 trials (from memory).

I can't tell you how many times I have run a blind ABX test where I got three or so answers right in a row but then could not get them right from there on with the result being that I simply guessed right the first few times. It is not that hard to guess which way a coin lands three times in a row. But seven or eight times, gets really hard.

You and Al did great here experimenting. Would be great if you get together and repeat in this manner. First listen sighted, with your own material or whatever you think is revealing to you. Do as much as this as you like. It is actually encouraged in testing protocols (it is called training). Then the formal part of the test starts and all you want to do is identify which cable is which. Don't worry about characterizing them. If you can't tell them apart reliably, it doesn't matter what you think of their sound. Do this 7 or 8 times. If you get them right every time, then you are done. Unfortunately if you miss one or two times, then you have to keep going. I have written an article on this here: http://audiosciencereview.com/forum/index.php?threads/statistics-of-abx-testing.170/

Anyway, let me tell you much credit also for participating in this type of testing. Much appreciated.

Actually, the knowledge of which cable was which was hidden from me for each of the four iterations. I knew the identity of the two cables but I was never aware or told which cable was playing in the system until after the completion of the test. In other words, at no time during the testing was I told ahead of time which cable I was hearing. I knew only that there were two cables.

 

[amirm]

Unfortunately this type of "flawed" test, reported in good faith as information to a forum audience, is being used to fuel anti-audiophile arguments in a noisy environment.

Anti-audiophile? Is that what you call any audiophile that dares to listen to audio science? The fact that they love music, invest in good audio gear doesn't qualify them as audiophiles???

No one has jumped the gun here as you fear. We have said little about cables. We have praised the people who did the test, and a cautionary note was given and rightfully so, by Ron that our normal audiophile comparisons may not generate reliable results. That is it.

Creating drama, recruiting the troops under false pretense above to go to war just adds noise and animosity here. We have plenty of it already so no need to pile on it this way.

 

[jkeny]

No, ABC/hr allows quality rating which is actually close to what Peter was doing. He definitely was not doing an ABX test which only identifies if the two samples are different.

Amir, as you have been told before, you should talk to & absorb what the experts in the field tell you - in this case Soundandmotion is an expert (in comparison to you) in perceptual testing.
I'm not going to get into your endless denial & deflection on this as you are patently wrong, yet again & it's a waste of time trying to educate you as you are beyond learning.

 

[jkeny]

I see where you are going. I was obtuse on purpose with John. I wanted him to run the test. If he ran it and reported that he is not subject to all the factors that you well list above, and that it always sounded the same, then in my book he would be dishonest. We can't deny what makes us human as you say.

Amir, I know more about psychoacoustics than the simplistic understandings you demonstrate. Your continued intellectual dishonesty is your major barrier to learning anything but it's interesting to see the defense techniques used by a technician's mind & see how they are unwilling to break through their limited worldview.

 

[Steve williams]

Amir, as you have been told before, you should talk to & absorb what the experts in the field tell you - in this case Soundandmotion is an expert (in comparison to you) in perceptual testing.
I'm not going to get into your endless denial & deflection on this as you are patently wrong, yet again & it's a waste of time trying to educate you as you are beyond learning.

I agree about Sond and Motion and would love to hear SAM's thoughts who BTW already stated this was an ABX test

 

[Gregadd]

Once again we compare getting the wrong answer to guessing.

 

[KlausR.]

... what the experts in the field tell you - in this case Soundandmotion is an expert (in comparison to you) in perceptual testing.

Well, it can't be jj, so would it be possible to know his name in real life?

 

[microstrip]

Anti-audiophile? Is that what you call any audiophile that dares to listen to audio science? The fact that they love music, invest in good audio gear doesn't qualify them as audiophiles???

No one has jumped the gun here as you fear. We have said little about cables. We have praised the people who did the test, and a cautionary note was given and rightfully so, by Ron that our normal audiophile comparisons may not generate reliable results. That is it.

Creating drama, recruiting the troops under false pretense above to go to war just adds noise and animosity here. We have plenty of it already so no need to pile on it this way.

I wrote "anti audiophile arguments" in a several sentence post concerning high-end audio, as usually you quote a small part distorting it, ignoring the main arguments. But no problem, I will edit to "anti-high end audiophile arguments".

 

[jkeny]

Well, it can't be jj, so would it be possible to know his name in real life?

You are looking for an authority figure to hang your hat on rather than judging the expertise demonstrated in his posts on perceptual testing?

Typical technician's mind - unable to think for oneself- as I said the techniques for self-delusion & self-defense are fascinating

 

[Steve williams]

Once again I'm not Mike. Mike actually told me that what your sidekick wrote in red in the jitter thread on ASR is a lie. He has a large collection of screenshots taken before and after edits you guys made on his own posts on threads in the past. If you want he can email me them to share here? All of the edits to his posts were done after he was blocked from posting on the threads. And there wasn't even any warning to others he was blocked from the threads. It was made to look like he said these things himself, then just decided to stop posting. That's extremely dirty tactics in my book.

Why am I not surprised

 

[SoundAndMotion]

Difference paired comparison A vs B: you present all possible combinations AA, AB, BA, BA and ask whether they are same or different. Obviously you should be left in the dark about what the respective pair includes.

No, ABC/hr allows quality rating which is actually close to what Peter was doing. He definitely was not doing an ABX test which only identifies if the two samples are different.

ABX is not a paired comparison test. In ABX, you are presented with A (once in the original AT&T version, or as often as you like in the newer version), and you know it is A. Repeat with B. Go back and forth if you like (newer version only). A and B are both references. When ready, you get X, knowing it is A or B but not knowing which one, and you must answer which one it is. You are right Amir that ABC/hr is a quality rating and you are right that Peter noted a preference. But the answer he gave when presented with the unknown X, was just A or B. That test was one trial of an ABX.

 

[SoundAndMotion]

Well, it can't be jj, so would it be possible to know his name in real life?

Klaus, I very much appreciate the fact that there are pros and cons to either providing my name or remaining anonymous. I have chosen anonymity, knowing there are associated cons. The biggest is that I can claim any background without providing proof, and therefore why should anyone trust what I say. My answer is don't trust what I say, or pull a Reagan and "trust but verify". I'm happy to provide references to back up what I say, or at least point in the right direction.
In his enthusiasm, John revealed something I told him in PMs a while back (no worries John, but remember I have chosen anonymity).
Klaus, apologies for being cagey. I hope you understand.

p.s. I don't accept the label "expert". It implies my knowledge is superior to others' knowledge. It is not. But as I mentioned, I can cite references, if needed.

 

[morricab]

Hi Klaus,
I'm sorry. This is wrong.

Yes, this would presumably be a single blind test (I am assuming the cable switcher knows the cables).

 

[stehno]

Originally Posted by stehno
DaveC, your top-end manufacturer is right. At least to a point.

For example. No matter how revealing a system and how much of the recording hall's ambient information remains audible at the speakers, a room too narrow and/or too short will limit the speakers' potential dispersion of sound into the room. Speakers require at least a minimum amount of phsyical breathing room in the listening room for the most believable soundstage presentation.

The same is even more true for attaining a superior bass response and speaker placement. I'm not aware of any amount of information embedded in the recording that remains audible at the speaker that can compensate or overcome a speaker's less than ideal placement for producing a potentially fabulously musical bass response."​

Unless the speaker has controlled directivity, then it's moot. The room can be almost as small as you need the gear to fit in. The confusion comes when the mind can't understand what it hears compared to sees.

....

In theory, what you say makes sense. But only in theory.

If what you claim is true, then regardless of how revealing a pb system may be, you or anybody else with speakers already placed 3ft or more out from the front wall and with ears to hear should hear zero difference in the soundstage’s perspective size when the speakers are pushed right up against the front wall.

This should be a well known fact to the seasoned enthusiast that the soundstage will collapse toward a flat 2-D presentation to good degree or another the closer a speaker is placed toward the front wall. Hence, demonstrating that the perspective size of a soundstage is dependent on speaker/listening room limitations.

This real world speaker / listening room limitation and placement is equally impacting on the quality of a full-range speaker’s level of musical bass reproduction.

For the seasoned enthusiast, there should be little to no confusion whatsoever between what the mind can’t understand and what it hears compared to what it sees. It only has to do with real physical limitations of reproducing a music presentation.

For example. My listening room is 12' wide x 21' deep x 8' high and though I’m content with the soundstage perspective's size for most recordings, I know it can be significantly wider and deeper. Several years ago, I reconfigured a friend’s system eliminated several of his components and placed the remaining 2 components (a CDP and int. amp) on one of my racks. His room was maybe 17’ wide by 25’ deep x 8’ high. When I came back 2 months later to listen, his soundstage was what seemed to be at 2 times wider than my own and for the first time ever I heard a soundstage so deep it literally sounded like it was coming from well out into his front yard. Pretty incredibile actually.

At the very least I can attest from even just that one experience that I wasn’t confused one iota by what I heard vs what I saw. Especially when takiing into consideration his listening room was on the 2nd story of his house.

 

[jkeny]

Klaus, I very much appreciate the fact that there are pros and cons to either providing my name or remaining anonymous. I have chosen anonymity, knowing there are associated cons. The biggest is that I can claim any background without providing proof, and therefore why should anyone trust what I say. My answer is don't trust what I say, or pull a Reagan and "trust but verify". I'm happy to provide references to back up what I say, or at least point in the right direction.
In his enthusiasm, John revealed something I told him in PMs a while back (no worries John, but remember I have chosen anonymity).
Klaus, apologies for being cagey. I hope you understand.

p.s. I don't accept the label "expert". It implies my knowledge is superior to others' knowledge. It is not. But as I mentioned, I can cite references, if needed.

I was only referring to the expertise in perceptual testing that you demonstrate in your posts here & elsewhere, nothing else - the proof of your knowledge in this area is demonstrably evident in your posts.

Sorry if this caused you any trepidation!

 

[DaveyF]

Amir, as you have been told before, you should talk to & absorb what the experts in the field tell you - in this case Soundandmotion is an expert (in comparison to you) in perceptual testing.
I'm not going to get into your endless denial & deflection on this as you are patently wrong, yet again & it's a waste of time trying to educate you as you are beyond learning.

At the ASR forum, it was not more than a few minutes before I was ridiculed for hearing any difference with the Shakti's in my system. None...I repeat none, of the members over there had ever heard of this device, never mind having actually heard it in their systems! That did not stop them from bashing the product and then later me. So, I guess if you are talking of 'blinkered' minds, then Amir's whole forum membership might also qualify. As you say, he---and they, are all beyond learning, because unless it is has been AB'ed or ABX'ed or some other pointless and time wasting test that they live and die by, it couldn't possibly make any difference to the sound. I think Steve W is absolutely correct, why bother talking or posting responses to these guys...life's too short, IMHO.

 

[jkeny]

Once again I'm not Mike. Mike actually told me that what your sidekick wrote in red in the jitter thread on ASR is a lie. He has a large collection of screenshots taken before and after edits you guys made on his own posts on threads in the past. If you want he can email me them to share here? All of the edits to his posts were done after he was blocked from posting on the threads. And there wasn't even any warning to others he was blocked from the threads. It was made to look like he said these things himself, then just decided to stop posting. That's extremely dirty tactics in my book.

Just one example would do!

 

[jkeny]

The technicians & clueless on ASR continue to show their idiocy (no wonder Amir is considered a god over there). In the "thread "Close-in jitter" (a misuse of the term, jitter), watchnerd thinks because close-in phase noise is increased by blowing on the oscillator that it's a flaw - even DonH56 hasn't a clue when it comes to this - his experience is in HF oscillators, not audio so he has no idea about close-in phase noise (< 100Hz)

The fact is that oscillator close-in phase noise is affected by all the factors mentioned - temp, vibration - it's the physical laws of nature. That might be considered a flaw in the technician's universe that they all inhabit.