??? I have taken the test twice so what I shared was my experience which was identical to what the formal listening tests consisted of. You are not given any instructions on how to evaluate one loudspeaker against the other. It is up to you to do that however you choose. When you sit there, you are immediately presented with loudspeakers which tonally sound hugely different. It is that difference that you wind up judging...
You hear all the samples being tested, say 3 loudspeakers, and you give them score 1 to 10. No different than if you walked into a showroom and were choosing among them. There is no fourth loudspeaker that is the reference.
Discriminating between two variables to identify difference is vastly different from discriminating between two variables in which we are required to apply judgement (i.e. preference).
The former requires the removal of bias, while the latter is dependent upon it. Kahneman & Tversky; Keltner & Lowenstien; Loewenstien & Lerner; Bachara, Damsaio & Damsio have all published in relation to the primacy of emotion on our ability to make decisions requiring judgement.
Discerning a difference does not require any emotional facility. In fact, as ABX testing has repeatedly shown, emotion is an inhibitor to discrimination, and the purpose of ABX testing is simply to establish a statistically significant level of confidence in discrimination - not, it should be noted, an empirically significant one in which the subject’s intentionality is factored in (i.e. they could be guessing/lying and pass).
Bachara, Damasio & Damasio have shown that subjects with deficiencies in their pre-frontal cortex (the area of the brain that processes emotion) have difficulty making decisions requiring judgement, even down to simple tasks such as what clothes to dress in. (This was of particular interest to me as I spent two years as the co-ordinator of a youth/young adult drug rehab where delaying gratification and the ability to discern future consequences was imperative to their chances of long term recovery.)
What’s more, Bachara et al believe the amygdala (the source of processing memory, decision making and emotion) is using emotion to modulate memory and bias decisions,
of which there is no separation between the two functions.
Therefore, any test designed to suppress emotional response from the subject will lead not only to an impairment in judgement (biases) but an impairment in memory access/storage.
This is problematic in light of the fact that listening to music (live, or prerecorded) should (hopefully) invoke an emotional response in the subject whereby the bioregulatory process creates somatic markers associated with memory, and those memory markers are used by the brain for future decision making and consequence evaluation. If the notion of emotional response is suppressed through the testing protocol in the testing of components whose function it is to deliver emotion/intention recorded as sound then it allows a lesser notion to gain preference in the hierarchy of decision making.
Harman would like us to believe this is on-axis/off-axis frequency response curves. They use several speakers whose measurements are taken using steady-state signals to compile a spatial average. They then play music (amplitude and pitch over time) and ask for subjective preference ratings where each speaker is rated out of ten. They combine the objective and subjective data, find a correlation between measurements and preference and -
Viola! Audio science tells us all we need to know about choosing our speakers.
But does it? No, it tells us that people will rate a speaker highly in a comparative environment where the speakers under evaluation
vary wildly in their spatial averaging creating differences to be detected rather than a judgement in which the consequences must be considered. Well, duh. Rating differences is not the same as making decisions because decisions involve emotion and consequence and there is no emotional facility required in rating a speaker out of 10 based on comparative tonal differentiation. Just detection of difference and giving the degree of difference a number.
Is Harman taking three or four speakers that have the
same or similar spatial averaging and asking for subjective evaluations? No. If it did, the subject would be forced to evaluate the speakers based on other, less measurable, potentially ephemeral and wholly subjective variables, like for instance, how it made them
feel, which can be measured via skin conductance response, heart rate, respiratory frequency, skin temperature and facial EMG and - shock, horror - possibly lead to competing manufacturer’s products being rated more highly.
What then? I’d imagine we would see exactly what we see on this forum - the vast majority of us don’t have Harman speakers.
Did we buy them with our eyes, then? Our egos? Our lack of erectile functioning? Very possibly so. We can’t exclude those biases from our decision making processes.
But that’s not to say that Harman’s testing is not problematic, and further investigation from independent researchers wouldn't be valuable.