For me, the main question is that your formulation is exclusive - if a speaker does not fulfill this criteria you immediately consider that it should sound inferior. I am not yet persuaded of it.
Go find a pair of Martin Logan hybrids. put on some music, I don't even care if you're totally unfamiliar with it. Sit down in front of the speakers and listen for a minute until you get a feel for the tonality. Now stand up, slowly, paying attention to that tonality as you do. Depending on which ML hybrids you're listening to and how their tilt is set, that may do it by itself. If not, now shuffle slowly to one side, in a gentle arc that takes you closer to the front of the room, to the side of one speaker as you change position, still facing the speakers. Pay attention to what happens to the tonality, not the stereo balance, but tonality, as you move off axis.
You've just demonstrated to yourself how completely different the amplitude response of the first reflections will be compared to the direct sound of the speakers when it comes back and mixes with that direct sound a split second later,
regardless of the material they reflect off of. Think, for a moment, about the effect that will have on the overall tonallity of the playback. If you remain unconvinced of the importance of smooth, even off-axis response, I'm afraid you'll stay there. But consider this:
The speaker manufacturer cannot anticipate what material the off-axis sound will be reflecting off of. He cannot anticipate how far away that material will be or when those first reflections will mix with the direct sound. All he can do is minimize the distortion that comes out of his product, or minimize the volume (maximizing the distortion) of the distorted, off-axis sound relative to the more accurate, on-axis sound, and call it "controlled directivity." I think minimizing the distortion, and the amplitued change, is the better strategy and the listeners in the Harman test agreed.
If you performed the listening test I just described, I think you would come over into the light where speakers off-axis sound very much like they do out front.
By the way, I'm just picking on the MLs because they are the example in this thread and are particularly bad in this regard. I also find it fairly disappointing that they are dipoles,
that are meant to use reflected sound by design, and have this problem. But the bottom line is you can find this problem in lots of box speakers and most horns. Klipsch, at all price levels have it in spades.
Tim