In most amplifiers, the feedback is applied to a tube or transistor somewhere near the input of the amp. That tube or transistor isn't really linear, so the feedback signal is distorted before it can even do its job!
As a result IMD and higher ordered harmonics are generated and the ear perceives both as brightness and harshness. Class D allows you to overcome this problem, so the feedback isn't generating these distortions in the same way. In our case (I can't speak for other class D amps as there is quite a lot of variation in their designs!) the sources of distortion tend to cause lower ordered harmonics, much like in a tube amplifier, but at a much lower level.