Okay. As suggested in another post I mostly look for three things: poor low level detail, interference effects, and inability to reproduce high SPLs. I consider these all distortions, because the end result is audible sound that is not faithful to what was recorded, ie. distortion.
Poor low level detail is deduced by using a recording that I know from previous playback, on a better performing system, has musical information that, literally, I can't hear on the offending gear.
Interference comes from many sources: I use the process of elimination, by completely shutting down anything in the vicinity, and ensuring that cross-interference between components is minimised, and then listen for quality changes. By reversing the shutdown, etc, one can pinpoint where the weaknesses are, and get ideas for minimising them.
High SPL capability is fairly obvious, just put on the right material and listen for the quality starting to compromise at some volume.
How to eliminate the issues depends on the underlying causes: it may be insufficient power supplies, poor filtering of the mains, poor internal construction of the system components - I would look at the problem in the same way a service technician does who is given an audio box to fault find, troubleshoot, having been told simply "that it's not working well".