Jack, it depends on the caps and the signal levels, especially in power amps. Large electrolytics have fairly high series resistance (plus self-inductance) and thus are "slow" (though many if not most cut off above the audio band) so we (designers) add smaller, "faster" caps. How many, how good, and what size is a function of cost and how long they need to provide charge until the bigger caps "catch up". So, with no bypassing, transients will quickly get "snubbed" since big caps can't provide fast charge. With some smaller caps added, transients will fare much better but eventually charge must come from the larger caps (and then the wall).
I guess that's a long way of saying "it matters, but it's complicated".
Good question! - Don