The issue is the choice of capacitor. If its got problems, bypassing it might actually sound better. IME its better to just give it your best shot with the best cap you can. This usually means Teflon as it has the best characteristics at audio frequencies. Polystyrene can be pretty good too- if you can get some made with a properly thin film. To my understanding the Germans made the best film but got out of it a good 20 years ago or so leaving just the Chinese film which is about 6x thicker. That might have changed in the last 20 years- I've not followed it.I see examples were done in commercial amps and preamps. I tested them in my amps and found that the combination sounds better with bypass as it gets the best in clarity and bass slam. Of course, if I can get away with one cap, that would be ideal and cheaper.
Oil-filled parts can also sound very nice but run the risk of electrical leakage which can wreck havoc no matter where they are in the amplification train. The only part I've seen that doesn't have this problem is the V-Cap ODAM but there may be others. The Teflon has better performance though.
I agree with @sparkie 's assement about resistor noise. You have a better shot if you simply have a single resistor with a much higher Wattage value than needed (that was how they got low noise in the old days) or by using a wirewound, since its inductance is likely not a problem at audio frequencies.