The question is how hard is it to filter out and is the cure worse than the disease? Meaning will the filter harm the signal more than not using a filter? I guess you would have to experiment yourself to know for sure.
I haven't built one myself but have designed a passive filter which I've simulated and appears to do the job nicely on the ESS Sabre. Also a passive filter at the output of my NOS DAC does more good than harm to my ears. But yes the ultimate test is by listening.