Yes it makes an audible difference. Enough for me that it's the only way I will ever listen to DSD now with passive speakers. Unless going active where the DSP on the DSD is a must. Then you have to decide what the lesser of the 2 compromises are, passive crossovers, or multibit SDM DSP. In this situation, the lesser of the 2 comprimises will likely be the multibit SDM. After all it's still the finest way possible to do DSP. Far beyond what's possible on the best Sharc DSP chips.
As for the money spent on the analog source, I find it far more cost effective to buy master tape DSD rips, guys like Bruce did from NativeDSD and Analogue sounds, and get my vinyl rips for free from my buddy with the $100K plus rig.
Try for yourself with HQplayer. When you play a native DSD track and DirectSDM isn't checked, it gets processed through the multibit SDM engine. When checked, it bypasses the multibit SDM engine. In most SDM DAC chips, you can't bypass this engine, but you can on some, but only with DSD. Your GG doesn't have the SDM/SRC processing in it. the DSD section is just a simple low pass filter. So if you send it DSD trough HQplayer with SDMdirect enabled, it doesn't go through any multibit processing. This is the purest path possible. In this scenario the only SRC/SDM the audio sees is the initial 1 bit from the original ADC/DAW. So what this shows is there's no substitute for using the best 1 bit DSD SDM/SRC algorithms the first time around. This can only be done with the original ADC/DAW package. This is another reason DSD is better that is overlooked by the PCM advocates. You can never have a signal path this pure from start to finish with PCM, unless you use an inferior R2R DAC. But that technology is outdated now. The last R2R chip the BB 1704 was unveiled back in 1998. The days of SDM tech, not living up to R2R standards are in the past now. The advanced SDM algorithms in DAW like Pyramix and HQplayer will put any R2R to shame.