Just to share i find conect my Roon core NAS to 100Mb port in addition to my dCS Bartok sounds way better than the 1Gb port, and also rock stable without any drop.
Adding another switch upstream make the background more black, and I'm surprised by connecting upper stream by copper ethernet cable sounds better than fiber FMC in my case.
The sound signature of upper stream ethernet cable still noticeably propagate to down stream.
1 G will for sure sound better to you.
I was in simmilar place 5 years ago.
It looks like the noise is Very significant problem in your system.
Start with LPS use for all devices and get all power switching supplys out of chain.
Get the 2,4 wifi and turn off 5 ghz wifi In your rouer /access point
or better get your wifi totally off and use wire ehernet To controll your playback.
wifi is extreme noisy thing .
Check your power and test balanced power trafo solutions like Keces to see if this helps.
Add the ethernet isolating device like PinkFaun Isolator .
Make sure all your ehernet cables are with good amount of isolation.
Sablon Audio makes excellent ethernet cables.
After that you will see that using fibre also helps .
use it between the 2 swiches and get to your server with ethernet cable.
you need to use specyfic SFPs.
Finisar FTLF1324P2BTL Is an example of that.
There are also some good Planet tech SFPs as well as Startech.