This is a valid observation and I understand what you are describing in terms of relative differences as I have also experienced such differences in the past and present when comparing USB to AES/EBU (with non--Taiko gear). The thing is that USB is highly reliant on the implementation and quality of the interfaces on both ends. I can't speak for Nagra as I have no personal experience with the brand but I do know that the USB interface on the Extreme is built to relay the best sound quality possible, both technically as well as musically. The large majority of users seem to agree but don't let that discourage you! I always advocate for anyone to use the connection method that works best according to their own ears. If it sounds great, don't worry about it and enjoy the music!
but sourcing the 'best usb' is only 1/2 the equation and the result depends on the receiving end as well.
it is certainly possible that some dacs sound better with their aes/ebu input than with a usb input.
consider, for example, Taiko's own totaldac d1-12mk2. it consists of multiple parts and while the extreme feeds that package of boxes via usb, the totaldac takes that usb, reclocks it and feeds the DAC section via aes/ebu. so it is a matter of choice how one considers the system to be partitioned. the d1-12mk2 can be considered as a 'reclocker' followed by a 'dac section'
[extreme] usb==>[reclocker + dac section]
or
[extreme usb==>reclocker] aes/ebu ==> [dac section]
one d1-12 user here actually uses
[extreme usb==>reclocker==>reclocker] aes/ebu ==> [dac section]
and finds the second reclocker (aes/ebu in and out) to be significant.
whether or not the reclocker is considered to be part of the DAC or not is really semantics.
you would have to bypass the reclocker and feed the extreme usb output directly into the DAC section of the totaldac to make
a fair usb vs. aes/ebu comparison.
of course, YMMV.
enjoy the music, i am! (well, in between those $@* gaps)