Just out of curiosity and to eliminate the obvious, that hasn’t been mentioned. There are no external sources of interference nearby are there?
Active central heating? USB cables running close to any power cables that could pick up interference?

There is the possibility of external interference, but that’s simple enough to identify by physically ‘breaking’ the earth loop somewhere (by removing the USB connection, and then listening to the audio outputs for example). If the noise disappears… then you have an earth loop problem. It’s very common in this kind of situation.

The ‘galvanic isolation’ solution (audio transformers or opto-isolators) will cure it completely.....you ideally need to have just a single ‘earth’ connection in your setup.

You may be able to reduce the problem by varying the earth routing, and indeeed moving the various power supply connections near to each other. But at best you’ll only reduce it.

What you do need to resist is the temptation to remove any ‘safety’ earths (that is the 3rd pin connections that may be fitted to some types of equipment).
The internet is full of comments like “..don’t worry about it… I’ve been doing it for years and I’m still here ... it’ll be fine...”

Those comments are just simply wrong, and can lead to very dangerous situations.... In the worst case scenario, removing safety earths can kill you!..

Do NOT remove safety earths. Use ‘galvanic isolation’.....

I’m still waiting to test out one of the cheapest of those options , the Behringer HD400 ... most reviews look good, so it should work well.. and it is cheap!

One unit I can already recommend is the ART DTI dual transformer isolator.
That’s a super quality unit at a reasonable price. Lots of connection options (in and out) and an excellent spec. Works really well in this situation.
EDIT:  I have now tried out the Behringer HD400 and can confirm it performs very well.. especially at the price!
It has a slightly greater insertion loss (about 1dB) over the ART DTI unit I mentioned above, with the same ‘line level’, into a nominal 10K impedance load. (THE ART DTI only has a 0.5dB insertion loss at that impedance). The frequency response is pretty much flat between 20 HZ and 20KHz..  that’s better than I was expecting.
It doesn’t have all the connection options of the more expensive devices, but it’s a pretty cheap and effective way of ‘breaking’ an earth loop in the audio path…