I think it's dac, source, usb cable, spdif cable, software, network, storage medium, upsampling, format-conversion, usb reclocker and usb converter-specific!!
Seriously, too many variables to be able to generalize. I only have experience with one dac so whatever I say will cover only the smallest of possibilities.
I can say my toslink doesn't sound quite as good as my coax, but it's pretty easy to switch between the two.
Coax vs. USB requires me to reboot the Pi which takes about 30 seconds. For now I don't hear usb sounding any worse, and probably better.
The new breed of USB reclockers (Intona, Remedy, Regen etc.) are sposed to take USB to another level although I haven't heard any of them yet.
-Mike