Perhaps in the audio world, this 'magic' simply means something that has not been properly measured or quantified.
Being a scientist sometimes doesn't help in that it often creates questions that have no way of being answered. I've read white papers and theses where the questions bumped up against the instruments' inability to measure the proper data.
For instance, most everyone agrees that the human threshold for hearing is @20kHz yet I've read that if a tone is played within that threshold and an even or odd-order harmonic of that tone, well above the threshold is added, we can perceive a difference. If played by itself the upper harmonic tone can not be heard yet when played with the lower tone, a difference is detected between the pair and the original single tone. The author attempted to explain that while we can not hear the upper harmonic, we can perceive its effect on the original tone's vibration in our ear. The problem was that he had to leave the numbers behind in his explanation because there were no instruments that could measure the nuanced effect of these harmonics on the human ear. Without hard numbers, his conclusion was considered more conjecture than theory...yet the phenomena itself was properly evidenced and documented therefore, it wasn't disputed - just the cause.
Along the same vein, you mentioned that many folks enjoy listening to music through tube amps even though the amps tend to have higher distortion numbers than their SS counterparts. Having had many SS and tubed amps run through my system over the years, I've come to prefer the tube sound - particularly SET, in spite of its inherently high distortion compared to the other topologies. Aside from the other factors, I believe I can tolerate, even enjoy higher levels of even-order harmonic distortion associated with SET amps while the odd-order distortion eventually becomes tiresome and grates on me a bit, even at much lower levels.
I'm sure its mostly a matter of personal preference but I equate the odd/even distortions with bitter and sugary taste. I much prefer the taste of sugar to say, bitter coffee and will tolerate an overly sweet dish to one that has even a hint of bitterness. Of course, other folks may have the opposite preference..its all good.