There are definitely a lot of great mix engineers out there. It's not that.
My short answer: Real music - like real life - has to have a lot of moment-to-moment variation in volume. That's dynamic range: the difference between the quiet instant and the loud moment (like a drum hit). Great mixes have a lot of dynamic range.
But here's the tension: What you perceive as "volume" or "loudness" isn't the volume of those quick, transient moments (like a drum hit). Instead, it's sort of the *average* across time of how much power/energy the music has.
So if you were scrolling through Apple Music or Spotify, and one song kept a lot more dynamic range than the rest, it would instantly sound really weak/soft/quiet compared to everything else.
An artist can make his song louder by giving *all* of the moments more volume/energy. But when you do that, it's at the expense of dynamic range. And here's the important thing: music without dynamic range takes on a harsh, constant, buzz. And squashing the dynamic range MUST introduce some level of distortion, where the sound waves get clipped or squared-off instead of their natural sine shape. Music without dynamic range is fatiguing... like a chain saw. This phenomenon is exactly why commercials are both 1) louder, and 2) sonically obnoxious and annoying.
Anyway, it's been hard to stop. Since dynamic range compression (a big part of mastering) will change the way something sounds, mix engineers will preemptively squash their mixes before the mastering engineers even get it. Mastering engineers complain that they can't "master" something that already has no dynamic range. Artists who get their mix back and then audition it against all the modern music out there will complain that their song is way too quiet and needs to be louder.
The problem has saturated pretty much every form of music consumption with the exception of vinyl. Streaming services use algorithms to level off everyone's music to the same loudness, so even artists who fight to keep dynamic range in their mixes will get their shit SQUASHED in the final step of consumption, and there's nothing they can do about it. Terrestrial radio is the worst. Radio stations want to be LOUDER than the next one on the dial, so they add insane amounts of leveling that sounds absolutely awful.
I would argue that all of this is one of the major reasons why music has fallen off the map of importance. It's consumed in ways (no dynamic range) where it has no chance to deliver a powerful, transcendent experience with an emotional punch.
Sorry for the long response. I'm sure the engineers can take issue with my use of words like "power, volume, level, loudness" - those words each have very precise scientific definitions that aren't interchangeable... but you get the idea.
This quick video gives a pretty good visual representation: