I also have this issue! Well described by OP. I have very few playlists personally (I mostly listen to albums). I'm running from the .pacman on Arch Linux with Hyprland (e.g. Wayland). AMD GPU.
