I personally think what you have now fits great for lofi hip hop though! I listen to a lot of the genre (maybe my most listened to) and my main issues with the genre is that many tracks often lack memorable melodies (not that there's anything wrong lack of melody though, mostly my subjective opinion), and I think the sax melody (along with other things) especially help with differentiating your track while also tying it nicely into the jazz roots of the genre!