This is caused by the sprites using RGB instead of Indexed colour mode. E.g. Aseprite can adjust that.
That did the trick! Everything was made in Aseprite so that should be an easy fix lol, thanks for the help!
