Found the issue! It had to do with how I was building the keys to reference my generated water autotiles :/. Hence why it was showing up when entering water (or when the game restarted and it was trying to reference the player, as if they were on water).
