It was a Unity bug and they're fixing it, the workaround that should make it work good enough is changing the sprite type from single to multiple.
