Hey, thanks for using the tool! I am about to release a new update that is changing the way scores are saved internally, so I'll look into fixing that issue. The update will be live soon, so stay tuned!
Edit: Removing items seems to work just fine if I leave the leaderboard name unchanged. Looking forward to the update, and thanks again!
The update was released, and your described problem shall no longer be an issue!