You can't avoid being cursed. You need to find a a way to get rid of the cursed item.
Though first you might need to do something else to access the place to do that.
