That's right, that error occurs when you try to return to the village at that entrance, but I don't know if the error occurs during the day or at other times as well
