You have to be a true succubus or incubus to be able to return. If you refused the Queen, there is an option to change that in the tower
