A little advice. Enter the tower when your maximum achieved zone is at least 30-50 levels higher than the requirements for the first floor of the tower.
You're a lot farther along than I am (I left my 'time machine' back in Trimps) , but at the 'just recently unlocked towers' point I wouldn't advocate waiting that long.
Even a few of a new type of heirloom make a massive difference, and there are some to be found in the towers. Add in the mild bonus of the low hanging attack/health crown-bought boosts, and I think it makes sense to burn a few keys pretty quickly.