Yeah, it's a tough balance. I think the ideal case is that there is enough reward for leaving the tower, that you are willing to take the risk. The best moments for me are when you are pushing your luck to see if you can just get the treasure opened in time before getting back to defend the tower. Finding ways to encourage that type of player behavior (and discourage turtling) are high on my list of todos.
I think a big part is just that you eventually just become invincible and staying at the tower is the winning move. Open to have suggestions if you have any!