Are there any negative effects to using growth orbs?
Not directly. There's an NPC near the sewer that can remove all applied essences and return them to you as untyped. The only "downside" is not being able to manually decrease your stats past that point without doing that. There's still scaling factors and limits, so this only really applies if you want to see a specific amount of growth in combat.