fighting and walking outside decreases energy, Endurance (or was it strength?) and high level will make them last more.
if you just started, resting outside using 3 suplies and 5 food for each member will make energy bar full.
I usually deal with it using the invigorate spell, after getting past 10 Magic Affinity it will restore 50% energy (for each use) AND will lower stress.