Love the idea of forceful encounters from bandits, perhaps maybe they just have a bit more combat prowess, (dodging etc) than the skeleton dudes, if they hit you and u fall stunned thats when they "strike" and have their way. Maybe they could just make off with some gold if youre unlucky enough to have that happen, and some extra damage, also keeping the possibility for group interactions and trains.
I think it would be important to include an option to turn off said forceful encounters and just keep combat though, and rather than having their way with you when they knock you down, they just steal the gold, or equips like you suggested, and run away.