I don't think locking certain weapontypes from being used together while dual wielding is currently possible, although you might be able to rig something up with Yanfly's equip requirements plugin to prevent certain weapons from being equipped entirely if another weapontype is already equipped.
That is actually a good suggestion though, so I'll see what I can do about adding support for that into the plugin.
On another note, you can use if conditions in action sequences to do different things when different weapons are equipped on the same skill. I personally have my attack sequence check the actor's weapon motion type to see if he's using a ranged weapon to decide if he runs into melee range, or steps forward for a ranged attack. The <Target Action> section of your action sequence on your attack skills will be repeated twice, once for each weapon, so an if condition that checks what motion is being used in that section will update when the weapons are swapped. You can use this to have a sequence where if the actor was using a crossbow in his main hand, and a sword in his offhand, the actor would shoot from ranged and then run in for his second melee attack, and even the reverse, although personally I think it looks better to swing in melee and then point blank shoot a crossbow into melee as well, but what do I know? :)