You're exactly right - the base skills offer a decent baseline kit and synergy with the special stuff for each element!
Mana Blade already gives some free defense/resist penetration, so I think it's safe to go for Agility/Vitality (defensive) or Focus/Dexterity (crit) depending on whether you want to be tanky, focus on spells, or focus on skills. Cuz you still have access to the full Longsword skillset too.