Ah very sorry you were disturbed at work! But I have tested it, and it does seem to only be a graphical glitch. I used a Sword and Spear, initial attack does both. Then after a sword restricted only skill it will swing with the sword twice but the damage is that of the different weapons (Tested using your suggestion). This happens regardless of the hand any item is in.
Greetings. I have identified the problem, and released version 2.66 of the dual wield plugin with the implemented fix. The updated version is available for download on the project page now.
Sorry for the inconvenience, and thanks for reporting this bug so it could be squashed. Never feel like you're bothering me (even when I say I'm at work), when reporting even the simplest of glitches like this one.
-Ramza