You should multiply the movement with time.deltatime then or even better move it to FixedUpdate()
Well, I did do that, but there is probably some other mischief going on that I can't see.
