Perhaps to make it easier you can let a color stay still as soon as you gathered all of the same color in the right area?Or perhaps make them move only when you touch hem with the mouse cursor?
