Hmm, double clicking as in double right clicking ? that would be harder to discover unless it's explained somewhere; as double right clicking is not very common in PC usage. If I understood wrong and you meant moving it to left click and using double click activation that could work to reduce its accidental activation; it might require some time to get used to no zoom there. But hard to say how it would work when more functions are assigned in the future.
Another option is binding it to a key like "~" given that the top row is camera things; and/or transforming it to a radial/pie selector which pops up when key is hold.