Think the problem here is, that maybe your controllers imitating the keyboard and therefore you have some odd configurations. I personally had the same issue with my steam controller, i had to change the settings in the config panel of the steam controller menu. Don't know if you have something like that for Logitech.
The logic how players join and get there colors will change in the future, maybe the system is much more solid after that but unfortunately that is not something for the near future. :(