A nice, simple idea -- which means you can experiment with the mechanic to add fun. Think about how the player selects their colour:
- Do they press a particular key to choose a specific colour? -- is this challenging enough?
- Do they press a key to cycle through colours (e.g. red->green->blue->red) -- this would add challenge if against the clock
- Do they have to visit an object to change to a specific colour? -- this would make it more of a puzzle if certain colour-changers were in awkward places.