I can tell a lot of passion when into this project. The reliance on a controller though is pretty rough.
If you plan to continue this project, I would definitely iterate on different control schemes. Possibly even allow button remapping if needed.
I've been trying to theory craft how I would do a keyboard control (remapping shouldn't be too hard after that). I think walking and speaking buttons should be on W A S D - that should be pretty simple. But changing color is more complicated in my head. I'm currently taking the normalized direction from the left stick and mapping that to a color wheel. I'm unsure how to that with a mouse. But I will give it a shot with the first pass of any updates we do (along with other controller type support).