I had thought about a key press, like you said, to change costumes, but after playtesting without it I rather enjoyed trying to avoid bumping into others by making sure to go around them. I understand that it can definitely be frustrating at times (especially when they are clumped together!) but I think it adds to the challenge.
I spent a while trying to figure out how to have the number of houses that light up increase nicely over time, glad you liked it :)