Yeah, I agree that's the way to go! Easy to implement on unity.
I did it for the first two buttons, but I rushed the third and forgot about this edge case, which was done in the last hour or so.
Last minute bugs are the essence of game jams I guess :)