An interesting idea. It needs a little work but it is not bad at all.
Here is a video we made while playing: