I agree, I think that'd be a great way to make it more of a true instrument - I think something like that would be really cool to have as a fully dedicated tool with swappable sounds! I'm not involved in the gameboy music scene but I know that it's pretty hardcore so I'd imagine there's a lot of neat stuff out there already.
Likewise, thanks for discussing the nitty gritty with me! :D