This is a very detailed and relaxing game! It could just be me, but it couldn't find how to remove an exhibit (ex: empty display case).
When you have it selected you can press 'delete' to remove it. Or O if you are playing in your browser on mac also works
Ok thank you!