Many of the mansion rooms use the following code to reset the mansion UI and adding this to the farm manager function seems to clear away the panels before the farm screen is shown. I'm guessing that only a small portion of _on_mansion_pressed() is needed but it is widely used as a general reset.
_on_mansion_pressed()
yield(self, 'animfinished')