no you have to have them in the mansion but in alll honesty it's not exactly a problem especially cause most of your slave are usually not home either because of modification or getting higher grade after for then inventory menu you can select to check only potion there some other page in it dedicaced for each category of item, plus if you click on the slave they are an option to use item on them wich are limited to potion.