Which page did you refresh from? Refreshing from the library page should pull down every collection on your account and clean out any ones that have been deleted from the server.
I did it from the collections page. Here are the collections that won't go away.
