Another odd bug I got where my Return Gems stopped working. Every time I use one it cancels the teleport and displays "The Escape Gem doesn't work here." on any given map.
I was forced to sell my stack and buy new ones which then worked. No idea what causes it to happen though...