The action button works more like a radar ping, it shows you where the item is when you hit, but doesn't track it. You may be moving too far from the item when you hit space again, if no item is "pinged" the arrow will disappear!
