There is a guard infront of the castle but without the !Icon.
Did you use the fast travel system at least once after unlocking all requirements for the next event step of Grey?
