ok, i tested the dungeon and figured out that watching asmund's scene (in the stop next to the bonfire) and then picking up his whistle in his room BUT finishing the dungeon without going down in the prison to blow the whistle will disable it forever.
it is a pretty specific sequence of actions, because not picking up the whistle at all will still allow you to do so after you finish the dungeon and go back in to explore more, and the event will work as intended. this also means you had at least 7 AGI to collect the jester badge in the top left section early.
i will report the bug on the discord, for now, you need to reload a save prior to defeating king harald (or restart the game)