Hey Zombikittie! Thanks for commenting :) I'm really sorry but this may be a bug with the game! Unfortunately when the interact icon shows, a player might not yet be able to actually open the door. This is because doors detect the player with colliders independent of the layermask for the interaction icon.
It should be okay as long as you are right up against the door! I know this is annoying and I will probably work on a bug fix for this soon, I've just been super busy recently so haven't had much time to work on the game! Hope this helps :)