The whispering happens every now and then, but you'll know if it's a preacher/intruder if you hear it say "I'm inside your home," or anything similar.
Also, sorry for the long gaps of time it takes for me to respond. I'm not on itch.io as much.