Thanks for playing! The knife does make a difference to how people react. If you equip your knife and people see it then they will get alerted. If you don't equip it then they won't get alerted, unless they're already alerted.
I also have to say that seeing a broken window also alerts them.