The point is that the word isnt convenient, its anti intellectual which robs people of being able to discuss topics in more specific and nuanced ways. It is too broad and has subsumed every word to describe narrative components.
If somone asks me "whats the lore" there are at least 5 different things they could mean. Which is inconvenient and useless because I have to ask clarifying questions. Just say "backstory" or "plot point" or even "information".
Whatever word you are using "lore" to replace you should use that word instead. "Lore" is just the current trendy word to say and I cant wait for it to die out.