In theory, you can progress without ever being involved in the guilds, but you miss out on a lot of content, and you make things harder for yourself.
This is probably a good enough approach. I'll mark their room as unsuitable and leave the rest of the house available.
The tattoo doesn't let them know where you are (I don't think I ever say that in the game), but like all tattoos it can be use for identifying people (especially since these are magical). If you get sold as a slave, they'd probably count it as a loss and move on. After all, cows are replaceable... :^