Please first check the Book[Tab] for the current quest info, also press enter on that quest to see where you must go.
The first quest guides you to a new spot, then you need to find the builder (which is not intractable yet) and lead him back to that spot. After that story segment he becomes intractable.