Change the church by placing "rowdy" buildings near it. Goblin huts are probably the easiest to use for this purpose.
