why don't you try attacking it, the book say fighting power is soldier count multiply by morality.
I think that barbarian camp is also the end of the version, I've already drop their count in 3000, nothing still happens
there's a cloak man near the castle, ask him for money or soldier