If slimes are around "attacking" the goblin their happiness will not increase. The same can be said when they are already fully sad.