I think i fixed that by choose "Balanced" in Control mode in ControlNet. Previously ControlNet was more important
