You need to use lure during the fight (Preferably few of them). Enemy get tamable status. Then you need to finish the fight in any way (Except losing or escaping). If you have a free spot in the prison (3 max for now) then the creature is captured (100% chances is after using 5 lures)