ah, i thoguht you need to keep two gems
It doesn't seem possible :D Lizard leaders agreed thatthey have to give up on the treasure. They didn't plan to have it back.
Spoilers:
And in fact when you choose to give the second gem back, during the feast Othra will give you it back again :D
