No, you just get the amount of points you had at the start of the invasion, doesn't matter which ones stay.
Yeah, there is a bug if you click on the right tile and drag your mouse over the left one. It allows you to place the left tile into any space for tile on the right.
Invaders have a very simple and exploitable logic, if you build your town in a funky shape. It's fully intended :)