On the map days, it says select you the number of the room you enter.  The first map Dragon Temple the S is right next to the 3 so I would have to go to 3 I assume but then after 3 if I want to get to 2 I would have to go through 5?  

Yes. The 3 must be your first room.

To get to 2, you would go through 5, but I can see how one would argue that it isn't required. If you feel you could go around the north passage at the top without visiting room 5 technically, then you may do so. It isn't that strict.