So you've chosen the Crimson Armor route. The first key that you can obtain is on the floor 4-3 in the room 4-1. The second one is on the floor 4-2 like you mentioned. If you collected both of them you'll have to buy the key in the shop. The boss route is pretty straightforward. You'll be able to find one key on the each floor starting from the floor 4-4 except of floors 4-6 and 4-7 (there are no keys on these two).
Thanks for the quick reply! Your response encouraged me to persevere and I have managed to make some additional progress now. (Specifically, I previously somehow missed the instructions on level 4-5 about how to activate the lanterns, now that I've figured that out more rooms are opening up). Anyway, thanks for making this game, it is really fun!