Oh no, Did I screw up and got the Fire Crystal too Early??
You can get it early too. Just talk to Ayane at the kitsune shrine and everything will be fine.