One option would be to have time advance in increments when you leave and go do something else. For instance, if you visit the shop and then come back, all the plants will advance 1 growth stage. Or if you go talk to an NPC, the plants will advance their growth. Basically incentivize going to do other activities and returning to your plants to do maintenance and check if they are done growing, rather than just sitting there waiting for real-time to pass.