Think I diagnosed the issue, it has something to do with how many items have changed, how many trees cut and how much land sowed, the lag seems to increment with that rather than the days.
Do experiment with this.
Also I meant to mention some more points but forgot too (lol)
-The hotbar is not indicated well within the inventory, which was confusing the first play around: it's the top bar of the inventory btw.
-Hotbar needs hotkeys for numbers.