Thanks!
The naive approach would be to save the position every frame to an array and replay that. but frame rate may differ when playin and replaing so it's not really a good solution.
What we went with is storing each action you take. "Left pressed", "Left Released", "Jump pressed", "Jump released", "Right pressed" and "Right released".
We've stored that along with the time at which the event occurred in an array.
The ghost mouse actually uses the same "movement controller" class our player does, it just doesn't listen for keystrokes, but for events from our "Replayer" class that goes through that array and raises those events in order, if the specific time for that event was reached :)
Due to floating point precision errors and some non deterministic things (I assume). It rarely got dessynchronised, and sometimes the mouse would not make the jump, where player clearly reached the platform. so we've added some additional events every now and then where we recorded exact position and velocity of the player, and when replaying, we would force set that on our ghosts.
Our leaderboard backednd supported us sending those replays as serialized strings, so it could be a really cool feature to include in our game where you could download the run from the leaderboard server of top players and replay it locally using our system so you could see how the heck are they getting times so low in their sppedruns :D Well.. we did not finish it due to not having enough time and choosing to polish the game instead <3