Got it! Try to make that more obvious to the player through UI and such. Perhaps a counter that shows current speed, and there are also buttons next to it that you can click to change the speed/difficulty?
