That was my first intention of the game, to have the shadows affect the object directly, but then I ran in to too much performance issues so I had to pivot into this where the shadows are a reason to move your character because of visibility of the level.
Thanks for your feedback!