My educated guess was correct and the shader gurus confirmed that there's no way to fix it, it's just the hard limit of float-values haha...
