I think you're right on the money. It was an increasingly simplified version of a wider original idea, that we ultimately just didn't have the time needed to execute. I actually spent several hours during a final crunch attempt, trying to get sound going, but I just didn't like the results I was producing and scrapped it.
On the 20th floor (all elevators) there actually is a hint, though it is admittedly incredibly subtle. There's a section with red writing on the wall, and one elevator in between the only squiggles that are mirrored on both sides and the closest one the art pack had to an "up".