I believe I found the problem. There is a wall in front of the Joestar Mansion that prevents you from going there that's used earlier when you are there with Erina. You can still get to Riverside directly from Downtown if you head this way. I'll still release a patch so others don't have this problem.