Alright, I think I fixed it
It seems like it indeed was some weird race condition and/or timing thing. Once I chopped off a piece of the 8-bit counter I used for the address line to introduce a delay in the clock and fiddling with the tick speed settings it finally displayed a full screen of pixels!
