Indeed I was washing them completely. I sat there on a spot until it's completely clear (I use flashlight on it as well, it's hard to see otherwise), then I move on. Before I was done with another small section on the same window there were already dirt specs on the spot I just completely cleaned. I tested this multiple times in a row to make sure I wasn't just trippin.
Just retested it in the newly released 0.8.2c and I did not have the same issue anymore, no dirt specks after waiting a full in-game day. Huzzah!