Hello Schizomi :)
As nabrappy said (thanks for the reply btw!), you can customise the tileset.
Another solution is to use Event as a third layer : position an event on top of the window tile and give it the apparence of the curtain tile.
Last solution is to migrate your project to Rpg Maker MZ, which use 4 layers (I think a free demo is available).