There is a knocker on the door. In order for you to get in you need to knock it a certain sequence for 4 times.
If you find a letter in the chest underneath the bridge, it will tell you the sequence. There are also 4 fans lying around the courtyard. Look at each fan carefully.
