the walls need a few seconds to be fully broken, the going down is part of the "animation".
Should've probably explained it further in the description