That's actually intentional; the challenge is meant to be that the blocks have to "work together" to see what should go where.