I just figured its cause our noble probably might have dressed down to appear like a commoner to sneak into the castle. Or that the other commoners just wouldn't recognize our noble and mistake them as a commoner. xD