I think it's less about the head getting chopped than about having "the head take up about 1/3 of the image," as Karpathy says. So what the net is learning is composition, or balance in an image, which is really cool. The rule of thirds is actually pretty well know to people in photography:
possibly, but none of the cropped examples have cropped chins. It's also well known in photography that you can cut off someone's forehead, but never their chin.
https://en.wikipedia.org/wiki/Rule_of_thirds
(Our deep-learning framework http://deeplearning4j.org missed his list, but it's got working convnets, too.)