With regard to the class conditioned regime they are experimenting with; this is merely attempting to explain more precisely _how_ deep nets are able to distinguish features between classes. We already know that they do; but we lack a detailed model of precisely why they do and my understanding is that many base assumptions made by e.g. statistics will not help you at all with neural networks (for instance, overparameterization leading to better performance on out-of-corpus rather than overfitting).
> many base assumptions made by e.g. statistics will not help you at all with neural networks
There is lately a lot of hate against classic statistics on HN. I don't know why. Does it help to understand why and how NNs work? Not yet. But saying that it is utterly useless and won't provide any useful insights in the future sounds to me like telling the young Steve Jobs that dropping out of college and taking calligraphy classes instead of is utterly useless. And still, I am writing this on an Apple product, which set the standards for digital typography...
I have actually seen a fair amount of respect for classical stats on HN; perhaps it is due to my attention filter.
I'm thinking of some wonderful posts describing where/when/why linear regression can offer performance which is very close to the best from a NN-- except that regression models train much faster on much less data AND are interpretable.
Some discussion about how a NN works well for data where there is a lot of (statistical) structure to the data-- two close pixels in an image are likely to have very similar color/luminosity (and if not, the difference is important to the model, i.e. an 'edge'). But that NN don't do as well in a domain where the different features of the data don't have such relationships, say an econometric model or many biological models or ...
I'm not an expert, but statistics may be the branch of mathematics we wind up using to solve the unknowns of machine learning. I have no hate for classic statistics, sorry if I gave that impression.
