On: The truth about deep learning

[riyadparvez]

There are a lot of innovations recently, not just the computing power -- layer by layer unsupervised training, batch normalization, ReLU, dropouts, momentum, gated memory unit, neural language modeling, encoder-decoder just to name a few.

[ewjordan → riyadparvez]

Yep, there are a lot of innovations, and plenty that people are holding back as well.

None of it involves the word "deep", it's all just bog standard 1980 style nets with tweaks.