I think for most people that hurdle of understanding is immense, and isn't likely to get any better. ML/DL infrastructure will continue to improve and deploying AI systems will become more accessible as time goes on. But I don't see familiarity with (non) convex optimization, statistical learning, topology, etc. becoming mainstream. Without these its hard to see a reasonable path forward for AI.
For me this year in DL was a lot narrower. it mostly revolved around systems becoming more human in capability. I think most of this is in the article already:
We can now hopefully say that ImageNet has been solved.
AlphaZero tackled structured games with a single algorithm [1].
Tacotron2 produced completely passible TTS [2]
GANs improved dramatically and saw some good theory to back it up [3] [4].
We also started to care more about how well our models do in general, which to me shows maturity:
Adversarial examples showed their teeth, and hopefully convinced everyone to care about robust models [5] [6].
Reinforcement learning algorithms were shown to have poor transferability [7].
In 2018 I hope to see a new kind of CNN. Residual style networks are the norm now, since they mostly solve problems with gradient flow. But take away all the skip connections and we're mostly left with a vgg-style linear net with box filters. I'd be really excited to see a network with image-sized conv filters that could adapt their shape (and therefore representational power) to a given feature or signal.
Hopefully 2018 is the year where people stop calling AI a one-trick-pony. I don't hear it as often these days, but I think its time to put that phrase to rest.
For me this year in DL was a lot narrower. it mostly revolved around systems becoming more human in capability. I think most of this is in the article already:
We can now hopefully say that ImageNet has been solved.
AlphaZero tackled structured games with a single algorithm [1].
Tacotron2 produced completely passible TTS [2]
GANs improved dramatically and saw some good theory to back it up [3] [4].
We also started to care more about how well our models do in general, which to me shows maturity:
Adversarial examples showed their teeth, and hopefully convinced everyone to care about robust models [5] [6]. Reinforcement learning algorithms were shown to have poor transferability [7].
In 2018 I hope to see a new kind of CNN. Residual style networks are the norm now, since they mostly solve problems with gradient flow. But take away all the skip connections and we're mostly left with a vgg-style linear net with box filters. I'd be really excited to see a network with image-sized conv filters that could adapt their shape (and therefore representational power) to a given feature or signal.
Hopefully 2018 is the year where people stop calling AI a one-trick-pony. I don't hear it as often these days, but I think its time to put that phrase to rest.
[1] https://arxiv.org/abs/1712.01815
[2] https://research.googleblog.com/2017/12/tacotron-2-generatin...
[3] https://arxiv.org/abs/1710.10196
[4] https://arxiv.org/abs/1701.04862
[5] https://arxiv.org/abs/1707.07397
[6] https://arxiv.org/abs/1710.06081
[7] https://arxiv.org/abs/1709.06560