The only way to really stay caught up is to read the papers that are being released (NIPS, ICML, ICLR, CPVR, etc). If you use a course or guide, then you're behind by definition. If you're okay with being a little behind, then the Deep Learning courses for NLP or Computer Vision are both excellent. NYU also has similar courses. You can also try following some major Deep Learning people on Facebook/Twitter as they post about the new stuff daily. People to follow: Lecun, Lawrence, Bengio, Cho, Karpathy, just to name a few. Interested to hear others ideas on how to stay up to date.
