SVMs are good if you want high accuracy without much fiddling and don't have many training examples. It is pretty simple to get off-the-shelf results from SVMs. However, SVM training is quadratic in the number of examples, and you have to get really hacky to train >10K examples.
Neural networks are good if you have many training examples, and don't mind doing hyperparameter tuning. I have trained neural networks over 1B examples on a single core. (Took a month.) However, you have to tune your learning rate and regularization, and there don't yet exist good packages to do this automatically.
It is also much simpler with neural networks to learn over custom data, e.g. mixing supervised and unsupervised learning (labeled and unlabeled examples), transfer learning, etc., because you can change your evaluation criterion and minimize it.
If you want to do deep learning, we have a much better understanding of how to do training using neural networks, particularly because we can train on such large datasets.
Neural networks are good if you have many training examples, and don't mind doing hyperparameter tuning. I have trained neural networks over 1B examples on a single core. (Took a month.) However, you have to tune your learning rate and regularization, and there don't yet exist good packages to do this automatically.
It is also much simpler with neural networks to learn over custom data, e.g. mixing supervised and unsupervised learning (labeled and unlabeled examples), transfer learning, etc., because you can change your evaluation criterion and minimize it.
If you want to do deep learning, we have a much better understanding of how to do training using neural networks, particularly because we can train on such large datasets.