Yes I know, it was a rhetorical question. Imho, if having more parameters causes problems, then the system should simply not use those extra parameters. But the theory is not there yet.
I think his point is no one can tell you from theory which regularization methods to apply to a particular problem to get the best results. You need expert knowledge, experience, and hyperparameter tuning.
Transfer learning helps with overfitting too. It is proven to get more generalized model if you use transfer learning that if you train a model on your own with same data (even with large datasets). You need expertise in deep learning but the good thing is that you don't need a lot of expertise in domain of the problem.
