Thanks for this insight. Can you kindly also suggest a good book for someone to start with Bayesian Statistics? I could really use a suggestion about first and second book on this.
About Probabilistic Graphical Models, is there book other than Daphne Koller's book that you would suggest?
I think PGM's are covered by a lot of "standard" ML texts -- someone else mentioned Murphy's book which is great and is humongous but is a good reference for pretty much every method under the sun.
"Machine Learning: a Probabilistic Perspective" is more an encyclopedia of algorithms I would say, and it has lots of typos. I personally would not recommend it (except for the amount of algorithms that it covers, many of which are usually not found in other books).
Are those really the best starts for "Bayesian statistics"?
Especially the first 2 are rather the standard "intro to ML textbooks", with a frequentist focus (ISL may even have zero Bayesian stuff - Naive Bayes is not "Bayesian" – while ESL still has maybe 10% bayesian content if that).
You make a good point. It's been a while since I flipped through them, they just come up in lots of discussions on this topic. I agree that the series you link to is really great for PPL and Bayesian methods. You may find that the library upon which it's based (PyMC3) is built on top of Theano, which has been abandoned and deprecated. PyMC4 is around the corner and uses TensorFlow Probability. Early, informal reports say it's 10x faster.
The former is a much recommended book since it's very comprehensive and builds everything from the ground up and was the basis for the entire course. The latter is a beast of it's own and we simply covered what was effectively the first chapter as part of the course.
About Probabilistic Graphical Models, is there book other than Daphne Koller's book that you would suggest?