On: How Bayesian Inference Works (2016)

[bjornsing]

> Bayesian inference is a way to get sharper predictions from your data.

Funny, if I had to summarize it in one sentence I'd describe it in the opposite way: Bayesian inference is a way of making less sharp predictions from your data, with quantified uncertainty.

[ptero → bjornsing]

OK, I would counter-propose (as I tend to work in a dynamic world):

Bayesian inference is an efficient way to track your estimates and uncertainties as you accumulate data.

[j7ake → ptero]

Often people avoid a fully Bayes treatment of a problem in order to make the problem more efficient. A full Bayes treatment can be much less efficient than a more "shortcut" approach, such as Empirical Bayes.

[sjg007 → j7ake]

Considering these priors, Bayesian inference quantitates your ignorance.

[gugagore → j7ake]

Perhaps they meant "data efficient" as opposed to "computationally efficient".​

[_0w8t → bjornsing]

Quantified error bounds can be added to most machine learning algorithms using Conformal Prediction and similar ideas, see [1]-[2]

[1] - https://scottlocklin.wordpress.com/category/tools/machine-le...

[2] - https://www.amazon.com/Algorithmic-Learning-Random-World-Vla...

[closed → bjornsing]

Sometimes you are very certain prior to updating your beliefs (in the form of a posterior), which can lead to very sharp predictions from (and possibly in spite of) your data.

[digitalzombie → bjornsing]

If your data have noises/randomness, it most likely does, and you use your statistic to reduce the variance of the noises then it would be sharper would it not?

[gogoengie → bjornsing]

Quite so. There is a difference between creating uncertainty versus revealing uncertainty.

[bernardlunn → bjornsing]

Or Bayesian inference trades certainty for speed ie how we work in real life.

[nightski → bjornsing]

Sure if having a sharp prediction means ignoring uncertainty.

[chibg10 → bjornsing]

That's a very strong statement. There's domains where machine learning tends to have better predictive performance than (Bayesian) statistics, but the converse is true in many other domains.

I would summarize it as Bayesian methods work best in areas where there's often not enough data, there exists significant expert knowledge, and you can properly specify a model. And yes, they do quantify uncertainty.