Hacker News new | past | comments | ask | show | jobs | submit login

Thank you for the explanation.

As I understand it, correlation is not sufficient to prove causation. "Null hypothesis" is one statistical tool, an abstraction over our understanding of the real world.

Consider the case where all of those infected with disease D die, while 99% of those infected with D and treated with T survive. Further consider that T is widely administered. Clearly, the correlation between being infected and dying is very low although with a deeper understanding we know that D in fact causes death.




>As I understand it, correlation is not sufficient to prove causation. "Null hypothesis" is one statistical tool, an abstraction over our understanding of the real world.

Absolutely. The only reason we use it is because most of the other alternatives turned out to be not that great in practice, but it has plenty of limitations to keep in mind (like correlation != causation, but also the common p-hacking that happens, etc etc).

>Clearly, the correlation between being infected and dying is very low although with a deeper understanding we know that D in fact causes death.

That's true, although working back from just the people who die and focusing on those, the method is eventually able to unearth the correlation too. With the people who die, we would naturally start to ask why them? Did they do anything differently than the rest of the population? We'd see pretty quickly that there was one huge factor in common, which is that none of them were using T.

I'll admit that's not enough to say just looking at correlations and trying to disprove nulls is going to be enough in every case. I would just say it's a pretty good starting point when people disagree on a fundamental level about what should be obvious, or who should be proving what. It's really kind of dumb, but in practice in the case of people missing something (most deficiencies, like not having enough vitamin B12, or not having enough thyroid hormone) stats do a pretty good job of noticing that all the people dying mysteriously happen to be missing the same thing. (But of course it misses plenty of things too. And where it gets a lot harder is understanding why any of this happens)




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: