Hacker News new | past | comments | ask | show | jobs | submit login

The obsession with p values is the most absurd thing.

You write as if there are "positive findings" and "negative findings". This isn't true in orthodox statistics. There are only "negative findings" (the null hypothesis is rejected) and "null findings" (nothing is rejected, nothing is confirmed). Only the negative findings get published.

What doesn't exist is "positive findings". Nothing is ever confirmed: not the null (it's assumed to be true) and not the alternative (it's not even tested).

Now who wants to print a journal in which 95 out of 100 articles say "we learned nothing" and the other 5 can't be reproduced? Much better to print a journal in which every articles claims a results, even if none of them can be reproduced.




All 100 are results.

The studies saying the five can't be reproduced, where are they.

If I'm designing an experiment to attempt to confirm a theoretical model then finding similarities in the 10 prior attempts that failed could give me clues as to what to try. Certainly if 10 respected labs have done things in exactly the way I was going to try then it's worth questioning long and hard whether I really need to repeat that procedure.

Why did these all fail to reject the null hypothesis. That's a powerful question.


"Why did these all fail to reject the null hypothesis. That's a powerful question."

Because that's nearly always the outcome. By conventional statistical metrics, the null hypothesis isn't rejected >=95% of the time.

"The studies saying the five can't be reproduced, where are they."

They don't get published, because of the aforementioned statistical problem. The bias toward positive results isn't irrational; it's a natural response to the fact that the vast majority of what any scientist produces will be a "negative" result.

The way you learn what not to try is by studying under experienced scientists, and talking to other current practitioners. For any field, there's a vast shared experience that guides experimentation. As a new researcher, a good place to find this kind of information is in review articles and book chapters. But mostly you get it by working with experienced people.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: