Hacker News new | past | comments | ask | show | jobs | submit login

"I have not failed 700 times. I have not failed once. I have succeeded in proving that those 700 ways will not work. When I have eliminated the ways that will not work, I will find the way that will work." -- Edison

By publishing null results you avoid 100 scientists going the same endearing but unproductive routes, which can also help build good hypotheses and speed up progress.




Getting a null result doesn't prove that the null hypothesis is true. It means that we can't say anything with confidence based on this experiment. There is a huge difference between failing to make X work and proving that X doesn't work and I fear that if people start conflating the two then many potentially productive routes will abandoned far too early,


"Absence of evidence is not evidence of absence."

A null result (no significant evidence for anything, a p value above 0.05) is truly null. It doesn't even confirm the null hypothesis.

Common wisdom says that Statistical Hypothesis Inference Testing doesn't work because researchers engage in "P Hacking". That's a half truth, researchers really don't understand the meaning of the p-value.


Which is completely bogus. Absence of evidence, when evidence is expected is indeed (possibly weak) evidence of absence.

The stronger "absence of proof is proof of absence" would be fallacious.


> Absence of evidence is not evidence of absence.

A more accurate way to state this is “Absence of evidence is not always evidence of absence”. Or “Absence of evidence may or may not be evidence of absence depending on the circumstance”, though it doesn’t roll off the tongue as easily.


Well, the long version is this:

"When trying to reject the null hypothesis (sensu Fisher), absence of evidence (for its invalidity) is not evidence of absence (of its invalidity)."

Fisher wasn't the only statistician, he wasn't the only confused one either. We can go with Neyman and Pearson instead:

"When trying to reject the null hypothesis with a sufficiently powerful experiment, absence of evidence is evidence of absence."

This requires a power analysis, which is something most studies lack. Constantly reminding everyone of these details and misguided philosophical differences is tiresome, so I prefer to go with Laplace, Bayes, and Jaynes:

"Everything is evidence."




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: