Hacker News new | past | comments | ask | show | jobs | submit login

It's harder than you might think to control for multiple comparisons. The Bonfaroni correction assumes that each experiment is independent, and so penalises correlated experiments unnecessarily harshly.

On the other hand, other tests typically require the researcher to make explicit assumptions on the correlation structure of the experiments despite the fact that it is not directly observable.




You are probably thinking of Sidak correction when you state independence is needed. Bonferroni correction does not need independence. You are absolutely right about Bonferroni being a severely conservative correction though -- at least the 'first order' one that uses only the first term of the Bonferroni inequality. One can take more terms to be less conservative but those aren't as easy to apply as you need to know the joint distributions over larger and larger tuples of events.

Another more recent technique for 'exploratory' yet correct technique is to exploit differential privacy and dithering.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: