Hacker News new | past | comments | ask | show | jobs | submit login

Although in "real" terms 2σ is still pretty significant - but not firm enough to go "yes, absolutely" - but if they've got 2σ it's unlikely to be just noise, albeit still perfectly possible. IIRC 2σ is about a 1/20 chance of being erroneous. 1σ is almost meaningless - 2/5 chance of being erroneous.



The significance of 2σ really depends on the context. If we are looking for something at a particular place in the energy spectrum and we see a 2σ detection in the right place . . . well, that is tantalizing but not convincing. However, in this case, we are looking for unexpected signals across a whole range of energies. By sheer random chance, there will be a number of 2σ peaks that are meaningless noise. It gets a little more interesting if we see weak detections in two different experiments because the combined likelihood of two experiments having random peaks at the same place is considerably smaller. However, 1.2σ is nothing but noise, so I would say nothing to see here . . . at least, not yet.


The look-elsewhere effect is already taken into account. This is what is meant by "global".


Hmm . . . you did write that; I missed that. That brings the result up to tantalizing, but not convincing.


1/20? So if 20 tests were performed at the facility, we'd expect at least one of these to appear spuriously? Presumably there have been 20 tests performed there?


This failure mode is correct and very real. It's actually one of the reasons why we're starting to see significant pushback against p-values. https://xkcd.com/882/


This is a helpful table for converting sigma levels to percents and fractions (although those you listed are pretty good rules of thumb):

https://en.m.wikipedia.org/wiki/68–95–99.7_rule#Table_of_num...




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: