As another ATLAS physicist, I can say that this is an excellent article from Prof. Schott. He is very politely arguing that "someone messed up". I'm not sure I agree so much with the point of combining the LEP experiments, which do have some tension with each other. Unless the combination is specifically taking into account correlations between uncertainties at the different experiments on the same collider (which exist, but it's really hard to handle).
Another take many people in the field are expressing is that it's simply infeasible to reliably interpret statistical models at that level (especially one that is dominated by systematic uncertainty), since they are based on approximations and assumptions e.g. that certain nuisance parameters are "nicely" distributed and uncorrelated. See e.g. comments from Prof. Cranmer [1] who is one of the folks who developed the standard statistical formalism and methods used in modern particle physics experiments.
Why don't people use nonparametric methods to get around the problem of assuming certain parameters are "nicely" distributed? (Not a physicist, but curious – this seems like the "obvious" solution.)
Nonparametric methods are often used when the assumptions of parametric tests don't hold.
In physics experiments they want to fix the structure of the model and know the assumptions. They want to know the distribution and parameters to hold. If assumptions don't hold, they must find out why, find better assumptions and fix the model.
To say it differently: physicists are not trying to discover statistical laws. They are trying to discover physical laws trough statistics.
It sounds like they know certain assumptions regarding parameters that are not of interest are wrong. So why explicitly model those, if we don't care about their distribution? We (apparently) only care about an accurate estimate of W boson mass.
That works if the thing is something you can remove from the experiment and model separately, then put it back. In CERN many variables are tied to this one huge machine that is one of its kind.
I admit I'm not familiar with the model used to aggregate the boson data. But there's an entire community of nonparametric/semiparametric statisticians that works on problems just like this. It seems crazy to me that that millions of dollars are spent to build the machines to collect this data, yet the papers are written using statistical models with distributional/independence assumptions known to be false. (The tweet linked above seems to be saying something similar.)
Is there a concrete reason we can't be naive and just bootstrap confidence intervals for example? Of course I defer to the physicists here – but I'm curious whether there's some simple high-level reason the usual tricks don't work.
Sure, but I'm not talking about high energy physics in general, I'm talking about the estimates in this specific paper. Which the guy who is doing bleeding edge stuff seems a little suspicious of (see tweet).
This does not seems like bleeding edge, it seems like "Gaussian approximations for everything."
In fact, this is the reference for the technique they used: https://cds.cern.ch/record/183996/files/OUNP-88-05.pdf. (Although the criticism seems to be leveled at the way they estimate the correlations, not that linear estimator specifically?)
Because it’s all interrelated and too many variables make it impossible to nail down anything with certainty if you don’t assume some invariants somewhere?
> We observed for quite some time some features in the our data, which we could not explain. Once one of my PhD students came into my office and told that he finally figured out this feature: the protons in the ATLAS detector do not collide heads-on but under a very small angle, allowing the not interacting protons to continue their travel through the LHC on the other side of the experiment. Indeed he was right - we have not been considering this effect in our simulations, however - after some calculations and speaking to the machine experts - it turned out that this effect induces a feature in our data, which is opposite in sign that we observe; so we have been left with an effect that was twice as large and unexplained. In the end it turned out to be caused by the deformation of the ATLAS detector by its own weight of more than 7000 tons over time.
I know these people are incredibly smart and conscientious. And the standard model is extremely successful and well confirmed. But that's a lot of degrees of freedom.
"I do not think, we have to discuss which new physics could explain the discrepancy between CDF and the Standard Model - we first have to understand, why the CDF measurement is in strong tension with all others."
That's... cute. I doubt it will stop the theorists from flooding the arxiv with explanaitions in the coming days/weeks. Recall what happened when there was a barely 3 sigma (local) statistical fluctuation in LHC data:
I work in this field (different experiment); despite the downvotes this is a reasonable question. Reposting my comment from above, since there is confusion here (the other sibling comments are incorrect).
In particle physics, sigma denotes "significance", not standard deviation. Technically what we're quoting as "sigmas" are "z-values", where z=Phi^{-1}(1 - p), where Phi^{-1} is the inverse CDF of the Normal distribution and p is the p-value of the experimental result. So, 7 sigma is defined to be the level of significance (for an arbitrary distribution) corresponding to the same quantile as 7 standard deviations out in a Normal distribution.
In other words, "z sigma" means: That a result like this occurs as a statistical fluke, is just as likely as a standard-normal distributed variable giving a value above z.
I would add: If the null hypothesis is true, then "the result like this... (in this case the null hypothesis is of cause that the standard model is true)
If the null hypothesis were true, and the experiment were repeated infinite number of times with a different sample each time then "the result like this or more extreme ...
I agree with adding the "more extreme" part, but I'm not so sure about the infinite number of times part. Certainly, the p-value is (roughly speaking) the probability of seeing a result at least as extreme as the observed result, under the null hypothesis. But one doesn't really need to introduce hypothetical infinite sequences of replications to make sense of that definition.
Isn't the bit about repeating the study over and over again the whole basis of frequentist statistics, though? (Indeed isn't that why it's called frequentism?)
Quoting: "The plot shows the first 50 simulations. In the first simulation I picked some distribution {F_1}. Let {\theta_1} be the median of {F_1}. I generated {n=100} observations from {F_1} and then constructed the interval. The confidence interval is the first vertical line. The true value is the dot. For the second simulation, I chose a different distribution {F_2}. Then I generated the data and constructed the interval. I did this many times, each time using a different distribution with a different true median. The blue interval shows the one time that the confidence interval did not trap the median. I did this 10,000 times (only 50 are shown). The interval covered the true value 94.33 % of the time. I wanted to show this plot because, when some texts show confidence interval simulations like this they use the same distribution for each trial. This is unnecessary and it gives the false impression that you need to repeat the same experiment in order to discuss coverage."
> sigma denotes "significance", not standard deviation.
Nitpick: this is still a standard deviation in some (potentially very contrived and nonlinear) coordinate system. (As a simple example, a log-normal distribution might have a mean of 1 and a standard deviation effectively of multiplying or dividing by 2. Edit: also, multidimensional stuff might have to be shoehorned into a polar coordinate system.) But in practice you'd never bother to construct such a coordinate system, so that's more a mathematical artifact than anything useful.
No, there is no coordinate system. This is referring to the distribution of a test statistic for hypothesis testing. It's a 1-d real scalar, and coordinate transforms don't have any meaningful statistical representation. Of course there are much higher-dimensional distributions, in all sorts of coordinate systems, involved in sampling the test statistic, but at the end of the day this is all you are left with. If you change the underlying distributions of the model, then of course you will change the test statistic distribution, but that's meaningless, since the whole point of the test statistic is to quantify an observation in the context of a given model.
Anyway, as I mentioned elsewhere, the motivation for calling it sigma is that, by construction, it maps onto the quantiles of the standard Normal distribution. So an N-sigma result will have the same p-value as N standard deviations in a Normal distribution. So you can associate "sigmas" with "standard deviations of the Normal distribution". Perhaps this is what you are trying to say, but it does not make sigma a standard deviation in any statistical sense, i.e. it is not necessarily related to the variance of the relevant distribution.
For what it's worth, sigma is chosen for this purpose specifically to evoke the notion of "standard deviations". But quoting the std dev. directly is useless, since the distribution is unspecified. So we "convert" the statistical significance to the corresponding number of standard deviations of the Normal distribution, since that is a familiar distribution. If you like, it's another way of stating p-values, which physicists prefer because ours can have lots of zeros :)
Error bars account for the known and quantifiable sources of uncertainty. They don't (can't) account for unknown or unquantifiable sources of uncertainty, such as aspects of the experimental design that were not properly accounted for or unpredicted/unmodeled interactions with other particles or forces.
Known unknowns and unknown unknowns, as Rumsfeld would put it.
About a decade ago I saw a very nice figure of estimates of the speed of light over time showing this effect. Unfortunately I haven't been able to find it since.
I don't know if I have a favorite part, although the weather forecasting/probabilistic prediction and estimated error in speed of light estimates might be.
I think error modeling doesn't get enough attention, nor does error due to model uncertainty per se, and the paper explains the consequences of those two things pretty well from an applied perspective. I also think it nicely integrates model uncertainty, error modeling, and complex systems modeling all at once.
I don't think there's anything really groundbreaking in it but I think it does a good job of explaining the importance of certain things that are often really overlooked.
Just a small comment about the speed of light. The way we now define the speed of light, measuring it doesn't really make sense anymore. The speed of light is now DEFINED as 299792458 m/s and the meter definition is based on the speed of light. So in principle, one can only measure the meter and not the speed of light anymore.
Would it be the same in a hypothetical completely homogeneous gas?
My assumption is that turbulent changes in pressure cause diffraction, causing the light to not take a perfectly straight path. I don’t know enough about physics to know if that’s right.
The speed of light in a medium is the inverse of the refractive index of that medium. It's caused by the interaction between the electrons in matter (bound to atoms) and electromagnetic fields. It does not require inhomogenities larger than the atomic structure.
Materials seem to have different refractive indices for some sort of complicated quantum reason that only relies on the inhomogenity because of materials being made up of atoms containing electrons, not an sort of macroscopic inhomogenity.
If this result turns out to be right, at least part of that confirmation will involve producing a theory which reduces down to explaining other results - either from previously unknown systematic error or interactions.
Even if you only consider statistical errors, there is no guarantee that the actual value is in the confidence interval. The graphs in this article seem to use 1σ bars which have a 68% chance of containing the result, assuming a normal distribution.
Talking about particle/quantum physics, is there a book/youtube channel/whatnot that would describe (some/main) experiments and results that have convinced physicists that classical physics does not work when you go small. I mean, I know about double slit experiment, but I guess it is a long journey from that to the Standard Model.
So instead of the heavy theory, I'd like to see the stuff that made people scratch their heads in the first place.
The first real head scratchers were the black body spectrum [0] and the fact that atoms are stable.
Rutherford [1] showed that atoms consist of a tiny, positively charged nucleus and rather large negatively charged shell. It was hypothesized that electrons are flying around the nucleus like planets around the sun. But we already knew at that point that moving charges emit radiation, which causes the electron to lose energy and move closer to the nucleus. So it should pretty much immediately collapse into a point. Bohr then showed that if you assume that only certain orbits were allowed, it works out pretty nicely. Nowadays we now that there is such a thing as a ground state, meaning the lowest amount of energy the electron can possibly have around a nucleus is enough to keep it moving.
The idea for quantizing things came from observing the black body spectrum. If you sum up all contributions classically, you get infinity. Planck tried to see what happens if you assume that energy comes in little packets instead of a continuous spectrum. He didn't have any justification for it, but it matched the observations pretty well.
As a sibling poster commented, the blackbody spectrum was also inexplicable from a classical point of view (see https://www.feynmanlectures.caltech.edu/I_41.html Section 41-2), but I think that the specific-heat problem was known before the blackbody problem.
Quantum physics is a separate (but related) branch from particle physics so using the slash "quantum/particle" is mixing up two different things - which one do you really want?
Theoretical Concepts in Physics by Malcolm Longair is a mix of history and physics, by explaining how physicists came to discover their theories. I actually don't think it says much about modern particle physics though. It includes quantum mechanics.
Introduction to Elementary Particles by David Griffiths if you just want particle physics. Griffiths also has an intro book on quantum mechanics.
I'd be interested in all observations that make "non-classical" thinking of physics necessary. I have no deep enough knowledge to distinguish quantum physics and particle physics. So for example what is the experiment that makes us think that particles have something called spin? And what experiment makes us think that the spin can only have distinct values? And so forth.
Anyway, the books you proposed look interesting, thanks.
Maybe not exactly what you're looking for but these two videos discuss the results of a Fermilab experiment that hints at a crack in the standard model.
I would guess that unless some other team replicates this result, it's probably a measurement error somewhere. Physics can be very delicate and tricky, and it's easy to make mistakes. But, even the mistakes are opportunities for learning, so it's not a waste.
Do we actually know how physicists define "mass" in this context? Because, in physics many words have technical meanings that only physicists can know. For instance, as a layman when I see the word "particle" I imagine a spherical thing with an extension in space. But a physicist would laugh at me because in physics a particle is not a particle, it can be a statistical bump in data, it can be a field, it can be a wave, anything but a spherical particle. But a physicist would call a wave a particle and see nothing wrong with it. The same goes for mass, what physicists call mass can be voltage for instance. So does anyone know what "mass" means in this context?
> The same goes for mass, what physicists call mass can be voltage for instance.
You’re probably thinking of how a proton has a mass of 938 MeV/c^2. This is still a mass and not a voltage. 1 eV (electronvolt) is the amount of kinetic energy that an electron would have after being accelerated though an electric potential of one volt. By the mass-energy equivalence 1 eV is equivalent to a mass of ~1.783x10^-36 kg and a proton has a mass of ~1.673x10^−27 kg.
> You’re probably thinking of how a proton has a mass of 938 MeV/c^2.
Yes, that’s what I was thinking. But it seems that there is a problem with the definition of the word “mass”. Clearly there are at least two definitions. First, the weight of an object. Here weight is measured and weight is called “mass”. There is no equivalence, same thing is called weight and mass. Weight and mass are synonyms. This mass has nothing to do with electricity and has nothing to do with motion.
The second definiton of mass is related to electricity and motion. It has no meaning outside electricity. In this case, they accelerate an electric current and measure its kinetic energy and call this kinetic energy “mass”. Again these words are synonyms. Why do physicists like these silly word plays so much, I have no idea.
The weight of the object is not its mass. The weight is the force an object experiences due to a gravity. You have the same mass on the Earth and the Moon, but you weigh 1/6 of your weight on Earth on the Moon because of the gravitational force is 1/6th the strength of Earth's gravitational force.
The definition of mass is subtler, but you seem to confusing units with the definition of the concept. Units are necessary because you need a scale to measure physical properties. You can't measure a length and say that it's "ten". You needs units attached like feet or meters. An eV (electronvolt) is a unit of energy. Just like a kilogram (unit of mass) originally was defined as the mass of a 10cm x 10cm x 10cm cube of water at room temperature, the eV (unit of energy) is defined as the increase in kinetic energy of an electron (which has a fixed and known charge) accelerated across 1 volt. But neither the definition of the kilogram or the eV define the concepts of mass or energy, they just merely define units, which humans chose, used to measure mass or energy.
Now how does mass and energy relate to each other? Simply put, the Special Theory of Relativity, developed by Einstein in 1905, states that mass and energy are equivalent to each other. Now the word "equivalent" has a precise but complicated meaning that I will not explain here (if you do want to understand it, take a course in special relativity). This relation is defined quantitatively by E=mc^2 (Energy E equals mass m times the speed of light c squared). Let's first look at another relationship, distance = velocity * time. This equation can be be rewritten as distance/time = velocity. If we use meters to measure distance and seconds to measure time, we can "divide" the units and define the units of velocity as a meters per second or m/s. Same thing with E=mc^2. We can rewrite the mass m as m = E/c^2, and let the units of mass be eV/c^2. Using eV/c^2 or kilograms or whatever to measure mass has no effect on the definition of the concept mass itself (which you can think of as an intrinsic property of objects independent of units which affects their behavior in known ways).
Why do physicists make all of this so complicated? They don't. It is reality that is subtle and complex and hard to understand. Because the purpose of physics is to describe reality, it has to be subtle and complicated.
Particle and wave are just both incomplete ways to describe things - but for some phenomena combining both gives the best results : that is the very weird domain of quantum physics.
(Like if you're trying to predict what happens when speeds approach those of light, you have to make weird relativistic corrections stemming from the observed speed of light being the same for all observers, regardless of their relative speeds.)
If you're lucky to not be in one of those weird cases, objects as spherical things with a sharp boundary in space and using the normal composition of speeds work just fine.
Someone has run an experiment, and in this experiment they created a large amount of evidence that seems to say that quite an important particle in particle physics weighs something slightly different from what we thought it should.
This is important because the weight of that particle was predicted by our generally-accepted theory of how the universe works. If the weight is different, it means the theory hasn't taken into account everything that it should.
I work in this field (different experiment); that's not really true. In particle physics, sigma denotes "significance", not standard deviation. Technically what we're quoting as "sigmas" are "z-values", where z=Phi^{-1}(1 - p), where Phi^{-1} is the inverse CDF of the Normal distribution and p is the p-value of the experimental result. So, 7 sigma is defined to be the level of significance (for an arbitrary distribution) corresponding to the same quantile as 7 standard deviations out in a Normal distribution.
> Surprisingly, the researchers found that the mass of the boson was significantly higher than the SM predicts, with a discrepancy of 7 standard deviations. —JS
This is from the editor's comment at the top of the article, I'm guessing it was a mistake, but that might be why people are getting thrown off by it
I'm one of those dumb people that didn't have much math or greek in school, so this weird-looking o in the title was quite literally Chinese to me. Now it turns out that people in the know also misunderstood its intended meaning because it's in a different field.
For years I've argued foreign symbols and single-letter variable names mainly seem to serve to keep a walled garden around the sciences, and this was cemented when I eventually went for a master's degree and I was expected to do this as well in compsci to get a better grade even if there is no advantage. If we could just write what we mean, I suspect people would find that more useful even if it makes it look less cultivated and more mainstream.
(To be clear, this is not criticism on the person I'm replying to, but split between the author of this specific title and most of the sciences as a whole because it's a universally supported barrier (if only ever implicitly), aside from a few science communicators.)
It is just a convention, specifically for interpreting and presenting experimental results. We also use sigma to represent standard deviation in other contexts, of course. Sometimes it represents Pauli spinor matrices. Sometimes it's an index for spacetime tensors.
Life would be hell for any practitioner without single-letter abbreviations. In fact, we like them so much, that's why we adopted the greek letters (we ran out of alphabet). And, for better or for worse, convention runs deep in scientific literature. In practice it reduces a lot of redundancy, makes it more efficient for researchers to skim and understand results. But the cost is a years-long learning curve to break into any scientific field's literature.
FWIW, the linked article is from the journal Science, which is a technical publication. Often "sigma" is omitted in sci-comm articles, or at least is translated for the reader. They will say something like "there is a one in X million chance this is a fluke".
Looking up from my screen filled with sanity saving conveniences like having to type /sigma to get a really smart looking lower case greek character to display so the masses can't make sense of my math.
Much of these formulae used to be handwritten, and still are handwritten at a blackboard / whiteboard in physics classes.
It's much easier to draw a fancy symbol by hand than write several simple letters quickly and legibly, and it also takes much less space.
We've been having the privilege to write using computers for last 20-25 years, when PCs became widespread, relatively cheap, and running good enough software. And this is outside the lecture hall settings anyway.
> Much of these formulae used to be handwritten, and still are handwritten at a blackboard / whiteboard in physics classes.
That is honestly the best argument I've ever heard (you're the first I see mention it). With as much as I hate writing rather than typing, I can see the point there actually. Maybe this practice is not as wholly stemming from elitism as it first seemed.
If you don't know what the Greek letter sigma means, you aren't going to know what the phrase "standard deviation" means, either. The notation isn't the issue. The issue is you can't fit stats 101 into a headline, and there's no getting around that.
When you see a symbol you don't recognize, like 'σ', you can just paste it into google and it'll tell you.
I personally don't see why greek letters are such a big sticking point, there's only 24 of them, and unlike Greek children you don't have to learn them all in one go.
Don't agree with this, it took a few weeks of physics classes to get used to using greek letters as variables, and without them you'd drown in re-used letters.
> it took a few weeks of physics classes to get used to using greek letters as variables
That's a very small price if you're actually involved with physics regularly, but HN is a relatively mainstream place.
I had physics for 4 years in school but this wasn't part of the curriculum. At some point I asked why we were told (seemingly-to-me falsely) that there were only 3 phases of matter when on google videos I had seen something about superfluidity. The teacher made a joke about my stumbling over that word and then the buzzer went so... that's the kind of physics we had.
And that's for someone who went to school in one of the richest (GDP per capita) and most-developed (HDI) countries in the world. I don't know what it's like for anyone tuning in from a less well-off place, or for someone who had physics decades ago without refreshers (for me it's only a bit more than one decade now).
Something tells me I should have looked for a statistics paper that replaced GDP and HDI with some random symbol and used that instead. That's the kind of thing you're promoting and I just don't see why. TLAs aren't everything but they're better than single letters.
> without them you'd down in re-used letters
eh, literally the opposite? Using (abbreviated) names you'd not drown in re-used letters.
well the abbreviated names include the letters, you can get in trouble when questions tend to have many symbols appended together.
I should clarify, though, that I was thinking of college physics classes, which are definitely more mature, both about exploring new knowledge instead of memorizing facts, and about learning to actually speak in the experts' language.
Using symbols for common concepts without defining them is, however, absurd. (Not counting a few -- c, e, hbar, m, maybe q?)
I disagree with the parent post about the use of Greek letters, but it seems like a valid point worth of discussion. Certainly in the spirit of HN.
I’ve seen an increasingly worrying trend of using downvotes to voice disagreement, rather than as the intended purpose as a kind of crowd-based moderation. And before anyone lambasts me for complaining about downvotes, I’m complaining about the trend, where the above comment is just a exemplar.
> I’ve seen an increasingly worrying trend of using downvotes to voice disagreement, rather than as the intended purpose as a kind of crowd-based moderation.
Actually Paul Graham did intend downvotes to express disagreement. The theory was that if people could express disagreement by downvoting there would be less people posting insubstantial comments to disagree.
Just like I code in a programming language, I'm not proposing to turn everything into English prose. Rather, using (abbreviated) names for variables and perhaps a bit more common language in papers (but that's maybe a separate topic).
Also I'm not sure what you mean by "back", is it referring to what we iirc called story exercises in Dutch primary school ("Jan goes to the store and buys seven ladders, then sells three..." etc.) or was this a thing a few hundred years ago or so?
Yeah, imagine making a calculation or transforming a complex formula with words and full sentences. Algebraic notation was a pretty big invention for a reason. For instance, the reason why we use single letters and indexes is so it's not confused with products. Try to write and manipulate the Schrodinger equation with words. Imagine solving the hydrogen atom, it already takes like 50 pages with algebraic notation...
And I don't really understand the "I didn't do math and Greek in School". I barely had a foreign language, but if you're actually learning the concept you memorize the letter as well. You can't understand what a wave function is and then not remember that its symbol is Psi. And if you don't know what a wave function is, it won't help to write derivate_2nd_order(waveFunction, time).
EDIT: obviously we're not talking about stories to teach newcomers, you're talking about writing equations in scientific articles and books with words.
I guess its all coming down to mnemonics, aiding our memories and communicating.
Sure this is the "state of the art", but despite the fact that pure language notations might be even worse, i cant help to think that people thinking like the parent might find something even better.
Maybe something inspired by braille notation or something that is invented while trying to understand how our brain works (just speculating here) will be even more expressive.
I actually like seeing an adult be bothered by the fact that the same symbols that turn science more expressive are also the reason that there's a big ladder for newcomers to understand whats being expressed given its all very arbitrary (someone in
the XVI century choose a random greek letter to represent X).
Imagine how much science would improve with more "brain power" being also able to try to solve some problems given there are less arbitrarity..
Anything other than a single letter variable with at most subindexes, bold, upper/lower case simply doesn't work in maths and science. And because we only have 26 letters, you do have to go to Greek.
Actually, that might be a good exercise: try doing some moderately abstract equations with variable names such as you'd write in a programming language and you'll find yourself shortening them pretty quickly. We literally do it sometimes when modeling an equation for a new domain: we start by writing words and at the end of the blackboard they already became a symbol.
It’s funny; when I was reading the HN comment I was just saying to myself “it would be so nice if the person had used the symbol for phi (φ) rather than spelling it out”. So my reaction was the opposite to yours since my brain comprehends that notation more easily than words.
Using symbols reduces the amount of text your brain has to parse. It makes it much easier to reach consensus on a shared understanding of things. The price to pay is to learn this new notation or language.
Yeah, don't we have the mantra "less code is better code" or something like that? Too many verbosity and our brain turns off. I did type it out the Psi because I couldn't be bothered to type it in mobile, but yeah, it's so weird.
Chinese people learn dozens of thousands of ideograms, I am pretty sure the problem with understanding the science has nothing to do with a few Greek symbols.
The price that was paid for learning the language of math is something that everyone who needs to work with these things are happy to pay. If the notation doesn’t make sense, it’s discarded for ones that does.
It may not make sense for layperson but that’s not really the audience.
Writing formal mathematics as programming languages is basically what automated theorem provers do. The proofs are mostly unreadable.
Mathematical notation really isn't that hard as long as you treat it as its own thing and learn it properly rather than trying to use a likely imperative model of computing programming as a reference point.
I've heard my complaint be called an excuse before, but consider that it's also the first barrier. Not a big one when you already decided to take a course, when you're seriously interested, when you just look them up and soon enough you know the conventions in the field. That's different from casual reading on HN, though. And tell me, are scientists not a class that is looked up to where you live? The common Joe might not explicitly say so, but if someone is a "scientist" then you don't expect them to be stupid or muck out stalls; they do have some real status. This doesn't come from doing things that seem like any common Joe can do it, yet a lot of the work is just that. Once you start paying attention to how often "new research" in the news amounted to a big survey and very basic statistics, or playing around with a Kinect in a train station to learn about walking patterns, it doesn't seem so different from what regular HN readers do for a living. If you take the paper behind such a survey, it'll turn out to be full of complexity that is a lot harder to get through than necessary. It could be a lot more accessible, but then they would lose status.
It seems to me that brevity is the real excuse here. Moreover, if it were just about symbols but papers were otherwise accessibly written (to reasonable extents, obviously), that would be different still. This is not the case.
Appearances are probably also important for funding. I'd bet that if you submitted same proposal twice, once phrased in a convoluted way and once phrased in a "we're gonna blow up some material multiple times and see how far the shards fly" style, a number of times to independent funding committees, there would be a statistically significant correlation with which proposal gets funded.
Why does opaque bad research form psychology, social sciences or paleoclimatology constantly gets media attention and support? I absolutely do not believe it has anything to do with notation.
And let’s be real. If you couldn’t understand sigma notation in school, the chances that you would comprehend complex science are very low no matter what kind of verbiage or, as it often would be more apt to say, verbal garbage it is wrapped in.
I absolutely agree with you that oftentimes bad research is disguised with ten dollar words. And oftentimes it is disguised with convenient agenda (no matter how true or good this agenda is by itself). But I don’t believe it has anything to do with Greek letters.
A measurement being 7 sigma out would still be Chebyshev bounded by 1/7^2 ≈ 0.02
I.e. the probability of it being ≥7 sigma out is interestingly at most 0.02.
Basically, the result of an experiment has to be boiled down to a single numerical value, called the test-statistic. Typically the test-statistic is a (log) likelihood ratio. It is the distribution of the t.s. that must be considered when determining the significance of a measurement. Obviously the measurement itself only gives you a single value of the t.s., so you need to know the distribution to ask "does this result seem significant?". This is done by considering all the factors of random variation (statistical and systematic) that could have an effect on the t.s. Often, the distributions of these individual random factors are assumed to be Normal, but the resulting distribution considering all of their conspiring effects is very seldom normal distribution. Even in the central limit theorem, I think the distribution of the LLR ends up being something like a noncentral chi^2 distribution.
Especially since, isn't this an average / error of a mean estimate? So even if individual observations are non-normal, this would be a perfect place for Central Limit Theorem.
None of those things matter to the central limit theorem.
If I have IID observations with finite 2nd moment (variance), then their average will pretty quickly converge to a Gaussian distribution. And I can relax a lot of this and still recover a variant of CLT.
Of course maybe the calculation is different, eg it’s not like there are N independent observations, but rather some other complex condition solved for the mean estimate.
Also not knowing anything about this topic, I'd assume it wasn't normal because we're talking about mass close to zero, and mass must be greater than zero.
The mass in their result is 80433.5±9.4 MeV/c^2. The result of the experiment is a Gaussian like distribution. If you consider a Gaussian distribution with μ=80433.5 and σ=9.4, the probability to get a result that is less than 0 is 4E-15899105.
Yes. The meaning of 7 sigma is: It's very very unlikely that this is a statistical fluke, it must have a different reason (new physics, systematic error, ...)
When you look at the graph at the bottom, several independent measurements have non-overlapping error bars, and are even on opposite sides of the Standard Model prediction. So, yeah, somewhere along the line there've been bad measurement errors...
Since error bars are typically +-1 sigma, you expect about 1/3 of all measurements to be further away from the true value than the error bar, if all error estimates are correct, and uncorrelated. That's actually a check a lot of doctored data fails.
This is why these measures have to be taken with a grain of salt (but are still useful).
Probability is subjective, in this case because it's dependant on the design of the experiment / quality of the analysis of that experiment to determine a p-value of a given result.
The book "Bayesian analysis in high energy physics" is a short and sweet introduction. If I got the title wrong I'll update it later.
A fair dice roll can only have positive values {1,2,3,4,5,6} but it has a clearly defined std deviation: sqrt(105/36) -- there's no clear reason this isn't the 'best choice' that's just a case of application.
You can calculate the mean μ and the standard deviation σ of a dice roll. You get μ=3.5, σ=sqrt(105/36)~=1.707... . It's not very similar to a Gaussian, but sometimes these numbers are useful anyway.
It's more interesting if you calculate the distribution of the sum of rolling 100 dices. It's easy to calculate, becuase μ=100*3.5=35, σ=sqrt(100*105/36)~=17.07... But now the distribution is very similar to a Gaussian with μ=100*3.5=35 and σ=sqrt(100*105/36)~=17.07... https://en.wikipedia.org/wiki/Central_limit_theorem They are not equal because the sum of the roll of 100 dices is bounded between 100 and 600 and the Gaussian is not bounded. For most applications, you can just use the Gaussian instead of the exact distribution.
GP is probably referring to the coefficient of variation, sigma/mu (standard deviation divided by mean), which normalises out for example the unit of measurement.
However, the 7 here is basically (x - mu)/sigma, so it is normalised (in that sense), anyway.
No, I think the problem (in principle) is that "standard deviation" has a special meaning for Gaussian distributions, which extend to infinity in both directions. A quantity that has a fixed range has most likely an asymmetric distribution, so one would expect an asymmetric error bar as well. But for a sigma<<the value, it's often not a big concern.
A good example is efficiency measurements. I can't count how often I have seen students say something like: Our detector is 99%+-3% efficient. Obviously a detector can't be 102% efficient.
> "standard deviation" has a special meaning for Gaussian distributions,
I have a master's degree in statistics and this is the first I'm hearing about it.
> Our detector is 99%+-3% efficient. Obviously a detector can't be 102% efficient.
In the absence of any other context I'd guess that they're using an approximation to a confidence interval that might be perfectly fine if the estimated value was nearer the center of the allowable range.
Well, special in two senses: First, in the canonical formula for Gaussians, sigma appears directly. For the case at hand, the confidence limits associated with 1 sigma, 2 sigma etc. in physics match exactly the area under the curve for a Gaussian integrated +- said sigma around the mean. That's were that connection actually comes from, and a physicist will always think: Within 1 sigma? That's 67%.
Hearing 99+-3% is a very strong indication that the person used an incorrect way to determine the uncertainty, most likely by taking the square-root of counts. But you are right, if the efficiency would be around 50%, that approximation is not so bad.
What's wrong with saying "Our detector is 99%+-3% efficient," if they are giving the output of some procedure that constructs valid confidence intervals? The confidence intervals will trap the true value 95% of time (or whatever the confidence level is). If it does what it promises to do, I don't see the problem.
Because a 99+3=102 is not a valid upper interval bound. You cannot have >100% efficiency for a detector. Also, your expected value cannot be centered. So maybe 99+1-3 is a valid range (but I would be very suspicious if the bound includes 100%)
I agree 102% is not a possible value for the efficiency of the detector. But if the confidence interval traps the true value of the efficiency 95% of the time upon repeated sampling, what's the problem? That's all that's required for a confidence interval to be valid. Some CI constructions do in general give intervals that include impossible parameter values, but if they contain the true value 95% of the time, there's no issue. The coverage guarantee is all that matters.
(One should not confuse a CI with a range of plausible values, in other words.)
Ok, true, in that sense, it's fine. However, in 100% of cases I have observed so far (and they were far too many), it means that the person who gives such a result used sqrt(counts) as the error estimate, and that's not correct -- not only for the upper bound, also for the lower bound.
https://non-trivial-solution.blogspot.com/2022/04/do-we-have...