The zoo of frequentist statistical tests in the article tend to give results that are easily interpreted in misleading or wrong ways (but kudos to the author to prominently include Bayesian methods). Uncritical usage of these techniques is the #1 reason behind the replication crisis in science.
Bayesian methods should be preferred as the default.
I agree that we should take a harder look at Bayesian methods, but I don't think that sweeping comments like this do much to advance the discussion. Any circumspect person will naturally be suspicious of something that is held up as a panacea.
It's much more constructive (and compelling) to call attention to specific problems with how frequentist methods get used in practice, and talk about how Bayesian methods can help with that problem.
Whether or not they should be preferred, there is massive institutional and knowledge-related inertia behind frequentist approaches. Rather than throw out the baby with the bathwater, throw out all legacy code and retrain millions of scientists in Bayesian, we should consistently urge better use of frequentist approaches.
To transition the next generation of scientists to Bayesian will require universities/stats depts/other depts to do so. Have you seen this to be the case right now? Because in my experience of a few different research settings, almost no one I know, of around 50+ researchers, except those passionate about statistics (very rare person indeed) is using Bayesian methodology.
How to improve Bayesian knowledge in the world? This has its own set of challenges, as Bayesian thinking is arguably more mathematically challenging for most numerophobic people. IMO, it will take multiple generations of effort to transition everyone over.
> ... We should consistently urge better use of frequentist approaches.
Many "frequentist" methods can be rephrased as heavily-simplified special cases of the Bayesian approach. Most easily, by assuming a flat prior distribution for the relevant parameters (which is basically an artifact of the parameterization you choose anyway) you can assert that any MLE-based approach will yield a correct Bayesian posterior mode, which in turn is an optimal Bayes-estimator, assuming a constant loss function.
The limits of this whole approach are fairly clear, of course - for one thing, Bayesian stats is generally based on working with the entire posterior distribution, not merely a point estimate of it - and there are good reasons for this. But the basic point stands, and many "tweaks" on the basic frequentist approach can in turn be justified in Bayesian terms. This is not to say that frequentist statistics is all that we'll ever need, but merely pointing out that this whole argument of "we should work with what we have, and focus on making sure that frequentist approaches are put to good use" actually sounds rather vacuous. It's correct in a very limited sense, and hardly something that Bayes proponents are unaware of!
Frequentist stats just aren't that useful for experimental results because they don't tell folks what they need to know. Even ignoring that, researchers understanding of even basic concepts like P-values is so bad that one might as well throw out everything, baby and all. In one survey [0], 94% of Spanish academic psychologists believed in some form of the inverse probability fallacy. A similar survey in Italy showed similar results.
Anecdotally, I can definitely say that the vast majority of American physical scientists I've talked to about this also fell prey to this. With the inverse probability fallacy, one attaches a Bayesian interpretation to P-values, so I don't think training scientists in Bayesian stats would be harder than getting them to use frequentist stats correctly (which is a minefield in comparison).
> Uncritical usage of these techniques is the #1 reason behind the replication crisis in science.
Do you have any evidence for this? My gut feeling is representativeness is a much bigger problem. Like you do a study on people of the same age race class during the same zeitgeist and then generalize to an eternal law.
The next year fashion has changed and the replication comes out all different.
Representiveness is an issue, but I think the most important are the multiple comparisons problem and the base-rate fallacy. It's commonly the case that a P<0.05 positive result has more than a 50% chance of being wrong. The replication studies I've seen mainly point to these factors being the issue. The fundamental thread is neglect of prior information.
The point estimate is base on sample space where as the parameter distribution is base the parameter space.
I think learning both is good and people who pit those two school of statistic against each other are a bit too zealot. They're both tools and use them as needed and when one is easier than the other.
Well, events over trials is frequentist definition. Unbiased estimate with nice statistical properties.
You know that it's an underestimate because you have prior knowledge about cancer incidence. Bayesian methods let you incorporate that knowledge into estimation process, pulling the estimate up towards a more realistic value.
Bayesian methods should be preferred as the default.