I think ideally, universities should have enough professional statisticians and programmers on staff to provide a mandatory but nonbinding review for every paper any researcher at the university decides to publish. They shouldn't act as gatekeepers to publishing, but would provide feedback to the researchers for the benefit of the researchers.
Compelling researchers to release their source code would be the next step, but they'll fight you tooth and nail on that one. Many researchers are in very competitive fields and think that releasing their source code might give some other team a boost. I think this attitude is contrary to the interests of scientific progress, but because it's a matter people have such strong personal stakes in, it could prove a hard fight.
Chemistry departments often have glassblowing technicians. Perhaps this would be comparable.
> I think ideally, universities should have enough professional statisticians and programmers on staff to provide a mandatory but nonbinding review for every paper any researcher at the university decides to publish.
I guess I wasn't clear enough to connect all the dots. These kinds of policies are made by academics. The people with the authority to mandate this are themselves academics who got promoted. The reason the universities will not do it is because of the nature of the people in charge.
Academics don't work for these people. They are these people.
Pretty much the same story with grant approvers and journal editors. Mostly from academia.
The argument to do it is to improve the quality of the research output. But guess who evaluated the quality of the research?
It doesn't matter that you and I know this is an incredibly unreliable way to do research. As long as academics can continue to publish their papers with crappy software practices, they will do so.
As I said, there is a lack of incentive to do this. Who will benefit? Probably over half of researchers do not want someone to find flaws in their methodology. In my discipline, it was common to leave out inconvenient details from a paper. Common enough to the point of psychosis: The researchers convinc themselves that the flaw is not a problem, so it's not even a question of integrity any more. It's poor incentives leading to a few generations of researchers who are trained to be blind.
Compelling researchers to release their source code would be the next step, but they'll fight you tooth and nail on that one. Many researchers are in very competitive fields and think that releasing their source code might give some other team a boost. I think this attitude is contrary to the interests of scientific progress, but because it's a matter people have such strong personal stakes in, it could prove a hard fight.
Chemistry departments often have glassblowing technicians. Perhaps this would be comparable.