The journals take works others have done (work often paid for with public money) and get unpaid volunteers to peer-review it. Next they require the authors to sign away their copyright to it. And then they turn right back around and sell it to very same universities where it came from for thousands of dollars a year.
And for what? So they can have have profit margins rivaling big tech? (Their major expenses are lawyers and server costs for pdf documents…)
And why do researchers publish there? Because they need to in order to keep their jobs, or to get funding. You need those high-impact-factor and prestigious journals. And so you sign away your research.
Most 1st world researchers don’t care or don’t know about the costs. But most of the world is not so lucky. Yes you could go around begging the authors for a free copy (but definitely don’t go to scihub or library genesis and download them for free, that would be illegal…).
The “space for improvements” starts with burning down the journals and scattering their ashes among the fields to fertilize new science.
Again, we had scientific progress (and problems with it) before the current system.
The only currency that matters is if your results can be reproduced, as as far as I'm aware the overwhelming majority of journals make no attempt at reproducing the results before accepting it. Hell, it doesn't seem like they even make an attempt at make string there's enough information provided to even attempt to reproduce it.
When I come across a paper that does any sort of modelling, I simply assume I won't be able to reproduce it if the code isn't included. They all seem to leave out important information.
> Again, we had scientific progress (and problems with it) before the current system.
Again, the scale and accessibility of science is different now and then. We had commute before steam engine, it doesn't mean we can get rid of all engine-based transportation now without consequences.
> The only currency that matters is if your results can be reproduced
This is extreme reductionism. LHC results cannot be replicated unless you have multi billion particle accelerator in your backyard. JWST cannot be replicated. Many other large scale experiments are nearly impossible to replicate. In fact, these are often considered "important" ones. And this not necessarily due to price, but rather combination of price, unique expertise and willing to spend effort.
You know who will be able to tell you if the experiment description makes sense and has chances to be replicated if you ask them? Peers. Peer review at will never worked (and there were attempts) and never will because nobody likes to do peer review. I'm yet to see any proposal for peer review without editorial invitations which has at least theoretical chances to succeed.
> I won't be able to reproduce it if the code isn't included.
So you do mean "reproduction", not "replication". This is very low threshold and definitely not "only currency that matters". If I send you the code that alters the results somewhere deep inside, you'll be able to reproduce them.
> Many other large scale experiments are nearly impossible to replicate.
Thankfully, the LHC is not what all of science looks like. There is a great deal of science that can be reproduced without needing billions of dollars.
> If I send you the code that alters the results somewhere deep inside, you'll be able to reproduce them.
But the current status quo for most papers is that you don't even have that. If you public put out code that manipulates data in this way then it can be scrutinized.
The alternative is to attempt to replicate the results, and waste months of time trying to guess what the authors did (since they won't tell you), and in the end never being able to reproduce it, not knowing if it was because key information was missing or because it was falsified.
That is, if you can spend the time attempt to reproduce the results, rather than publishing your own novel and revolution results. And if you'll find a journal who'll be interested in publishing your failure to replicate.
I've tried emailing peers for code and they would ignore me. I then got my advisor to talk to their advisor because I couldn't replicate their program and I got code sent to me. It was nowhere near what was explained in the paper and didn't give the results. Nothing happened. No retraction, no nothing. Just made it harder for me to publish my paper because it is very hard to beat results that were made up.
So what I'm saying is, I agree with your point here. The proof is in the pudding. Journals are convincing everyone that anything not in them is false and garbage but once garbage gets into them it creates and impossible bar to pass. Now repeat this for 50 or so years in a highly competitive space and what do you get? Well I think we all know the old saying: garbage in...
The LHC has multiple teams that will replicate an experiment. The facility is so big that you have different groups. You also have different detectors (such as ATLAS and CMS) where you replicate the experiments because each one has a bit of difference. And except for the very high energy stuff, other labs around the world can reproduce the results. Not to mention that particle physics has a 5 sigma significance threshold...
But for a lot of this, what happens is that they make the claim, they contact peers, send out the data, and have their peers confirm. You ever wonder why experiments like these have hundreds of authors[0,1]? That's because that shit is peer review. [0] has literally 8 pages of authors, which is in the format of first initial, last name. There's also 5 pages of university/lab affiliations.
Everything you're talking about here is done outside journals. And guess what, there are two versions of the arxiv work for [0], July 31st 2012 and August 31st 20122. Guess when it was "published". September 17th of that year. The announcement? July 4th! Btw, the journal is showing v2.
This is how big science is done. The replication and verification is happening intra-paper. You of course have to trust the machine, but that's true for everyone and a limit that can't be bypassed until a later date when a new one is built. Which frequently the first tasks are to confirm other observations, such as the JWST confirming exoplanets that were previously observed.
So to sum up: Replication happens, on big important experiments, outside journals
> [reproduction] is extreme reductionism
I'd still argue, with _aavaa_, that reproduction is the only currency that matters. Your experiment isn't worth shit if it can't be reproduced (i.e. it is indistinguishable from fraud)
> LHC results cannot be replicated
Most of the work is replicated, prior to publication.
> JWST cannot be replicated.
Observational data is confirmed. And JWST replicates its predecessors (an important part of its mission). (I'll even add LIGO in here and repeat the LHC and JWST comments)
> You know who will be able to tell you if the experiment description makes sense and has chances to be replicated if you ask them? Peers.
Everyone agrees with this. We are just saying the reviewers aren't necessarily your peers and the people you should be asking.
> because nobody likes to do peer review.
Lots of people like to do peer review and it happens all the time. If you're talking about reviewing for a journal, then yes I agree with you. But if you're talking about replicating work, building on, doing it to learn, or for fun, then this happens all the time. In fact, grad students do this professionally. How many works did you rebuild and try to replicate?
> I'm yet to see any proposal for peer review without editorial invitations which has at least theoretical chances to succeed.
You haven't dismissed __aavaa__ or my claims, only asserted they are wrong. If you would like to say why we are wrong, we're clearly open to hearing it and responding. But you've just asked us to trust you.
> So you do mean "reproduction", not "replication".
Reproduction is a form of replication. Replication is a spectrum. Reproduction is the minimum and with a lot between confirming experiments within a different context frame. But I don't think anyone is saying that just running your code constitutes reproduction. I'd say it is confirmation of your code. The source availability is about verification, as in I can probe it and look for errors, not so much reproduction.
The journals take works others have done (work often paid for with public money) and get unpaid volunteers to peer-review it. Next they require the authors to sign away their copyright to it. And then they turn right back around and sell it to very same universities where it came from for thousands of dollars a year.
And for what? So they can have have profit margins rivaling big tech? (Their major expenses are lawyers and server costs for pdf documents…)
And why do researchers publish there? Because they need to in order to keep their jobs, or to get funding. You need those high-impact-factor and prestigious journals. And so you sign away your research.
Most 1st world researchers don’t care or don’t know about the costs. But most of the world is not so lucky. Yes you could go around begging the authors for a free copy (but definitely don’t go to scihub or library genesis and download them for free, that would be illegal…).
The “space for improvements” starts with burning down the journals and scattering their ashes among the fields to fertilize new science.