The LHC has multiple teams that will replicate an experiment. The facility is so big that you have different groups. You also have different detectors (such as ATLAS and CMS) where you replicate the experiments because each one has a bit of difference. And except for the very high energy stuff, other labs around the world can reproduce the results. Not to mention that particle physics has a 5 sigma significance threshold...
But for a lot of this, what happens is that they make the claim, they contact peers, send out the data, and have their peers confirm. You ever wonder why experiments like these have hundreds of authors[0,1]? That's because that shit is peer review. [0] has literally 8 pages of authors, which is in the format of first initial, last name. There's also 5 pages of university/lab affiliations.
Everything you're talking about here is done outside journals. And guess what, there are two versions of the arxiv work for [0], July 31st 2012 and August 31st 20122. Guess when it was "published". September 17th of that year. The announcement? July 4th! Btw, the journal is showing v2.
This is how big science is done. The replication and verification is happening intra-paper. You of course have to trust the machine, but that's true for everyone and a limit that can't be bypassed until a later date when a new one is built. Which frequently the first tasks are to confirm other observations, such as the JWST confirming exoplanets that were previously observed.
So to sum up: Replication happens, on big important experiments, outside journals
> [reproduction] is extreme reductionism
I'd still argue, with _aavaa_, that reproduction is the only currency that matters. Your experiment isn't worth shit if it can't be reproduced (i.e. it is indistinguishable from fraud)
> LHC results cannot be replicated
Most of the work is replicated, prior to publication.
> JWST cannot be replicated.
Observational data is confirmed. And JWST replicates its predecessors (an important part of its mission). (I'll even add LIGO in here and repeat the LHC and JWST comments)
> You know who will be able to tell you if the experiment description makes sense and has chances to be replicated if you ask them? Peers.
Everyone agrees with this. We are just saying the reviewers aren't necessarily your peers and the people you should be asking.
> because nobody likes to do peer review.
Lots of people like to do peer review and it happens all the time. If you're talking about reviewing for a journal, then yes I agree with you. But if you're talking about replicating work, building on, doing it to learn, or for fun, then this happens all the time. In fact, grad students do this professionally. How many works did you rebuild and try to replicate?
> I'm yet to see any proposal for peer review without editorial invitations which has at least theoretical chances to succeed.
You haven't dismissed __aavaa__ or my claims, only asserted they are wrong. If you would like to say why we are wrong, we're clearly open to hearing it and responding. But you've just asked us to trust you.
> So you do mean "reproduction", not "replication".
Reproduction is a form of replication. Replication is a spectrum. Reproduction is the minimum and with a lot between confirming experiments within a different context frame. But I don't think anyone is saying that just running your code constitutes reproduction. I'd say it is confirmation of your code. The source availability is about verification, as in I can probe it and look for errors, not so much reproduction.
But for a lot of this, what happens is that they make the claim, they contact peers, send out the data, and have their peers confirm. You ever wonder why experiments like these have hundreds of authors[0,1]? That's because that shit is peer review. [0] has literally 8 pages of authors, which is in the format of first initial, last name. There's also 5 pages of university/lab affiliations.
Everything you're talking about here is done outside journals. And guess what, there are two versions of the arxiv work for [0], July 31st 2012 and August 31st 20122. Guess when it was "published". September 17th of that year. The announcement? July 4th! Btw, the journal is showing v2.
This is how big science is done. The replication and verification is happening intra-paper. You of course have to trust the machine, but that's true for everyone and a limit that can't be bypassed until a later date when a new one is built. Which frequently the first tasks are to confirm other observations, such as the JWST confirming exoplanets that were previously observed.
So to sum up: Replication happens, on big important experiments, outside journals
> [reproduction] is extreme reductionism
I'd still argue, with _aavaa_, that reproduction is the only currency that matters. Your experiment isn't worth shit if it can't be reproduced (i.e. it is indistinguishable from fraud)
> LHC results cannot be replicated
Most of the work is replicated, prior to publication.
> JWST cannot be replicated.
Observational data is confirmed. And JWST replicates its predecessors (an important part of its mission). (I'll even add LIGO in here and repeat the LHC and JWST comments)
> You know who will be able to tell you if the experiment description makes sense and has chances to be replicated if you ask them? Peers.
Everyone agrees with this. We are just saying the reviewers aren't necessarily your peers and the people you should be asking.
> because nobody likes to do peer review.
Lots of people like to do peer review and it happens all the time. If you're talking about reviewing for a journal, then yes I agree with you. But if you're talking about replicating work, building on, doing it to learn, or for fun, then this happens all the time. In fact, grad students do this professionally. How many works did you rebuild and try to replicate?
> I'm yet to see any proposal for peer review without editorial invitations which has at least theoretical chances to succeed.
You haven't dismissed __aavaa__ or my claims, only asserted they are wrong. If you would like to say why we are wrong, we're clearly open to hearing it and responding. But you've just asked us to trust you.
> So you do mean "reproduction", not "replication".
Reproduction is a form of replication. Replication is a spectrum. Reproduction is the minimum and with a lot between confirming experiments within a different context frame. But I don't think anyone is saying that just running your code constitutes reproduction. I'd say it is confirmation of your code. The source availability is about verification, as in I can probe it and look for errors, not so much reproduction.
[0] page 25 https://arxiv.org/abs/1207.7214
[1] appendix C page 41 https://arxiv.org/abs/2207.00043