> Can you be specific about what kind of (productive) gatekeeping the society brings to the table that is not and cannot be fulfilled by a peer review process outside of their purview?
There are certainly other ways of getting to the same thing, but what I think a peer review process in isolation lacks is a sense of trustworthiness of the peer review.
I.e. as a layperson, I'm not particularly qualified to gauge the quality of an arbitrary paper. I am implicitly not qualified to gauge the quality of peer reviews, because I can't gauge the paper itself.
I can defer to the opinion of experts, though. These societies unify the diffuse expertise of who knows how many people. I.e. I don't work in sociology, so I have no idea who is a reputable sociologist, but I have some degree of confidence that the peer reviewers they choose aren't total quacks.
It's a useful heuristic for non-experts. I don't know anyone or anything in rocket science, but I can defer to what NASA says. I'm not a marine biologist, but I can defer to the NOAA. Etc, etc. There's value in that heuristic, even if it's not perfect.
> I have no first-hand experience, but my intuiting would be that a lot of the gatekeeping these societies do is focused on reputations, organizational hierarchies, and funding considerations.
This is indirectly useful to my thinking. The society is focused on funding and income, which is largely predicated on them being trustworthy (theoretically), which gives them an incentive to provide quality peer reviews. I would imagine a society with 0 standards loses subscribers pretty quickly.
arXiv could certainly do the same thing, but my impression is that arXiv is more focused on making papers available than being an authoritative entity on anything. "This paper is on arXiv" doesn't imply the same level of trust that being in a journal does. They derive value from being open rather than from being trustworthy.
arXiv definitely has value as well, I don't mean to imply that it doesn't or is bad in some way. It just has different goals, from my limited knowledge. It's likely invaluable to people in their field, especially for those qualified to gauge the trustworthiness of an individual study.
arXiv could definitely build some kind of trust system like that if they were interested, but I really think their primary audience is experts who don't have a need for a trustworthiness score simple enough for a layman.
There are certainly other ways of getting to the same thing, but what I think a peer review process in isolation lacks is a sense of trustworthiness of the peer review.
I.e. as a layperson, I'm not particularly qualified to gauge the quality of an arbitrary paper. I am implicitly not qualified to gauge the quality of peer reviews, because I can't gauge the paper itself.
I can defer to the opinion of experts, though. These societies unify the diffuse expertise of who knows how many people. I.e. I don't work in sociology, so I have no idea who is a reputable sociologist, but I have some degree of confidence that the peer reviewers they choose aren't total quacks.
It's a useful heuristic for non-experts. I don't know anyone or anything in rocket science, but I can defer to what NASA says. I'm not a marine biologist, but I can defer to the NOAA. Etc, etc. There's value in that heuristic, even if it's not perfect.
> I have no first-hand experience, but my intuiting would be that a lot of the gatekeeping these societies do is focused on reputations, organizational hierarchies, and funding considerations.
This is indirectly useful to my thinking. The society is focused on funding and income, which is largely predicated on them being trustworthy (theoretically), which gives them an incentive to provide quality peer reviews. I would imagine a society with 0 standards loses subscribers pretty quickly.
arXiv could certainly do the same thing, but my impression is that arXiv is more focused on making papers available than being an authoritative entity on anything. "This paper is on arXiv" doesn't imply the same level of trust that being in a journal does. They derive value from being open rather than from being trustworthy.
arXiv definitely has value as well, I don't mean to imply that it doesn't or is bad in some way. It just has different goals, from my limited knowledge. It's likely invaluable to people in their field, especially for those qualified to gauge the trustworthiness of an individual study.
arXiv could definitely build some kind of trust system like that if they were interested, but I really think their primary audience is experts who don't have a need for a trustworthiness score simple enough for a layman.