Their own judgment on reading the paper, talking to other scientists they know, going to conferences, watching or participating in online discussions, etc.
I used to be on a health list that was ostensibly looking for a "cure" or treatment for cystic fibrosis. They frequently posted links to abstracts of papers and discussed it on list and I had mixed feelings about that. I often didn't understand the medical terminology and it was just a blurb and yadda.
At some point, I talked on the phone with a guy with a PhD that I knew through an entirely unrelated email list. And he told me to not bother to read the abstract if I couldn't get hold of the full paper because it often said something different from what the paper said. I was relieved to hear this and not at all surprised, so I quit reading those abstracts and trying to pretend to participate in those discussions or whatever.
Some years later, a woman on that same list offered to co-author a paper with me and give me credit and yadda. And then she changed her mind and began trashing me on list and unsubscribed.
Her son died of CF (at the age of 17) years and years before and she was in search for some pill to make it all better, obsessed with finding a cure because of his death. I use a lifestyle based approach. She was a smoker. I think the implication of my ideas is that it's partly her fault her son died. And I think her guilt over that possibility is why she's obsessed with finding an answer, but it has to be an answer that somehow absolves her of guilt and doesn't in any way suggest it's her fault. It needs to do the opposite for her to be satisfied.
I was homeless at the time that she offered to write a paper with me. Once she kicked me to the curb, I didn't really have any place else to go.
Anyway, regardless of your process, you need some signals of some sort. I relied on the opinion of the guy with the PhD in part because he had a PhD. And I think my life has been better for that.
It's good to have a so-called "bullshit detector" but one of the ways we filter information is by relying on our understanding that some people can be believed even if we don't understand what they are saying because it is over our head.
> our understanding that some people can be believed even if we don't understand what they are saying because it is over our head.
First, while this might apply to lay people trying to evaluate scientific claims, it should not apply to scientists trying to evaluate scientific claims in their own field.
Second, peer review does not even pretend to be the kind of filter you are talking about. The fact that a paper has been peer reviewed is not evidence that it is correct or that its claims should be believed. Indeed, many peer reviewed papers later turn out to be wrong. (For example, all of the papers that were part of the "replication crisis" that got a lot of recent press were peer reviewed.) All peer review signals as a filter is that someone presumed to have relevant expertise thought the paper was worth publishing. I think recent experience has shown that this signal is virtually worthless now, though it might have had value in the past.
My general understanding is that most people do take peer review to be such a filter.
I'm "just a lay person" but I'm also someone solving a real serious scientific problem and I'm dependent on trusting that people with PhD's who answer questions of mine about cell biology or whatever can be trusted because, unlike me, they had the capacity to pursue a PhD. I simply can't. I'm too busy solving a scientific problem like my life depends upon it because it does.
And I'm sure this is a really stupid to comment to leave. I've written several drafts and I don't know a good way to engage your comment.
I primarily replied to the comment about "infinite time" because I sort of have "infinite time" in that I'm an unemployed bum with a bit of unearned income who also occasionally tries to earn a few bucks and whose primary focus is on understanding a scientific question so that I don't die and all that.
And as someone with "infinite time" to devote to reading articles and what not because reading those articles and what not is a higher priority than almost anything else since my life depends upon it, I find that I still need filtering mechanisms.
Now that I've dug my grave deeper, I shall endeavor to tear myself away from the computer and try to get a little shut eye.
> My general understanding is that most people do take peer review to be such a filter.
If they do, then they're wrong, since peer review explicitly disclaims any indication that a paper is correct or should be believed. And this is supported by the fact that a large fraction of peer-reviewed papers turn out to be wrong.
> I'm also someone solving a real serious scientific problem and I'm dependent on trusting that people with PhD's who answer questions of mine about cell biology or whatever can be trusted because, unlike me, they had the capacity to pursue a PhD.
It depends on what questions you're asking them and what their basis is for their answers. If their basis is solid experimental knowledge of whatever aspect of cell biology you're asking about, then yes, their answers are probably trustworthy. If their basis is speculation about something we don't have much experimental knowledge of, then no, their answers are probably not trustworthy. In either case, you're not going to find that out by looking at whether what they're saying has been peer reviewed. You're going to find out by asking them how they know the things they're telling you and critically evaluating their answers.
> I'm too busy solving a scientific problem
> I sort of have "infinite time"
You can't be "too busy" if you have "infinite time". It's just a question of how you choose to employ your time. It would seem to me to be a better idea to employ it in becoming an expert yourself, so you can critically evaluate what other experts are saying, than to look for a shortcut filter, whether it's peer review or anything else. Asking other experts questions can certainly help with that; but it might be helpful to view that not as looking for answers from them, but as using what they say to help you build your own expertise.
It's just a question of how you choose to employ your time.
I'm going to try one last time to clarify what I meant and then I am done here because this is going all kinds of weird places I never expected.
I'm medically handicapped. I have a very serious and deadly condition and failing to die of it takes all my time.
That fact has prevented me from finishing my BS in Environmental Resource Management, a major chosen to further my career goals. My interest in and need for information on cell biology is driven by the fact that I was diagnosed somewhat late in life with a very serious genetic disorder.
So I wouldn't be interested in pursuing such knowledge if I weren't medically handicapped. I would be a city planner or something, like I wanted to be.
The fact that my medical condition has stomped my life into the ground and made me unemployable is why I have a lot of time on my hands and I choose to prioritize learning knowledge pertinent to addressing my health issues under circumstances where, no, it's simply not possible for me to do something like pursue a PhD in biology.
And I am getting results, so your opinions about how I should spend my time aside, I'm fairly satisfied with the choices I am making (all things considered), though I wish I didn't have this personal challenge. I wish I were healthy and life had gone differently. But it didn't.
I'm very sorry about your condition and that you're having to go through all you are going through.
> it's simply not possible for me to do something like pursue a PhD in biology
Becoming an expert is not the same as getting a credential. It's perfectly possible to become an expert in whatever particular condition you personally have, without getting a PhD in biology, or indeed any relevant degree. Indeed, since you say you are getting results, I suspect you are becoming an expert in your particular condition, whether you planned to or not. My wife has a chronic illness and she has been forced to become an expert in it (with no relevant degrees at all) even though she would much rather have spent the time on other things.
> I am getting results
I'm glad you are; that's the most important thing.
