There should be Nutrition Facts but for scientific trials. Independent agency just publishing quality assessments of the trial.
This should be an async non blocking evaluation. The statisticians who do it should be anonymous by default. There should be an appeals process for a scientist to explain why an unconventional new method is actually robust.
There should not be a single number published by this process, but rather a list of stats that speak to the overall quality of the trial on many dimensions (power, sources of bias, etc).
Only information that would not be the same on 99% of trials should be written on this label (no sec style everything is a risk word vomit disclosures).
There should not be a pre-emptive application for a label - it can only be gotten after paper submission to reduce gaming.
There should be an independent advisory org that scientists can literally call to ask for advice on structuring the trials. These calls must not be disclosed. Much like farmers can call the government to ask for help on xyz crop problem.
And these labels should never be used as the primary source of punishment. Any and all sanctions/penalties/dismissals must go through a new review process done by a different group.
Any scientist who gets a label in a particular year should be given a vote to review the review agency on several dimensions. These aggregate reviews should be published broadly but not trigger any automatic consequences.
Clear, accessible information is the basis for any self regulating human system. We need more of it in this field.
There's the Cochrane Collaboration. They don't tick off every item on your list but it's fairly close to what you're asking for. It's mentioned in the article as they do a lot of meta-studies. Unfortunately they only started trying to spot fraudulent RCTs in 2021. Also in recent times some people don't like them, because they did a big review of mask studies and found there was no reliable evidence that masks worked against COVID.
Cochrane (formerly known as the Cochrane Collaboration) is a British international charitable organisation formed to organise medical research findings to facilitate evidence-based choices about health interventions involving health professionals, patients and policy makers.[4][5] It includes 53 review groups that are based at research institutions worldwide. Cochrane has approximately 30,000 volunteer experts from around the world.[6]
The group conducts systematic reviews of health-care interventions and diagnostic tests and publishes them in the Cochrane Library.[7][4]
> Also in recent times some people don't like them, because they did a big review of mask studies and found there was no reliable evidence that masks worked against COVID.
Oh.
I quickly found this:
"The new scientific review on masks and Covid isn’t what you think" Kelsey Piper
Cochrane revisit topics from time to time to update their reviews as new studies appear. The question of mask effectiveness was reviewed in the past also. There's an interview with one of the authors of this round's review here:
So, a Cochrane review is a study which synthesises all available studies – all that we can find or identity – on a particular topic. It follows a highly structured format and is always preceded by publication of a protocol. All this is to minimise the bias. Also, it is extensively transparent. In this case we are looking at about 300 pages of review. Now, the review called “Physical interventions to interrupt or reduce the spread of respiratory viruses” is called in code A122 for short and I will be using that acronym simply because it is just too long a title. So the protocol was first published in 2006 and then the first version was published in 2007, updated in 2009, 2010, 2011, and then 2020, so this 2023 is the fifth update of this review. And the reason why we update the reviews is they are soon out of date if we don’t do that, especially in some fast moving topics.
This update didn't change the conclusions from any of the prior reviews.
Because masks are so politicized there were numerous attacks on Cochrane this time around, though nobody cared in any of the previous rounds. The Cochrane authors are aware of all the criticisms, but there were no justifications found in any of them to alter the conclusions of the review or their procedures for doing them.
> This update didn't change the conclusions from any of the prior reviews. ... The Cochrane authors are aware of all the criticisms, but there were no justifications found in any of them to alter the conclusions of the review or their procedures for doing them.
True. But as noted elsethread, Cochrane is not responsible for others misinterpreting the conclusions.
"The original Plain Language Summary for this review stated that 'We are uncertain whether wearing masks or N95/P2 respirators helps to slow the spread of respiratory viruses based on the studies we assessed.' This wording was open to misinterpretation, for which we apologize. While scientific evidence is never immune to misinterpretation, we take responsibility for not making the wording clearer from the outset. We are engaging with the review authors with the aim of updating the Plain Language Summary and abstract to make clear that the review looked at whether interventions to promote mask wearing help to slow the spread of respiratory viruses."
If science is going to be "self-correcting" then it has to make mistakes in the first place.
These mistakes will happen from the original scientists, they will happen at the stage of editorial boards, they will happen at peer review, they will happen if external third parties start systematically reviewing every RCT.
So Cochran must similarly be scrutinized for their errors, because they will be making them as well.
And that's even before we get to the political factors outside of science misinterpreting complex data for their own purposes...
It's wicked hard just to get reproducible results (one facet of the replication crisis). Much less the challenges you list.
Confusion and miscommunication is the norm. Rising above that takes Real Effort™.
One of my formative experiences was on a team trying to adopt the processes from the book Applying Use Cases. So simple. Like a recipe. Really, what could be more simple?
We had shared purpose. We all read the book (among others). We discussed. We all thought we were good to go.
And then the wheels fell off once real work started. Turns out we didn't agree. On anything. What is "the system"? What level of abstraction are we working at? What does this line (points at diagram) here mean?
Writing this now, experiencing PTSD flashbacks, I can confidently say I would have never succeeded as a scientist.
Vox? I would recommend seeking elsewhere for truth.
For example, after parsing through the ad hominem attacks and nonsense in that article, their main point is that the Bangladesh study found masks to be effective.
Except that study is junk and all the reported effects were found to be a result of researcher bias [1].
Vox also misrepresents the Danish study, which is probably the best study to date we have on masking effectiveness.
> Progress isn't a straight line.
Yes, but truth is most likely to be found in whatever facts are orthogonal to vox' narrative.
For example i might "explain" to you that clean code is about writing the least amount of code possible and you might start code golfing your production systems.
If you want to know what the paper says, read the paper. Journalists are not scientists, most of them do not have the necessary knowledge to understand academic papers, nor do they have an incentive for doing it well. They do have an incentive for generating clicks though, generally by twisting the truth to make things sound more interesting or provoking than they are.
Exactly. My guess is the person making statements about the study that aren't supported by it to begin with also just read some bad explanations of it. This is not worse. Criticising widespread faulty "interpretations" and pointing out it doesn't say what all articles say seems way more likely to get someone to look at the source (especially when engaged in online arguments) instead of using articles to back their claims up.
> that was a misinterpretation of the review in the Covid-skeptic sphere.
No, it wasn't. You should read the paper itself, instead of relying on (sadly) biased editorials about the paper. It literally says what the OP wrote:
> Wearing masks in the community probably makes little or no difference to the outcome of influenza‐like illness (ILI)/COVID‐19 like illness compared to not wearing masks (risk ratio (RR) 0.95, 95% confidence interval (CI) 0.84 to 1.09; 9 trials, 276,917 participants; moderate‐certainty evidence. Wearing masks in the community probably makes little or no difference to the outcome of laboratory‐confirmed influenza/SARS‐CoV‐2 compared to not wearing masks (RR 1.01, 95% CI 0.72 to 1.42; 6 trials, 13,919 participants; moderate‐certainty evidence)
The only place they found any plausible signal was comparing N95 respirators against surgical masks, but the evidence was extremely weak:
> We pooled trials comparing N95/P2 respirators with medical/surgical masks (four in healthcare settings and one in a household setting). We are very uncertain on the effects of N95/P2 respirators compared with medical/surgical masks on the outcome of clinical respiratory illness (RR 0.70, 95% CI 0.45 to 1.10; 3 trials, 7779 participants; very low‐certainty evidence). N95/P2 respirators compared with medical/surgical masks may be effective for ILI (RR 0.82, 95% CI 0.66 to 1.03; 5 trials, 8407 participants; low‐certainty evidence). Evidence is limited by imprecision and heterogeneity for these subjective outcomes. The use of a N95/P2 respirators compared to medical/surgical masks probably makes little or no difference for the objective and more precise outcome of laboratory‐confirmed influenza infection (RR 1.10, 95% CI 0.90 to 1.34; 5 trials, 8407 participants; moderate‐certainty evidence).
The editorial you cited was a low point in the history of Cochrane, where they gave in to public outrage and attempted to cast doubt on their own data.
From the editorial: "It would be accurate to say that the review examined whether interventions to promote mask wearing help to slow the spread of respiratory viruses, and that the results were inconclusive. Given the limitations in the primary evidence, the review is not able to address the question of whether mask-wearing itself reduces people's risk of contracting or spreading respiratory viruses."
Whether you think the editorial was them caving or not, they also issued it under their own name with the same weight as their other reviews, so they must have thought enough of it to do so.
Given that there's ample laboratory evidence of the filtering capacity of a good N95 or even a KN95 mask, and having worked with an N95 respirator in tuberculosis control settings for 17 years and never converted my TB test, I think I'll stick with the mask in future and I have no hesitation recommending winter masking to others who believe they are at risk of complications.
I've liked not being sick for the last three years.
Seriously, what's the difference between what OP wrote:
" [Cochrane] found there was no reliable evidence that masks worked against COVID/"
And the editorial: "the review examined whether interventions to promote mask wearing help to slow the spread of respiratory viruses, and that the results were inconclusive"
How is "inconclusive" functionally different from "there was no reliable evidence?" Seriously, how do you justify this pedantry while ignoring and obfuscating the truth?
People do much evil by focusing on the wrong facts, the wrong stories, and the wrong lessons learned, while ignoring the right ones. That you are willing to focus on apparently frivolous pedantry while ignoring the fact that so many were forced to use masks without any high-quality scientific evidence that they actually did anything, including children, and all the lessons that should derive from this, is in my opinion, very representative of this type of evil.
It's not "inconclusive" and "there was no reliable evidence" that are different, it's the promoting part that makes them completely different.
"We found no reliable evidence that abstinence prevents teen pregnancy"
"We examined whether promoting abstinence prevents teen pregnancy and the results were inconclusive"
The first is obviously wrong, and if the the second is true it would mean the government should look for other ways to prevent teen pregnancy, but it wouldn't mean that practicing abstinence as an individual doesn't work to prevent pregnancy.
> > "We found no reliable evidence that abstinence prevents teen pregnancy"
> > "We examined whether promoting abstinence prevents teen pregnancy and the results were inconclusive"
> The first is obviously wrong,
No. They're equivalent. They both mean "we looked, and we didn't find any confirming evidence." You're confusing "we found no reliable evidence of X" with "we found evidence of NOT X", which is different, and essentially never achievable in empirical studies (note: this is not an invitation to get side-tracked in pedantic debates about proving the null; I'm telling you how actual randomized controlled trials work, in real life.)
Proving a negative via statistics is ~impossible, so what you do instead is to look for significant differences in X, attributable solely or partially to the intervention. If you don't find such a difference (as was the case in the mask review), you say "we found no reliable evidence of X".
But when the Cochrane authors wrote "Wearing masks in the community probably makes little or no difference to the outcome of influenza‐like illness", they really did mean exactly what it sounds like -- the effect size in an aggregated pool of randomized controlled trials was statistically indistinguishable from zero. You can debate whether or not they looked for the right thing (X), you can debate whether or not adding another big randomized trial would help find X, and so on. But the plain-text interpretation is correct.
In medicine you cannot distinguish. It is all about the intervention and not about some theoretical best-case scenario.
The intervention is to ask people to wear masks. People comply as they do in real real life and then we measure the results. There was no reliable evidence that this made any noticeable difference.
Now you can change the intervention – instead of asking and mandating masks as we did, we could educate masks wearers more. Unfortunately we have no evidence that it helps.
Perhaps masking could help to an individual wearer? Alas, we didn't collect such evidence either.
Some studies are lab based. In those masks had some effect. But that's not how people use masks in real life, so these results don't mean much.
> But that's not how people use masks in real life, so they don't mean much.
I think saying "Using X is effective, but only if you actually use X" is obvious. The thing people want to know is "do masks stop the virus" which is an entirely different question from "How many people will wear masks", which is a different question from "What is the effectiveness of interventions to promote mask wearing"
The first question is pointless for someone responsible for public health. People want the answer to it because they don't want to think about all these related issues and have simplistic idea that they can protect themselves. But chances are their compliance is exactly the same as among people in those studies.
Therefore the real question is how effective is the intervention. It will be (or should be) asked by people responsible with public health policies.
P.S. Cochrane group is not for giving scientific answers to individual people. Its main aim is to evaluate the evidence of different treatments and provide guidance to policy makers and healthcare authorities.
If you are responsible for public health and the answer to the first question is "no" then you have no need to ask the other two. Figuring out what we can do to get people to do what works is important too, but it's not the only thing that matters. People can be educated and their habits changed.
We have similar problems getting schizophrenics to take their meds and getting communities with high rates of open defecation to use toilets, but nobody suggests that we give up on antipsychotics or sanitation facilities.
The first answer is too vague to have a meaningful answer in case.
Every other treatment in medicine including schizophrenia is tested how it works in practice. It is incurable disease and the treatments have many side-effects. Thus the question becomes not “does this medicine cure schizophrenia” but “does this treatment works better than placebo or another treatment?”. When studies are completed, we gather evidence by monitoring real life experience with this treatment.
> Every other treatment in medicine including schizophrenia is tested how it works in practice.
Medicine is tested according to how it works when people actually take it. People participating in research studies who fail to take their medications (or their placebo for that matter) are kicked from the program and their data is typically discarded entirely.
In fact, often clinical trials are statistically analysed by intention-to-treat, including all people who have been randomised even if they later don't receive the treatment.
Per-protocol-analysis (including only people who follow the study protocol) can also be used but it is more prone to bias.
Besides, with masks it is not simply wearing or not wearing a mask. Even a very diligent mask wearers may wear it in a way that makes it less effective without being aware of that.
In short, when the doctor prescribes a medicine it is important to understand the factors why the patient may not take the medicine as prescribed. If the real life situation is that most people take medicine in a way that makes it ineffective and so much that the clinical trial cannot find significant effect, then he shouldn't prescribe it. It is just a waste of resources and giving people false hopes.
> In fact, often clinical trials are statistically analysed by intention-to-treat
Fair! That said, intention-to-treat is more likely to greatly underestimate the efficacy of a treatment when non-adherence is expected to be high/isn't being monitored.
> In short, when the doctor prescribes a medicine it is important to understand the factors why the patient may not take the medicine as prescribed.
I agree, but the solution is to help the patient overcome those barriers not to throw out the medication. It's to give people the information and tools they need to follow the treatment. People who wear masks could be trained on how to properly fit them, take them on/off, store them, replace them, etc. The real life situation around masking included basically none of that. "Wear a mask" was basically all people were told.
It doesn't make sense to fault/dismiss masking if a large part of the population isn't wearing them because they were tricked into believing that masks don't work or that masks will actually make them sick, and another large part of the population wears them, but wasn't shown how to do it correctly.
It's important to be aware that those things are going on within the population, but the next step from there is still "educate the public" and not "abandon all efforts at masking" - at least not until a more accessible alternative which is also as effective as masking becomes available
The population was told that masks certainly work, in certain areas mandates made sure that compliance is very high >95%.
If we still could not find reliable evidence that masks are effective, then the policy makers should be told that.
There is very little you can do to improve mask wearing technique. We certainly explained these things to doctors, it made no difference in results. If you want to make more controlled studies, you can do that. Don't hold your breath however.
No, we should not continue requiring wearing masks because you are only doing that out of hope. That's not how we do things in medicine. It would be unethical. There are many medicines that show effectiveness in the lab but fail in clinical trials. We don't demand for those medicines to be used until we find more effective alternative. Many unknown factors could cause ineffectiveness in clinical trials, we don't need to understand all of them, just the fact that the drug failed to demonstrate effectiveness and safety in real life settings.
>> a more accessible alternative which is also as effective as masking becomes available
The point is masking was not effective. It has not shown effectiveness anywhere in the real world.
There is very much plenty of fairly reliable evidence that masks work. And the better the compliance the better they work. In nurse studies you get much better results than in population studies, for instance. Now that I'm looking I'm hard pressed to find any studies that go against this conclusion.
The latter sounds like advertising and education about masks rather than wearing the masks themselves. ie telling people to wear masks made no difference in spread probably because people’s minds were already made up about masking.
I din’t see it making any conclusion about masking itself
I'm completely sure masks hurt my pocketbook and my ability to keep my car tidy, and that forcing people to mask has additional costs. There are cost/benefit questions that aren't as trivial as you imply, and they should be made based on reliable data.
Your existence is valuable to me for, if nothing else, potentially interesting Hacker News discussion. I wish your comment was a better example of that. I am engaging in the hopes you can do better.
Throughout the pandemic, my family and I have been careful about being around others. We have masked with N95 or KN95 masks when we couldn't stay home. I cut my beard for a better fit. We got vaccinated as we were able, and enrolled one of our children in a vaccine trial (he wound up in the control group). We have tested pretty often, and as best we know none of us has contracted the virus. I'm meeting my responsibilities to society around not spreading coronaviruses.
I'm also trying to meet my responsibilities to society around engaging with issues honestly. It's disheartening to see people who probably share my position on an issue behaving poorly.
Pretending that downsides don't exist is more likely to lose you the argument than win it. Spouting abuse and insults means people stop listening. Please do better.
There are studies that show cheaply made surgical masks shed microplatic fibers which end up in the lungs of those who wear them over prolonged periods.
Absolutely false. There are lots of negatives to mask wearing, starting with inducing developmental problems in children and continuing on with massive increases in long lasting trash and then into more speculative issues with breathing. It's not a harmless activity.
> they also issued it under their own name with the same weight as their other reviews, so they must have thought enough of it to do so.
Data is data. Editorials are editorials. The fact that they're published on the same website doesn't change the data. If the Higgs boson was published in the same issue of Physics Letters B as another letter that claimed uncertainty of the result, would you treat them with equal weight?
> and having worked with an N95 respirator in tuberculosis control settings for 17 years and never converted my TB test
I mean...that's fine? Nobody is telling you what to believe or do. Most of what we do comes without evidence. But let's be slightly rigorous thinkers for a moment: there's a fairly obvious difference between a fit-tested n95 mask in a laboratory setting, where there are lots of other interventions happening at the same time (negative pressure labs, hoods, etc.), and putting on a loose surgical mask on a bus. We should be able to talk about that rationally, and not resort to superstition.
> I've liked not being sick for the last three years.
I haven't worn masks and I haven't gotten sick either. Other than Covid -- which I got when we were all wearing masks.
That's drawing an unnecessarily sharp description. To a first approximation all Cochrane pieces are editorials. They're interpreting what's actually out there.
> But let's be slightly rigorous thinkers for a moment: there's a fairly obvious difference between a fit-tested n95 mask in a laboratory setting, where there are lots of other interventions happening at the same time (negative pressure labs, hoods, etc.), and putting on a loose surgical mask on a bus. We should be able to talk about that rationally, and not resort to superstition.
No one's resorting to superstition. You're the one saying there's no value in an intervention that has empiric laboratory evidence to support it. The argument here is what matters at the population level. If the problem is performance, then we train people to select and use masks better, not simply say that there's no point to it at all.
The review found very few studies into the effectiveness of N95/respirators against ILIs, and from those studies they concluded "wearing N95/P2 respirators probably makes little to no difference".
Bear in mind a possible source of confusion here: TB bacterium are ~3 microns in size, but viruses are about 0.2 microns. The Cochrane review I mentioned is only about respiratory viruses. So it's possible that they may work against TB but not against flu or COVID.
I'm pretty aware of how large a TB bacillus is, thanks.
The NIOSH definition for an N95 is a device able to filter at least 95% of airborne particles that have a mass median aerodynamic diameter of 0.3 micrometers. While SARS-CoV-2 is around 0.1 microns in size, naked COVID-19 viruses in air are rare as they would be torn up nearly immediately, so they are almost always within aerosols. Typical respiratory aerosol range is around half a micron or so [0], and as the aerosol particle size gets smaller, so necessarily must be the amount of virus that is present.
Is this perfect filtration? No, but no one gets sick from a single virus they inhaled either, even with as communicable as the current Omicron variants are. There's a minimum infective dose and they help keep exposure under it.
The size of the single virus is a false metric here. There is a wide range of respiratory droplets containing virions. Those droplets can range from visible (way bigger than a mycobacterium ) to only large enough to hold one virion. The size distribution of those particles is the metric.
Yup, we lead lives where it's simply not that big an issue to protect ourselves. While I think my chance of dying from getting it would be very low the issue of long term damage is another matter--it certainly looks to me like it damages everybody, just not always to the point they notice. The damage is probably cumulative.
It also literally says "The high risk of bias in the trials, variation in outcome measurement, and relatively low adherence with the interventions during the studies hampers drawing firm conclusions."
It does, and that's true, but that doesn't contradict what OP wrote.
They found only mid-to-low quality evidence supporting the use of masks to prevent ILI. That evidence, for everything but the question of "n95 vs. other", showed an effect size statistically indistinguishable from zero.
You're essentially saying that the error bars on that effect size are big. They are. But they're still centered on zero.
The evidence they had was of such low quality that no solid conclusions could be made from it. What they found in the research may not reflect reality. They are explicit about this and stress the need for better research.
> "There is uncertainty about the effects of face masks. The low to moderate certainty of evidence means our confidence in the effect estimate is limited, and that the true effect may be different from the observed estimate of the effect...There is a need for large, well‐designed RCTs addressing the effectiveness of many of these interventions in multiple settings and populations, as well as the impact of adherence on effectiveness, especially in those most at risk of ARIs."
They admit that they were unclear about it and later were even more explicit.
"Given the limitations in the primary evidence, the review is not able to address the question of whether mask-wearing itself reduces people's risk of contracting or spreading respiratory viruses."
The review is not able to "address the question" let alone conclude anything about the impact of mask wearing. The review is inconclusive.
> The evidence they had was of such low quality that no solid conclusions could be made from it.
OK, fine. If I grant this (I don't, but let's run with it...), it means the following is also true: we mandated something based on such low-quality evidence that no solid conclusions could be made from it.
The basic problem is whether the data says masks don't work, or says that people aren't consistent enough in wearing masks.
I've seen it directly--one woman putting on a mask when I approached. The thing is she had been hiking near the back of the pack in a group that got together for the hike. She was at a far higher risk from being downwind of her group (this was not a family bubble) than of me being off to the side.
I can basically guarantee nobody there was experiencing any appreciable symptoms (10,000' up, miles from the cars--not something you're doing with any sort of respiratory infection) but most Covid spread is presymptomatic.
A solo hiker masking when someone approaches makes sense (and is what I did pre-vaccine), but not masking with your group but masking for a stranger? That's merely an illusion of safety and why masks "don't work".
There's also the problem that the Cochrane data included mostly studies of things other than Covid--when you go over their own data only looking at Covid you do see some benefit. Note, also, the pooling of masks and respirators--we already know masks do little against the Omicron variants. Respirators or don't bother.
Cochrane messed up badly in this case by looking at the wrong thing. I'm reminded of the BMJ study showing zero safety benefit from parachutes when jumping from an airplane.
I would like to point out what “makes sense” to people rarely reflects the underlying fluid dynamics at the relevant scales. Couple that with a poor understanding of just how many particles one infected person emits and it’s clear masks as worn are very ineffective for the vast majority of people.
Yes, the masks didn't work. Now everybody should know it.
First of all, they were using paper mouth shields or adidas branded useless cloths, not masks.
But even the dumb fcks using real n95 mask, i see people everywhere touching the mask from outside (where the viruses should be stopped if the mask works) and then touching everything else. And when coughing opening the mask and coughing inside the palm...
First, some laws probably don't have any positive impact.
Second, there's a difference between accurately summarizing trial results and extrapolating that to the impact of a new law. If there was a death penalty for not wearing masks, perhaps compliance would be better than in the trials and an effect would be shown. This doesn't mean that the trial analysis is wrong, you just can't draw a conclusion about the law from the trial data.
Third, laws have multiple purposes including Justice and Punishment. Some murderers might have zero chance of re-offending but we still want to punish them as a matter of Justice, not because it makes Society safer.
I generally think laws should be codifications of societal norms. Which also implies that as societal norms change so should laws.
So even things such as murder which people do and we don't want should be codified as illegal. But even if nobody committed murder anymore it should still be illegal as its against societal norms.
Point was that some questions are not about law. Im sure you are not going on red just because you could get fined for it.
But also, you'll be fine to go on red in the middle of night, where noone is around. Personal responsibility is unfortunately removed by centralised regulations and laws. Thats the problem.
In reality things are complicated and one principle[1] cannot account for all situations, so we deal with things differently as they come up. Murder happens to be an extremely old problem; public health is also, but knowledge of microbes and infection mechanisms are brand new, so we are still figuring it out.
[1] "personal responsibility" makes little sense as a moral catch-all when ignoring your own health can cause other people problems
- ...ignoring your own health can cause other people problems"
How much of Netflix cause health problem for you? What about food? How much of junk food are you eating? Or maybe we should ban tabaco and alcohol. Or profesional sport. All of that is potential threat for your health. Being overprotective could damage your mental health... etc.
Your comparisons between issues in modern cultural and political concepts are fascinating. Please tell me how overeating, obesity, personal addictions, competitive athletics and media consumption are related to microbial infection vectors so that I can write a definitive thesis about externalities and how they don't exist because I want to live in a libertarian fantasy world.
I will make a comparison, tell me how you find it: "Dumping waste into the river shouldn't be illegal because you dump your waste in your toilet every day and it goes to the river."
Does it sound reasonable? Sure. But I haven't defined scope and I haven't defined effect and I haven't even defined what 'waste' is; I just made up two scenarios which could have similar outcomes and then called it day -- but if you actually care about what happens to the world you need to get a little more detailed than writing silly grandiose political statements and making non-sensical metaphors comparing indicting someone for murder with crafting public health policy.
I'm mass transit where masks will required, most of the people wore the masks covering their chin. I believe the research is accurate and that saying that masking up more people doesn't work because they won't wear them properly
Also, while this study was inconclusive as to whether masks help prevent covid, it doesn't mean that all studies are inconclusive.
For example, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8768005/ is "a detailed performance evaluation of the mask is studied from an engineering point of view," which aim to look at how the physics of N95 masks hold up against covid. What the physics shows is that N95 filtration helps block covid particles.
Mouthwash is typically alcohol based. Alcohol is a pretty good disinfectant in general.
But it's efficacy will really only be decent while it's in your mouth. Once it gets diluted past a certain point, it's not going to be doing anything. You'd probably have similar results with vodka.
I know doctors who recommended drinking whiskey early in the pandemic for that reason (and also the usual reasons people dealing with trauma reach for whiskey)
"Many commentators have claimed that a recently-updated Cochrane Review shows that 'masks don't work', which is an inaccurate and misleading interpretation."
Next sentence: "It would be accurate to say that the review examined whether interventions to promote mask wearing help to slow the spread of respiratory viruses, and that the results were inconclusive."
... which is what I just said: some people got mad at them because their review found no reliable evidence that masking worked (or rather, that mask mandates worked, but these are virtually the same thing).
The null hypothesis for any medical intervention is that it has no effect. You start from that and then try to prove your hypothesis that it does have an effect, which is what medical studies are for. If you can't prove something works then we fall back to the null and assume it doesn't. So that isn't a misleading or inaccurate interpretation of the results, though it would certainly have been politically convenient for the Cochrane organization if their reviewers could have supported the claims of public health authorities.
That sentence doesn't say what you think it says. It says "interventions to promote mask wearing". That's not mask wearing, it's telling people to wear masks. It is both true that wearing masks helps and that it's hard to tell if promoting mask-wearing changed enough behavior to matter. Mostly, those interventions do nothing.
That's an ambiguous sentence. The main results of the study conclude:
Wearing masks in the community probably makes little or no difference to the outcome of influenza‐like illness (ILI)/COVID‐19 like illness compared to not wearing masks (risk ratio (RR) 0.95, 95% confidence interval (CI) 0.84 to 1.09; 9 trials, 276,917 participants; moderate‐certainty evidence.
The original Plain Language Summary for this review stated that 'We are uncertain whether wearing masks or N95/P2 respirators helps to slow the spread of respiratory viruses based on the studies we assessed.' This wording was open to misinterpretation, for which we apologize.
> no reliable evidence that masking worked(or rather, that mask mandates worked, but these are virtually the same thing).
No it's not the same thing, and that's the key point.
If you tell people that masking doesn't work (which is false) then of course mask mandates won't work because adherence will be low. A self-fulfilling prophecy really.
Compliance for COVID mask mandates was measured and found to be extremely high, especially at the start (>95%). These mandates were enforced by harsh penalties so high compliance levels is no surprise. Thus you can't argue mask mandates didn't work because of low compliance.
Also health authorities told people masks were highly effective. That's what justified the mandates. So you can't argue mask mandates didn't work because people were told it wouldn't work.
Therefore there's no self fulfilling prophecy here. It didn't even matter what individuals thought anyway, we all had to wear masks.
Although Cochrane much prefers to use RCTs, people have run regressions over the data and there was no link between levels of mask wearing and infection rates. It sucks but it appears that masks just can't stop aerosolized virus, which spreads like a gas. They aren't designed to do that so it's no knock against the manufacturers, who in some cases explicitly warned people that their products would be useless for that purpose (https://pbs.twimg.com/media/EfNmzptXkAEg9Od?format=jpg&name=...).
> ... which is what I just said: some people got mad at them because their review found no reliable evidence that masking worked (or rather, that mask mandates worked, but these are virtually the same thing).
This is not virtually the same thing. Comparing those two is wildly disingenuous and you know it.
The success of the Nutrition Facts labeling does not get enough publicity.
Rather than outlawing certain ingredients, or creating some kind of health score which a product must be above, Nutrition Facts is a way for suppliers to attest to information about a product in way that is legally binding. If it isn't accurate the penalties are steep.
Consumers then have the information they need to vote with their wallets. Markets cannot function properly without symmetric information, and Nutrition Facts essentially creates a functioning market where one did not exist previously.
Any effort to regulate what's in food would probably be better spent expanding what must be in the Nutrition Facts label. I guess it's nice that we are finally getting around to banning artificial trans fats, but anyone who can read has been able to keep those out of their diet for years. The same can be said about the next bad ingredient, and the one after that.
Interestingly, there is some gaming of the main number everyone looks at on the nutrition facts chart: calories per serving.
All snacks aim to fall at or below a certain number the FDA (or some other agency) put out as being considered a snack. Planet Money did an episode on different M&M varieties having different total weights to account for their different calorie counts. So you get fewer by weight peanut butter M&Ms because they’re more calorie rich
This doesn't sound bad to me? At large, people eat a package of whatever snack they choose to buy. Also at large, people assume different packages of snacks should be roughly comparable for important metrics. And calorie count is probably up there for important metrics.
I'll add, a family member has celiac which makes it so they can't eat gluten. Becoming certified "Gluten Free" requires a certification process that can be expensive and difficult. However many companies have realized they can label their product "Gluten Friendly" and get around the requirements. It is annoying.
Basically, there were stricter measures put in place called HARPC. From my linked article:
> The new directives mandate that the “Top 8 allergens” identified by FALCPA (peanuts, tree nuts, milk, eggs, wheat, soy, fish, and crustacean shellfish) must either be ingredients of the product and identified as such, or the manufacturer must take extra care (and cost) to ensure that there is no cross-contact with them. There is no middle ground or “out” for the manufacturer, which is why we believe “May contain” type label advisories are heading for extinction. And that poses a problem, at least in the short term.
> companies, when faced with the added burden of instituting and documenting cross-contact prevention measures as dictated by HARPC, may instead choose to add trace amounts of the allergen to the product, as doing so makes the allergen an ingredient of the product and obviates the need for preventative cross-contact measures for that allergen.
> it means that manufacturers will either take stricter measures to prevent cross-contact or add a trace amount of the allergen and list it in the ingredient list, thus eliminating the ambiguity that currently plagues us all.
> A compromise that might have avoided the unintended consequences of companies like Kellogg’s adding traces of allergens to their products is to have offered them a third option: A mandatory “May contain” label advisory for any product made on shared equipment or in shared facilities that did not meet the FSMA threshold for cross-contact prevention. Such label advisories are voluntary today, rendering them ambiguous at best, but a definitively worded and located advisory statement included on all such products would have provided a way for manufacturers to meet the requirements of HARPC without resorting to the addition of allergens.
So I'd be surprised if it was happening on an ongoing basis, but I can definitely see why people would be irked.
The penalty for having an allergen present is steep, and the process of certifying that yes, you are in fact allergen free, is expensive and difficult, while the cost of adding an allergen into your process, for pretty much any foodstuff, is cheap, and the cost of slapping a "may contain peanuts" label on is cheaper.
I believe the original comment was complaining about the perverse incentive there.
I don't have any food allergies, either, so I admittedly don't pay much attention to allergens (or even listed ingredients most of the time). I'm genuinely curious, too.
No, we are complaining about the way the government handles allergen labeling.
It used to be that companies could slap a "may contain [allergen]" label on things that didn't contain it but were produced in a factory where cross contamination was a possibility. Such labels are *widespread*.
I don't understand the government's incentive in trying to stop this--the actual result was when cross contamination was a possibility the companies reacted by deliberately adding the offending material.
The problem is that it's being looked at in a binary sense. Either it contains the offending material and poses a danger to those affected, or it doesn't and is safe. However, in the real world there's a third population--those who are sensitive to the offending ingredient but not dangerously so. Possibility of cross contamination? That is not going to be a deterrent to me as the worst case outcome is merely unpleasant. Does contain? I'm going to treat it with great skepticism.
> Ultimately, a lingering question is — as with paper mills — why so many suspect RCTs are being produced in the first place. Mol, from his experiences investigating the Egyptian studies, blames lack of oversight and superficial assessments that promote academics on the basis of their number of publications, as well as the lack of stringent checks from institutions and journals on bad practices.
A substantial part of what's happening here is that first-world countries with generally good cultures of research integrity are basing medical policy on studies done in countries where the system encourages researchers to cheat. British and US authorities can't put Egyptian or Chinese researchers in prison, can they?
This is the correct answer. Typically if something is done at an ivy league, other US and UK universities follow. Perhaps the other countries would follow as well shortly after that, or there would just be a divide between 'real' research and 'not', similar to when many of my friends stop reading after the word 'Hindawi'.
It wouldn't be limited to Florida. Elsewhere, having any reasonable questions about the severity regarding the religion of Climate Change would get one jailed for blasphemy.
Imagine same approach in politics. If some politics put lies in their speeches to manipulate with people, we should call them liers and put them in prison. But somehow it doesn't happen. Looks like society prefer conformity over responsibility.
As reasonable as that would be. I feel like it would just turn into people getting their degrees revoke claiming 'cancel culture' and becoming gurus with many mindless followers pushing loose weight quick schemes kind of thing. Because today if you face consequences, its no longer your fault, its everyone else trying to cancel you from doing the bad thing you are doing.
Why would the statisticians be anonymous? I'm aware of at least a couple cases in which an independent set of statisticians were provided the data from a clinical trial specifically for a re-analysis. In one case, they showed some pretty concerning inconsistencies and the other confirmed no effect on the primary analysis, but suggested some sub-populations that might have shown an effect if a future study was properly powered. That follow-up clinical trial was just published showing pretty remarkable effect in the sub-population. I don't think there's reason to believe either independent analysis was anything other than independent.
There is already a mechanism for companies to submit questions to the FDA prior to clinical trial initiation. I'm not in these conversations, but I know the type of questions can be things like: would you accept this endpoint as a proxy for this indication, would you be satisfied with the effect size we expect, and are there other safety concerns you would expect us to evaluate other than those in our current plan. I assume EMA and other regulatory bodies have a similar process, but I'm not positive.
Disclaimer: I work in pharma, but pre-clinically. I am not involved in these clinical or regulatory issues.
Yes, I have said this as "The UX of [medical] study papers is terrible". Some people do not agree, they think that it should not be made easier to understand, that non-experts cannot really understand medical studies, so they should not be more approachable. I think that's dead wrong.
Im struggling to see the difference between this and the current FDA process, and think it is 90% the same.
Drugs have "prescribing information", referred to in industry as "labeling", which follows a consistent format containing safety, trial results, side effects, and mechanism of action.[1] I recommend people read them for drugs they take.
This should be an async non blocking evaluation. The statisticians who do it should be anonymous by default. There should be an appeals process for a scientist to explain why an unconventional new method is actually robust.
Third party analysis is the main difference here. In the current state, firms run the analysis for FDA review using standard practices, and must explain and get approval for any unconventional methods
>There should not be a single number published by this process, but rather a list of stats that speak to the overall quality of the trial on many dimensions (power, sources of bias, etc).
Labeling contains many relevant numbers. Trial sizes, how many per arm, what was measured, and and final results. Maybe there could be some squishy qualitative summary, but that seems more risky. I would rather know that 1 out of 20 patients died than it got a "2" on the safety scale.
>Only information that would not be the same on 99% of trials should be written on this label (no sec style everything is a risk word vomit disclosures).
Labeling contains drug specific information.
There should not be a pre-emptive application for a label - it can only be gotten after paper submission to reduce gaming.
Drug labeling requires pre-application and and a standard 12 month review period by the FDA prior approval
There should be an independent advisory org that scientists can literally call to ask for advice on structuring the trials. These calls must not be disclosed. Much like farmers can call the government to ask for help on xyz crop problem.
The FDA provides advice on structuring trials and acceptable design, size, power, endpoints. Firms do this by scheduling calls with FDA staticians and experts. [2]
And these labels should never be used as the primary source of punishment. Any and all sanctions/penalties/dismissals must go through a new review process done by a different group.
Maybe there is a difference here. I'm not sure what you mean by punishment? In the current system, The FDA can use the label as "punishment". The FDA may require addition "black box warnings" for drugs that are found to have serious side effects (e.g. high chance of death). They can also pull the label entirely, meaning the drug can not be sold.
Any scientist who gets a label in a particular year should be given a vote to review the review agency on several dimensions. These aggregate reviews should be published broadly but not trigger any automatic consequences.
This is basically how it works for medical device labeling in the EU. There are several "notified bodies" [3] which are private agencies to review the safety and efficacy. The firm then takes their mark of approval to the government agency.
They say "medicine". I would say science in general, perhaps we could generalise even further to "any human activity is full of unethical people trying to exploit it". But 25%!? That suggest there is a big problem with how we "do science". Unfortunately I have no solution to the problem. Publishing everything (including raw data) for every research would probably help somewhat, but only teams repeating experiments/trials would ensure it.
I wonder if we suddenly took 10% of all money spent on science (let's say in medicine) and instead of novel research we used it to redo randomly chosen previous research. Would we loose or gain in terms of new cures? And if we gained, what if we spent 15%, or 25%? That's a great idea for a scientific study to find a point of diminishing returns on "research verification".
Would someone please write a research grant request for this?
Of course there are big problems with how we do science. Much of it is garbage. Imagine most software was written by junior engineers, without any code review or input from seniors. That is today's science.
Most scientific legwork is done by absolute beginners, i.e., graduate students. They often lack a support structure to focus on what they have learned so far. Most of the world is not Oxbridge, MIT, Stanford.
Where are the beginners' supervisors, you might ask? Chasing the latest trend to secure funding. Pondering how their line of research can be formulated as buzzword-du-jour markov chain. Ass kissing the dean to get department funding.
Having worked in research for 15 years, I am certain about two things: (1) The scientific method yields better results than doing things freehand. (2) Randomly axing 50% of academia would improve the situation.
That 10% investment would have a huge payoff too because it would shut down avenues that were only opened due to p-hacking long before they'd had a chance to seek further investment.
And as a bonus, read books by controversial figures who talk about these challenges through their own published studies and decided it isn't worth fighting.
What would be cool is to see "trust indicators" as part of a study's metadata as it is reviewed through time and continually shared by others. Could be a "study health score" or a checklist showing the study isn't biased heavily by sponsors, methods, or misleading language.
Especially given that medicine is becoming more personal with the advancements of AI and accessibility of tests, someone should be able to understand the health of a study at-a-glance reviewed by peer reviews rather than disclosed only by the authors.
A primary trust indicator of any research result is the reputation of the journal publishing the paper. Better journals demand more, not just in terms of the impact/insight of the results, but also the rigor of the experimental methodology used and how well the data was curated and the confounding variables identified and isolated.
And of course, the criteria for observational-based research differ a lot from mechanism-based, especially since the former can't control for nearly as many variables. The same goes for simulations or interpretive modeling, where experimentation is minimal.
Research studies differ quite a lot in how precisely the mechanism of action is identified, isolated, tested, and results interpreted. IMHO, blackening the trustworthiness of all of science is unhelpful, especially when some models and methods are surely more trustworthy and replicable than others. Better to identify and catalog specific sources of error (or imprecision) in order to remedy them than to just toss the baby.
If you read the article, the headline is wildly editorialized.
Whatever. Par for the course in 2023, right?
But this is Nature, a paragon of scientific literature, fueling the distrust of medicine.
There are great reasons to be skeptical of all trials and strengthen peer review and transparency, but this kind of headline is editorial malpractice, in my opinion.
No, not really. The headline says "plagued with", the subhead says "in some fields, at least one-quarter of clinical trials might be problematic or even entirely made up", and the article substantiates that. One quarter of all trials is more than adequate to plague the whole enterprise, and there isn't a problem with this headline.
That word has gotten a bad wrap by people on social media using it without substantiating it, or hiding behind an excuse of "Google it"; meaning 'I know but you don't so go educate yourself because I'm subtextually declaring your opinion invalid by way of your ignorance'.
We are seemingly in a fifth generation war amongst ourselves for the prizes of attention and public acceptance. Or more succinctly, "being right on the internet".
Agreed that the headline makes it sound like a strong majority, when the article isn't nearly as strong on that.
Still, "Carlisle rejected every zombie trial, but by now, almost three years later, most have been published in other journals — sometimes with different data to those submitted with the manuscript he had seen. He is writing to journal editors to alert them, but expects that little will be done." is concerning. I'm almost afraid to know what the list of rejected papers covers.
I mean, fair that "plagued by" is a very vague term that has no quantifiable meaning. But appeals to emotion to cast doubt on all studies is frustrating. "Healthy skepticism in the face of bad studies" would be a great headline and is accurate. But too much of the skepticism we are exposed to on a regular basis is not healthy.
If 1/4 of all studies have fundamental data issues then that means in some corners we are making medical decisions based on bunk, that absolutely is a plague. Why moderate the language to extenuate?
Because if 100% of that 1/4 of studies is all in, say, homeopathy, that gives a very different action plan than if it is a random sampling of all studies.
Still bad, mind you, but unfocused skepticism is its own plague that will cause more trouble.
Exactly. Its important to see this stratified across subfields. If the vast majority are in homeopathy then maybe who cares? If a substantial portion are in, oh lets say Alzheimers treatment then maybe that's more of a problem?
Basically this. So, fair that I shouldn't just be calling for moderating the language. I'm more wanting specific language with a distaste for unfocused skepticism. I say this as a skeptic. :D
You can be the sole person in the world with the Bubonic Plague and you'll still be plagued by it. Whether or not untrustworthful clinical trials "cause distress to" [1] Medicine I don't think is debatable; medicine should be based on treatments that have reproducible effects or else people don't get better.
At an individual level, absolutely. In group dynamics, though, you avoid things that have the plague. Same as you avoid things that have rabies.
Which is part of my point. If you say that "medicine is plagued" than a natural response is "avoid medicine." But, that is clearly a nonsensical outcome, all told.
We should be distrustful of medicine, of all science in general. Ignoring flawed methodologies or inconclusive results just means it's no longer science, it's ideology. I'm a physicist though so perhaps I could be overly jaded about science and peer review compared to most scientists
You simply can't apply the rules of publication in physics to medical biology research. Even highly quantitative biology is noticeably different in terms of standards of proof and quality of models.
I haven't seen any real serious replication problems in physics that didn't get cleared up, or anything else that would make me doubt the results.
I would generalize the statement: assuming a reductive order of sciences (medicine->biology->chemistry->physics), if one cannot trust a layer, it seems even more likely that layers above it should be trusted even less.
It has been since the introduction of peer review that science has taken a nose-dive. Peer-review is just such a terrible idea. All it does is bring politics and back-stabbing into science.
That’s being generous as only 1/10 can actually be fully reproduced from my understanding. It’s so bad that if you want to create a product based on research your first need to reproduce the result to make sure it’s not bs.
He claims to have worked with Tapper for "three weeks on a documentary" (they worked for 2 days on a 2-minute spot) that was "killed by corporate" (it went live one day later than planned).
I'd recommend not using pathological liars' words as evidence for other claims.
There's a difference between a scientist calling bs on some scientific practices than a grifting, crazed group of people calling bs on it. Can you cite the exact trials from your source that are potentially problematic so we can discuss the actual legitimacy of the methodology?
it's 2023, we have the means to cheaply record and store audio and video evidence for basically any medical experiment. we can record every patient reaction and opinion without relying on the reasearchers' hearsay. we also have the means to store and distribute all the binary/textual raw data gathered throughout the experiments.
maybe as an intermediate step we could make available all the recordings to the peer reviewers and only offer the raw experimental data bundled in the paper publicly? maybe in the future we can have 1TB studies without breaking a sweat? maybe all the money we give to publishers can be spent on servers to archive all the primary data so at least we aren't simply filling the pockets of MBAs?
> maybe as an intermediate step we could make available all the recordings to the peer reviewers
The issue is clearly not the amount of data available to peer reviewers considering it's already easy to detect major flaws in a quarter of published peer reviewed research. The issue is that peer reviewers do a shoddy job which should surprise no one having ever published peer reviewed research.
And to be fair why should they do better? It's generally unpaid, it's poorly paid when it is paid and it's not particularly well considered.
If we discover that we can´t trust researchers then what else are we left with? Doctor-patient privacy works if the doctor is truthful in their reporting
It's generally permitted to share de-identified patient data. As long as you're not sharing patients names, medical record numbers, birthdays, and a couple of other fields, you should be fine.
Maybe we could do a double blind (including scientists) study where everyone waived their rights & are recorded then in another "typical" conditions study do none of that and compare the two and see which one seems to have the best, most accurate results.
>it's 2023, we have the means to cheaply record and store audio and video evidence for basically any medical experiment. we can record every patient reaction and opinion without relying on the reasearchers' hearsay. we also have the means to store and distribute all the binary/textual raw data gathered throughout the experiments
This is a great approach IMO. Additionally skeptics (for example anti-vaxxers), should be physically present at the trials.
What is preventing someone from having 200 participants, but saying they only had 100 participants, and then only providing evidence for 100 participants?
The researchers will have to have gone through some kind of third party agency to get the partecipants. This agency should be queried to see the number they report
How does this agency determine who can meaningfully participate in the study? Are they going to have the expertise to make that determination for _every_ study that could possibly be conducted?
What is the difference (to a layperson) between cherry-picking participants and rejecting participants because they do not meet your study's criteria?
Who funds this agency?
Do the members of the reviewing agency have their own biases, and might those biases tarnish the reputation of a study that is actually well-conducted? (hint: this already happens in journals)
Pfizer had a trial with 21823 people in the Expirement group and 21823 people in the Placebo group. In the results they excluded data for 1790 from the Expirement group and 1585 Placebo group. However, _crucially_ Pfizer never claimed there were only 100 people in the study after starting with 200; you know pfizer excluded 3375 people because _Pfizer told you_.
Every clinical trial paper I've read has a discussion of inclusion and exclusion criteria. I think for the trial to be registered, it has to include this information.
"unrelated reasons" should not be an acceptable excuse though. either state the reason or it goes into the trash. and if they were private reasons you can still contact them to confirm they left on their own volition and/or they didn't finish the trial without getting into specifics.
you only need one lie to suspect the whole thing
This is a problem everywhere where the raw data isn't released (suitably anonymized).
In cognitive science, psychology, even computer science / ai / ml, business.
And the problem with rejected papers getting in somewhere else while being total garbage is pervasive. I've rejected a lot of papers because they were mathematically or statistically bogus only to see them get published elsewhere where reviewers were not so careful (a few times in Nature and Science).
We need an open science movement where you must release everything with your paper. The full pipeline to reconstruct every single result from the raw data. No hiding data. No hiding fmri scans. No "our code only runs on our machine". Etc.
I'm empathetic to what you're suggesting — I've published on open science and meta-science specifically, and think open data should be the default norm. The problem with clinical research, though, is that it starts running into conflicting considerations about patient and participant privacy. Even when people aren't patients per se, the focus often involves sensitive information.
You can just say "anonymize it" but that turns out to be more difficult than it seems initially, especially with many questions of interest.
Also, there's often too many opportunities to do science that is of real public benefit that comes with privacy expectations attached for all kinds of reasons. Cases where there is legitimate consented access but an expectation of privacy without data sharing.
People have tried to solve this problem in different ways (for example, methods where someone can analyze data without having access to it directly) and maybe those solutions will lead to a good resolution. But they often have problems of their own (overhead costs associated with providing anonymized remote data analysis), and don't solve all problems (guarantees of absolutely restricted access to personal data).
I think value-based care is the only real incentive on this. Otherwise, there is simply no reason for anyone to care enough. Even the insurers, they found a way to make money by making sure their premiums factor in these things. In the expense of the patient. As long as the drug doesn't kill people, who cares if it works if I make money off it as a pharma? Unfortunately, value-based care can only be pushed top-down. Patients are not in a position of power against pharma companies on this matter.
I now know of a few more people who have completely lost both of their hearing after taking just one Wellburtin-class pill.
At the time, no mention was made in the pill’s warning pamphlet.
It is still difficult to secure a class-action suit in America.
Meanwhile, such quality of life would plummet into a silent world, even if one knew American Sign Language fluently beforehand, that tidbit can go against the victim in court.
There unfortunately are too many perverse incentives that encourage fraudulent studies. Everyone should take findings "established in the literature" with a grain of salt. We should also incorporate more intuition and first-principles reasoning, i.e. obesity is a harmful state for humans even if 100 RCTs proved otherwise.
It’s hard enough to run a clinical trial guys. It literally costs $100 million at a minimum, yet the requirements that make it cost this much are not enough. We are basically going to regulate new medicine out of existence.
An interesting point he made was that in the wake of the Thalidomide (https://en.wikipedia.org/wiki/Thalidomide) scandal, the FDA started requiring drugs to be both safe _and effective_
The 'effective' part has proven to be a big source of complexity in the following years, because while it's relatively easy to prove a drug is relatively safe, it's much more difficult and subjective to prove a drug is effective. That closes off a lot of areas of research and development.
The kicker is that Thalidomide was never sold in the USA to begin with.
Anyway, as any reader can imagine, there would be a lot of negative social outcomes to allowing the sale of ineffective drugs. There's a lot of trouble now with medical devices and drugs not being effective, even though we have the rule. I'm not against regulation, I think medical sales are a really complex issue and I don't know how to even judge where the right balance of safety/effectiveness and innovation/freedom could be.
Please read the article before commenting. The problem is not how hard it is to run clinical trial. It's that made up data is an endemic problem. It doesn't matter if clinical trials are hard or easy to organise when up to a quarter don't actually bother and just forge their results.
But they have to make up that data! Because the work they are doing now was based on a other trial where they made up data, so you have to fix this data to match what was expected from the previous studies. Of course we need to protect their right to make up data. /s
You should read the article, they aren't asking for more regulation, just more skepticism out of publishers and that scientists keep their raw data available for inspection by peers.
Yeah who needs truth and accuracy anyway? Think of the small businesses who will never be because they couldn’t get a simple drug on market with minimal testing
“I think journals should assume that all submitted papers are potentially flawed and editors should review individual patient data before publishing randomised controlled trials,” Carlisle wrote in his report.
This should be considered part of every journal's idea of due diligence and this shouldn't be a new idea. Shysters, con artists and snake oil salesmen have been around for a long time. The purpose of a Journal is to publish reliable information and weed out the garbage. How can you do that if you are not looking at the entire picture?
That's literally the point he's making! The terms "flawed" and "zombie" reflect positive identification of dodgy data, so of course when the data isn't available, they are less likely to apply, hence why:
> This finding alarmed him, too: it suggested that, without access to the IPD — which journal editors usually don’t request and reviewers don’t see — even an experienced sleuth cannot spot hidden flaws.
To be fair, science is now a “publish or be damned” business. So many papers are not worthy of publication and, frankly, are of common knowledge anyway.
Of course, this is driven by the money backing the research which, I’m sure, this is why clinical trials in the medical areas can be of a poor quality.
I have multiple sclerosis, and I never know if a new medication came out because it's actually better or simply because they could get a new patent for a slightly modified molecule.
With the risk of starting a flame war, we recently had several well-publicised clinical trials that reported 95% efficacy of some certain modality. Yet, in reality, efficacy as defined in the trial turned out to be closer to 0%.
Instead of investigating what in the design and execution of the trial led to such a discrepancy, the problem was handled by denying there was a problem, changing the goalposts, reporting ad-hoc hypotheses as facts, silencing all critics, and forcing the public to take the modality anyway or lose jobs, school, and freedom of movement.
I can only guess you mean something regarding the pandemic? Do you have links to show things were "closer to 0%?" Sounds more than a touch outlandish. :(
I'm confident we will know even more about things as time goes on. I'm less confident on any nefarious motivations in most of it. Reality is that a lot of people died, and everyone was trying to gain control and an advantage over the situation. Mistakes were certainly made, but I am back to low confidence in thinking that everything was a mistake.
> I'm confident we will know even more about things as time goes on. I'm less confident on any nefarious motivations in most of it. Reality is that a lot of people died, and everyone was trying to gain control and an advantage over the situation.
It's not a matter of being nefarious. They (CDC, FDA, health authorities all over the world) really though it was important, but they've used unacceptable means to enforce their beliefs.
Science dies when PR takes over reality.
If reality disagrees with the trial, you have to debug the rial, and find what in the design or exection went wrong.
> Mistakes were certainly made, but I am back to low confidence in thinking that everything was a mistake.
Silencing critics by health authorities is not a mistake. It is an intentional act to enforce your views.
>Do you have links to show things were "closer to 0%?" Sounds more than a touch outlandish. :(
Take the Pfizer vaccine. The clinical trial's main endpoint was ~95% efficacy in one thing and one thing only, prevention of symptomatic Covid.
Not reduction in mortality, not reduction in serious disease, not infection, and not spread. The only thing the trial tested and reported was prevention of symptomatic Covid. This is also the sole indication in the package insert as approved by the FDA.
In reality, everyone I know got vaccinated and got symptomatic Covd. I mean everyone, no exceptions. The situation is the similar in my entire country and around the world.
Do you know of any studies on the discrepancy? My understanding was that Omicron came out and basically gave the middle finger to everyone's precautions. With what seemed like literally nothing working against it.
The discrepancy is so massive you don't need large studies. You can easily observe yourself.
(a) Make a survey of the people you know and compare their vaccination status to getting symptomatic Covid. Apply a simple statistical test to test whether it is consistent with the trial results.
[Spolier: it is not]
(b) [advanced] what is your best estimate of the vaccine efficacy given your results of the survey in (a).
The measles vaccine has 95% efficacy. You vaccinate and the disease effectively disappears.
> My understanding was that Omicron came out and basically gave the middle finger to everyone's precautions. With what seemed like literally nothing working against it.
Folks don't like comparing to the flu, but in this there are obvious similarities. With obviously similar outcomes on the ability of a vaccine to give sterilizing immunity.
Much to your chagrin, though, I actually can say that among my contacts, getting the vaccine basically led to people not getting symptomatic covid. Folks got what they thought of as a bad cold. Almost flu like, but I know very few, if any, folks that were so bad off that they were symptomatic covid. Most wouldn't have even qualified as having a bad flu. (It is frustrating how many folks underestimate how hard the flu hits.)
Contrast with family members that did not get the vaccine in time, and were hospitalized. It was truly different.
Symptomatic Covid is simply a positive Covid test + any flu-like symptoms. What you're describing is symptomatic Covid. This is what was measured and reported in the trial.
You might say that's not very interesting because it doesn't measure anything of importance. You would be right. That is exactly what critics say before the trials.
The trials were never meant to test whether there would be any mortality benefit, any reduction in serious disease, any reduction in hospitalization, or any effect on infection or transmission.
What they did meausre, turned out to be inconsistent with reality, though.
Symptomatic covid for the first round was far worse than that. Hell, even for later rounds, symptomatic covid was pretty intense. Again, I had family that neglected getting the vaccine and almost died with that decision. We know of many people that neglected the vaccine and did die.
So, if the concern is you are upset a miracle vaccine didn't get developed, you're losing my interest quick. Anyone that got upset that you had a few symptoms is overblowing concerns to a non-useful degree.
> So, if the concern is you are upset a miracle vaccine didn't get developed, you're losing my interest quick.
No, the concern is not that a miracle vaccine didn't get developed. The trial measured and reported whether people who got vaccinated got those "few symptoms" vs people who got the placebo. It claimed 95% efficacy in preventing those "few sysmptoms", but it did not do so in reality.
The concern is that the trial results do not agree with reality. That means that something is wrong in either the design or execution of the trial. It's a bug in the trial, and a bug should be debugged.
But it is easy to see that the "few symptoms" in the trial patients easily proxied to "safer outcomes" in the wild? I seriously cannot underline hard enough that folks that didn't get the vaccine put their lives in extreme risk for basically no reason.
Seriously, the numbers were drastic for vaccinated versus not in hospitalizations alone. To push the narrative that they were wrong to get vaccines out just feels misguided.
If you are pushing that we should continue to get better at trials and reporting? I agree with that. Any harder push there, though, feels nitpicking at best, and I don't see the direction you are hoping to go.
I don't think it's a bug in the trial, but rather evolution at work.
The vaccine worked pretty well against the Wuhan strain, but Covid breeds variants like it was a rabbit. The farther from the strain coded into the vaccine the less effective the vaccine is. It still seems to be pretty good at reducing the severity, though--the unvaccinated are dying at a far higher rate than the vaccinated.
Even in the initial data released by the FDA, Pfizer didn't test all patients for COVID during the trial. In fact, they didn't even test all 'suspected' cases during the trial. In fact, there were more 'suspected but not verified' cases among the test group than the control.
It was junk science from top to bottom, and this assumes any science was conducted at all. According to a whistle blower, the science was fraudulent.
> Folks don't like comparing to the flu, but in this there are obvious similarities. With obviously similar outcomes on the ability of a vaccine to give sterilizing immunity.
And there's quite a controversy whether the flu vaccine is worthwhile becuase of that. The Cochrane systematic reviews are quite scathing.
> Much to your chagrin, though, I actually can say that among my contacts, getting the vaccine basically led to people not getting symptomatic covid. Folks got what they thought of as a bad cold. Almost flu like, but I know very few, if any, folks that were so bad off that they were symptomatic covid.
That's the definition of symptomatic Covid - a positive Covid test + flu-like symptoms (regardless of severity). That is what the trial measured and reported.
(This is in contrast to Asymptomatic Covid which is a positive Covid test but without any symptoms at all)
> Most wouldn't have even qualified as having a bad flu. (It is frustrating how many folks underestimate how hard the flu hits.)
No one I know experience anything close to a bad flu.
> Contrast with family members that did not get the vaccine in time, and were hospitalized. It was truly different.
Around me it was a mild cold to medium flu regardless of vaccination, including people in their 80s and 90s, with all the pre-existing conditions you can imagine. The only exception was a vaccinated friend (late 40s) who got scary chest pains for several days when he contacted Covid. No treatment beyond Paracetamol and Ibuprofen.
> getting the vaccine basically led to people not getting symptomatic covid. Folks got what they thought of as a bad cold.
Am I mistaken in thinking that "bad cold" == symptomatic? Doesn't symptomatic just mean had symptoms? It sounds like you're talking about severe covid.
Not mistaken, but also not useful. In particular, it is hard to tease out folks that did have a common cold from those that had reduced covid. The vast majority of the covid positive folks I knew post vaccine were asymptomatic. Almost apologetic that they tested positive for it, but not at all sick or scared. Even my kids, when they tested positive, were more upset about implications than they were physically ill. (Indeed, for our kids, when they finally tested positive, we didn't see any symptoms from them at all...)
What is reduced covid? The ifr for a 30 something was .06%
before vaccines according to the study below.
If my math is correct, thats one 30-something dying for every 1667 infected before vaccines. I don't have hospitalization data handy, but I think "reduced covid" is just what most people had, vaccinated or not. That's not to discount the ones that did get it bad of course, and my condolences for any losses you suffered.
Of course it can still be true that the deaths happened more often in unvaccinated people (did that continue to be true the whole time?), while your individual risk of death was low (the .06 above in my case, and I had a pretty standard cold both times thankfully).
Just look up the hospitalization and death rates for folks vaccinated and not. It is stark in difference.
I had what was probably covid early on. Was like the time I got pneumonia. Asthma attacks in my youth were comparable, if much shorter lived. Getting a positive test case later was something that gave me a fever for a few hours. Scary, due to circumstances. But I was back up and moving in basically no time.
Correpondence published in NEJM regarding the landmark study that found that the first booster dose reduced Covid-related mortality by 90%. It turns out that it also reduced non-Covid-related mortality by a similar amount. So either the booster short is a magic elixir that reduces all deaths, or the boosted, as a group, were healthier at the outset.
In the response of the original authors they mention that
> However, boosters were generally not administered to hospitalized patients who were at high risk for death from any cause.
I think you can guess how it affected the hospitalization and death rates of the boosted vs. the unboosted. For some reason, this was not mentioned in the original study.
And the main point mentioned by the original authors in their response:
> However, a strong, unexplained association between the use of the booster and lower mortality not related to Covid-19 remains.
> During the B.1.617.2 (delta) wave in the United States, similar associations were observed between the use of mRNA vaccines and lower mortality not related to Covid-19 and mortality from any cause.
> Just look up the hospitalization and death rates for folks vaccinated and not. It is stark in difference.
Are those rates an argument against the claim that most people didn't have a bad case, vaccinated or not?
> I had what was probably covid early on. Was like the time I got pneumonia. Asthma attacks in my youth were comparable, if much shorter lived. Getting a positive test case later was something that gave me a fever for a few hours. Scary, due to circumstances. But I was back up and moving in basically no time.
How do you know that your possible second case's low severity is due to the vaccine and not the immunity you would have developed in the first case, or weakening of variants (or some mix of all 3), or even just random chance?
It's hard to ignore personal experience, but it only tells us so much. Like me with my 2 unvaccinated cases having an easy time, I'd be remiss if I generalized that to everyone.
What are you driving at? The rates for vaccinated versus not are a clear indicator that the vaccines helped. Hard to see any other way of interpreting that data.
You are correct that, if I did, in fact, have an early case of covid, I cannot be sure that the vaccine helped me with the later case. So, as far as that goes, my "evidence" is anecdotal at best and can't be taken fully as proof of anything.
You will have a hard time arguing against vaccines with the aggregate evidence above, though.
Sorry, let me clarify. I'm not trying to argue against vaccines.
I entered the thread at
> getting the vaccine basically led to people not getting symptomatic covid. Folks got what they thought of as a bad cold.
I asked for clarity there because it didn't line up with what I understood to be symptomatic covid (have covid and have any symptoms). It sounded like you were really saying the vaccine led to people in your circle not having severe covid.
I believe it is true that the vaccine reduced instances of severe covid. But my point in this thread is that most people already weren't going to have severe covid (based on ifr rates pre vaccine, though hospitalization data would be more useful here).
In other words, "The rates for vaccinated versus not are a clear indicator that the vaccines helped" is true as I understand it, and not something I'm arguing against. It does not contradict "most cases of covid were not severe, vaccinated or not" though.
Ah, fair. I am definitely playing loose in that area.
For specifics in my circle, I really only have my immediate family and some coworkers as direct evidence. Among those, I don't know anyone that got symptomatic anything if they were vaccinated. We had plenty of colds, but only tested positive during a time when that wasn't going through the family. (We only tested due to kid's having contacts that got covid.)
So, to that end, only vaccinated person in the family that ever had symptoms was me. And, as I said, it was super quick. Such that I can't say for sure the kids didn't have symptoms overnight that we just didn't see.
Pulling it back to "most cases overall were not severe," is tough, though. If that is somehow indicative that the vaccines didn't help me, that would also imply that they didn't help the population at large. And the data just doesn't agree with that.
Is that where you are asking? Or did I avoid the question?
I'm just trying to make the point that the vaccines helped at a population level (going from .06% to .0006% or whatever IFR is real numbers when you're talking about the whole world), but I think people overestimate the impact it had on them individually.
And it's easy to see why they would! Given the environment at the time (daily press conferences, scary news articles, demonization of the unvaccinated, mandates) I think it's easy to believe that the vaccine saved you from a death sentence if you get vaccinated and then have an easy case.
It's easy to not notice that in a room of 1667 infected unvaccinated 30 year olds (I don't know how old you are, just using that as an example), maybe over a thousand of them would have had a similar case that you did, and only one of them would have died.
On that, I think I'm in violent agreement with you. In particular, I actually was annoyed with how much stress folks put pre-teens through regarding vaccination. I had friends that were terrified of doing anything with their toddlers before they got vaccinated, despite the odds still being higher for the parents with a vaccine than the kids without. It was truly baffling.
For my part, I suspect it helped me. Childhood asthma and general obesity being what they are. I was almost certainly in elevated risks for my age group. To your point, my age group was still moderate risks, all told.
It seems that Pfizer basically rammed the vaccine through because it prevented covid with 95% efficacy for a couple months and made the case that it was too effective to continue the study.
We now know that antibodies from Pfizer decrease significantly and quickly after a couple months, so it seems very likely that Pfizer knew this as well and decided that after two months was the perfect time to conclude their study and start selling vaccines.
No. The trial was intended to conclude when they had sufficient data to get an acceptable confidence interval. It was to be periodically reviewed to see how it was faring against that yardstick.
They ended up tossing one of the intermediate reviews because it was overtaken by events--the objective was met, spend the time on analyzing that data rather than the now-irrelevant intermediate review.
The test did nothing towards establishing how long the protection lasted--they can't have rushed it through based on that being short because they had no measurement of it then.
You simply can't measure time effects in medicine other than by observing them. If you want to know what protection is like after a year you have to wait a year and then measure it. (This is also why we saw repeated changes to the shelf life of the vaccine--the vaccine makers simply didn't have the time to establish what the true shelf life was and thus could only claim what they had measured. Note that this is pervasive in medicine--stored properly most drugs are effective far beyond the stated shelf life. It's just the manufacturers have no reason to spend the money to certify this.)
And in blaming Pfizer you show your bias--why did every vaccine maker do the same thing at the same time??
If anything I'll blame Pfizer for making a weak vaccine. Moderna chose to go with a higher dose that appears to provide slightly more protection at the cost of more side effects at the time.
To the governments, who have no money but from tax payers.
This I think was the most egregious marketing lie in recent history. That everyone who was jumping up and down for their vaccine was under an impression it was free.
The same people rabbling all day about "transfer of wealth" saw no issue there.
I don't have a stance on covid or vaccines that is terribly unique. But that most people overlooked the massive economic reasons to move in the direction that it did, annoys me.
Downvoted for not parroting Democratic Bay Area values. We will be contacting all FAANG companies and everyone listed in Crunchbase to let them know your an anti-vaxxer.
It was interesting to find out that test subjects who pass away during clinical trials are excluded, erased, from the trial records. Their participation and data is not included in the final results, as if they were never in the trial to begin with.
Many pharmaceutical companies have policies which prohibit patients from participating in studies for anti-depressants if they have suicidal ideations. In practice, this significantly reduces the participation of any patient with depression.
Even if they aren't faked, the ability to shut down a trial that isn't delivering the right preliminary data is a big problem for society. Why should we trust these drugs?
A lot of these studies are prerun in the 3rd world before the real one is done for credit. If sideffects show up the drug can sometimes be mixed with one that has the same known side effects to fool the studies.
Nutrition for me it a big issue. It seems like and ideal area for scientific and medical study to give us light, but seems so hard to get truly objective info. Fat is the devial/good/bad/OK, sugar is the devil/bad/ok, etc. Should I avoid white bread like a hole in the head, or is it fine? Too many agendas and not enough truth.
This is just reporting bias. I've worked (on-and-off) in various scientific fields for ~30; it has always had its bad actors. (I even helped do the statistics for some!)
I'd urge you to consider following the situations:
1. Prescientific inquiry; and,
2. PreFDA food, drug, and medicine.
Both of those were orders-of-magnitude worse than what we have, now. Could we do better? Sure! Is it broken? No.
Disagree. Broken isn't a binary--the current system is far better than what came before, but that doesn't mean there aren't serious flaws in the current system.
This is entropy in action. We're not going to move forward as long as malicious behavior is rewarded with money in large-scale systems. I suspect in coming decades more incurable chronic illness will become rampant and polypharmacy will become more common and necessary for larger percentages of the population. It is really sad to see.
How can you do science if around a quarter of the data is just straight up noise? Even when analysing large amounts of studies the results becone contaminated very easily.
And why is it not standard practice to provide anonymous data or even publish the data? What reason exists for that? So that only the researchers them selves can analyze it?
Yes the problem is that the research system rewards publishing papers, but most of the work is collecting the data. So if you release your data then other groups can write papers based on your effort for far less cost. It's sort of analogous to the problem of open source business models in the software world: if company A writes the code and releases it for free and earns money from running a cloud service, and company B just offers a cloud service, then the second company can get much higher margins because they don't have to develop it.
Unfortunately it's not easy to see what the alternatives are, beyond simply not funding research through government/foundation grants. When science is paid for by companies you don't have this incentive issue because the research is judged based on (ultimately) whether it leads to successful products, not whether it leads to lots of papers getting published. You have other incentive issues of course.
I work in preclinical pharma research, so I have spent a good amount of time trying to recapitulate published data in animal models, so not quite clinical trial data. People who do this work learn how to evaluate trustworthiness. It can be as granular as "this lab is the only one publishing this kind of information, so we'll be skeptical" to a bit more broad "this class of drug isn't expected to have that biology" to "I trust this company over that institution". We route around the fact that some information isn't reliable, and that's not always because of dishonesty or fraud.
I'm not a clinician and don't deal with them regularly, but the impression I get is that new studies are published by researchers who have a lot of connections (dubbed thought leaders). They present at conferences. Other clinicians pick up the use case that matches their need (this patient has failed other therapies for this indication, let's try this new thing I'm now aware of). Then as experience grows, clinicians have more nuanced understanding of the use cases for that new information and its reliability. Frustratingly, this can take years, but that's bug that's also a feature.
That's actually mentioned in the article, which is quite good and worth reading:
> In 2016, the International Committee of Medical Journal Editors (ICMJE), an influential body that sets policy for many major medical titles, had proposed requiring mandatory data-sharing from RCTs. But it got pushback — including over perceived risks to the privacy of trial participants who might not have consented to their data being shared, and the availability of resources for archiving the data. As a result, in the latest update to its guidance, in 2017, it settled for merely encouraging data sharing and requiring statements about whether and where data would be shared.
Privacy concerns are the most important reason. A more cynical reason might that the odds that a papers is deemed 'ok' is far larger if the raw/anonymous data is not provided. Remember, the authors of the suspect studies provided their data voluntarily, and in the end it only hurt their reputation/impact.
But the point of RCTs is that you can get unbiased, high quality results without having to rely on "lore" spreading among medical professionals about which treatments are effective and under which circumstances.
I'm not too knowledgeable when it comes to how scientific experiments/trials are done - are the people who collect the data, the people who interpret the data, and the people who fund/benefit from the data different parties? Or are they the same people?
In my experience, they are different (speaking as someone witnessing it from layman's perspective). Here is what I have seen in Phase II & III trials:
a) Pharma identities the type of patients they need (e.g., 25-50 female, not pregnant, with specific ailment if Phase III), specific tests, and measurements required throughout the study.
b) pharma contacts third party (3P) to manage study patients.
c) 3P has relationship with dozens or even hundreds of doctors' offices, knowing what office can fulfill the test & measure requirement, and has the potential trial patient pool.
d) 3P has existing contract with these doctors' & hospitals. They get patients onto the study. ( <--- #1 reason this is farmed out in my opinion)
e) Doctors & hospitals perform the study and collect the data.
f) doctors & hospitals pass the data to the 3P
g) 3P passes it to the pharma
h) repeat e) through g) as many times as the study requires it. This can be once, or many times over years.
i) pharma pays 3P, 3P pays doctors & hospitals, and they pay the trial patients - each taking their cut along the way.
There are variations on how this is done, sometimes no 3P, sometimes pharma will have their own pool of patients and 3P. Also this is a very rough flow as there are often checks, audits, and validations (should be) done during the study.
Great question, and I'm unfortunately going to have to give the answer of 'it depends'.
Each study is different and therefore run differently. Many, many, factors determine how a study is run, analyzed, and published.
Most studies are very small, using only one site and a few volunteers. Most of these never see the light of day, as the results aren't publishable or are uninteresting. Think power law distributions with studies, not normal distributions. These studies are often so small that the collector and interpreter are the same person, typically they are also the grant writer and admin. If lucky, they may get some nurses or undergrads to help out. Again, I'd say this is ~80% of studies.
The really large studies that places like Pfizer run will separate out nearly all parts of a study. So consenters, nurses, intake, data admin, funding admins, stats guys, etc are all different people. These are very expensive studies to run so it's really only for FDA approval, not scientific inquiry and case studies.
Generally, most studies are very small and not publishable. They don't need to separate out everyone. Everyone kinda trusts that everyone else is doing their best. If something snazzy is found, then follow up studies will build on it's findings. Most of the time though, nothing is really found.
For vaccine manufacturers, it's all the same people. Even if you have whistleblowers come forward, they're ignored.
Here's one quick way to rig any clinical trial: Anyone and everyone that has any kind of negative reaction or health condition gets disenrolled. Since it's 'double blind' it appears on the surface level that there's no way to know who's in what group. Naturally, the end result is always the same: The test group had the same number of reactions as the control by the end of the study.
> "Even if you have whistleblowers come forward, they're ignored."
Do you have any sources for this? I'm rather disbelieving of it, but would love to be proven wrong. I can't imagine that _some_ major news outlet wouldn't love to stick it to the status quo with a whistleblower, unless the "whisteblower" made false claims about their proximity in the company to the dangerous/illegal actions they are trying to bring attention to.
Note that the article discusses research studies and not clinical trials for drug or device approval.
These research studies are important (look at how many were conducted on COVID-19 over the last few years) but are typically not held to a particularly high standard, as with most science. Which doesn’t excuse bad data or poor statistics (the latter supposedly supposed to be picked up in peer review).
Hmm, I read the article as explicitly calling out "clinical trials" (as referenced in the title and abstract) and it makes no reference research studies. I don't understand the distinction between "research studies" and "clinical trials", surely all research studies where an RCT is performed with real patients and real drugs is a clinical trial?
I meant “trials for research studies” as opposed to “trials for drug or device approval.”
The amount of record keeping and oversight of a drug approval trial is enormous (and as a consequence insanely expensive) — data handling, having disjoint groups at each stage handling and analyzing data, etc and detailed records of every manufacturing step — think ISO9000 on steroids.
Nobody would bother to go to that effort for a scientific exploration, nor should they. So the bar is much lower.
I am making no excuse for shoddy science! But it is quite unlikely for a licensed drug.
I don't think it makes sense to publish the results of clinical trials and other scientific experiments until after they have been independently replicated.
My institution's been included in studies primarily for patient access. If we're not included, then good luck getting enough patients, and even then it can take several years to enroll enough patients. Replicating such a trial would be near impossible.
Unfortunately some of the experiments require a whole team to carry out and take a huge amount of time to set up as well as having to be carried out under very specific conditions.
I'd trust the NHS if the National Institute of Clinical Excellence (NICE) actually published their minutes online. Most employees only know what they have been taught, so whilst I agree with the headline that some trial data is dodgy, it doesnt just end there.
There are multiple pathways to factor in, there is redundancy built in, ie secondary pathways, there is the fact that not all chemicals go where intended (best highlighted by radioactive isotopes). When looking at the history of patented medicine, this really started between WW1 and WW2, before WW1, most GP's prescribed what was found in the body on a like for like basis, and in some cases prescribed organs in various forms, like desiccated thyroid gland for thyroid related problems, for pernicious anaemia, raw liver used to be prescribed to women and so on, but that has its problems like contamination and diseases.
So from WW1/WW2 onwards the rise of patented medicine took hold, but the main problem with patented medicine is the human body hasnt evolved to use these new chemical compounds in the same way as unpatented chemicals which have been around for thousands of years. And todays GP's dont really highlight the side effects of the patented medicine, and because they simply dont ask what you have been eating and drinking etc, they have this hubris which sucker punches your trust and sucks you in like a black hole, until before you know it you are on half a dozen different drugs, your quality of life is going down the pan and you've been left a zombie wondering where did Hitler go wrong modelling the German state of the time on the British Empire? You Americans complaining about the cost of healthcare, should count yourself lucky those insurance companies are looking out for your long term interests and theirs!
Fortunately, hospitals in other countries publishes studies and as english seems to be the main language used for science, I have to tip my hat to the Chinese who are roaring up the charts in terms of investigating and publishing relevant studies that will complement a quality of life one hopes to achieve, and we cant forget Wikipedia, Pubmed and Google for connecting users with pertinent studies.
Saying that I do sometimes wonder if something like ChatGPT has written some studies due to the poor quality of english used, but generally they stand out like a sore thumb.
Anyway, does any know why there is a connection between MDMA, blood clotting and Manganese?
The 'vaccine deniers' have been calling the 'studies' and 'trials' trash for a long time. The evidence comes out that they're trash, and somehow the 'vaccine deniers' are still wrong?
They removed liability from manufacturers, and suddenly the CDC schedule exploded with new products. I mean, chicken pox has a vaccine now?
That which was asserted without evidence could correctly be dismissed without evidence. AT THE TIME the anti-vax crowd was basing their positions entirely upon anecdote, rumor, and often badly misread prepublication research and stats. Their methodology was inherently flawed. Even if the conclusions they came to have been "validated" their position was still built upon this same flimsy scaffolding. It's not like the "do your own research" blogs and videos somehow gathered the same evidence used by this paper. This also does not indicate that other positions held by the same crowd, which are similarly based upon "anecdata" and rumor, are somehow made more evident by this paper in Nature.
>That which was asserted without evidence could correctly be dismissed without evidence. AT THE TIME the anti-vax crowd was basing their positions entirely upon anecdote, rumor, and often badly misread prepublication research and stats.
The was an abundance of evidence that the covid vaccines had a reasonable likelihood of being unsafe. Every single previous attempt at a coronavirus vaccine had failed, sometimes catastrophically (killing all the test animals), that's why there wasn't an existing coronavirus vaccine on the market. Every single previous attempt to bring a mRNA treatment to the mass-market had failed due to safety issues. Even in the Pfizer vaccine trial there were overall more deaths in the vaccinated group than the placebo group, due to cardiac deaths (although it wasn't a statistically significant enough amount to draw a conclusion, it does demonstrate that the trial had no power to identify if the vaccine was net-harmful, as it didn't have enough participants to make a meaningful conclusion about the effect of the vaccine on excess deaths).
>Every single previous attempt to bring a mRNA treatment to the mass-market had failed due to safety issues.
Not a single prototype mRNA-based drug passed phase3 trials at any point - right up until the multiple ones within a month of each other were deployed globally.
The massive and remarkable coincidence of that, is truly a special moment in history.
I agree, the assertion that 'vaccines' are safe can be dismissed without evidence. There's no evidence concluding they're actually safe. In fact, we have given the manufacturers immunity because they're 'unavoidably unsafe.'
Just look at how the COVID trials were conducted. They didn't even test each patient. Only some patients that presented symptoms, and then not even all of those patients.
How long did they follow the health outcomes for approved vaccines in the test groups? 3 months at most, and many trials, not even that long. So if someone suffers a neurological condition, well, we just won't know about it.
If you look at the individualized risk for childhood vaccines and flu vaccines, there's effectively zero benefit if you live in a 1st world country, and possibly a great risk of neurological or immunological side effects.
For your comment to make sense to me, you are saying that the pharma sponsored clinical trials are untrustworthy data?
I've not heard that claim but am interested. Overprescripton certainly doesn't require that the trials were bad in any way.
And I'm also curious about this idea of overprescription, because I hear it sometimes from extremely political people but have never heard it from scientists (and scientists are always trying to find some way to critique current practice, so that statins don't rise to that level is a surprise to me).
This. Statins claimed to be free from adverse reactions, but it turns out that about 30% of the participants were taken out of their clinical study because of “non-compliance”. However, if you dig in further, the non-compliance was because of adverse reactions.
You can’t trust pharma companies if their data is secret.
You will likely be able to find scientific studies toward any conclusion you want to make but there is objective evidence of this and in my own personal life significant anecdotal evidence. Given the OP title and topic and the massive reproducibility problem in medicine and perverse incentive problem in medicine, you will not be able to know the truth about the safety of medications to a high degree.
It is like the reviews problem on Amazon. All medications come with studies giving 5 star reviews. But the reality is nowhere near the same. Blackbox warnings and such are all we have to place a significant ding on this problem before taking the drugs off the market for clearly causing intolerable issues in large swaths of people.
This is a complex muddled issue. Argumentation with citations of 'articles' from journals likely will not lead to truthful discovery, unfortunately.
That is medical misinformation. No drug company ever claimed that their statins were free from adverse effects. There are many statins on the market now, and patients who experience bad side effects from one will often do well on another. Getting the treatment right is a trial and error process.
No. no. This is not permitted. Trust the science or you're an antivaxer, Trumper, denier, racist, white supremacist.
Media and gov, or media at the behest of gov (as the twitter files prove, but it was obvious without them), censored as much as thet could, everything that went against what they wanted to push. It wasn't science but security/hygiene theater and it worked, because people did go along and did turn on their neighbours who opposed the measures.
But now we slowly get tidbits of things we can debate again
... Funny that, but no recognition that the entire lockdown, masks and vaccine mandate effort + economic destruction and theft (tax money to corps as "aid") was never a reasonable, logical or scientific response. It was an authoritarian and corrupt response of extreme Effectiveness and cynicism.
But here comes some guy to say "people have always quarantined in pandemics" or some other disingenuous claim that ignores the reality of what happened: healthy people denied basic human and constitutional rights.
The problem here is "healthy people". Covid's real key to success is the fact that most spread is presymptomatic. It's the apparently healthy people spreading it!
And note the lessons of history:
1) There will always be those who choose short term economic interests over safety when the threat isn't absolutely proven. They'll always close the barn door too late.
2) Places that take epidemic/pandemic threats seriously tend to fare better economically in the long run.
No. The problem is that fundamentally, you and people like you decided that authortiarianism is ok if the gov says "things are scary" and everyone should suffer the consequences. No debate allowed.
The risk profile for people was always ridículous and you were never going to contain it once it was widespread, which it was, but somehow we believed in the rolling "it's just 2 weeks".
1) it wasn't short term. It wasn't safety. Short and long term you hurt the vulnerable and you helped the rich and powerful.
2) Rich places will keep being richer and poor places poorer, and when the powers that be decided that enough was enough, all the concerns of the hypochondriacs suddenly didn't matter.
You could tell, if you paid attention, that politicians weren't afraid after the initial surge, but wanted "the masses" to be. You even have definite proof in many places, one of them being UK and number 10. Can't link it now, sorry. But Google downing Street covid rules or something on that note and you can probably find a lot. It wasn't limited to the UK. It was everywhere you looked properly.
You were scammed and you either were well off and didn't mind that much or you want to pretend you weren't for your own mental health. Because the truth is a hard pill to swallow.
Most places threw in the towel when Omicron came along, it was simply too infectious for containment measures to work. However, the same mutation that increased it's spread also reduced it's lethality. The end result was approximately halving the death toll, plus delaying a lot of infections until after the vaccine was available also cut the death toll considerably. All in all millions of lives were saved by the measures you shun.
And it's not a matter of wanting the masses to be scared. Rather, the pattern I have seen again and again of people treating those they regard as good people as safe. That's been the huge chink in the armor against Covid all along. Doctors mostly got infected from coworkers, not patients because they were careful around patients and let down their guard in non-patient areas.
That's not true. Most places still had vaccine mandates with Omicron and heavily propagandized hatred and segregation.
We always knew how Natural immunizaty worked and Omicron was easy to predict, but in 2020 it was a conspiracy theory to mention natural immunity.
> The end result was approximately halving the death toll, plus delaying a lot of infections until after the vaccine was available also cut the death toll considerably. All in all millions of lives were saved by the measures you shun
The halved death toll is a beautiful fabrication of yours. not at all supported by looking at excess deaths, which is another analysis that was frowned upon and ignored while implementing Draconian measures.
You didn't really saved much, you pretend you did and the excesss deaths are now more, funny that. Also, all the suicides, backlogged patients who didn't get treatment in time (cancer deaths) and the indirect deaths and lowered quality of life due to economic impact that disproportionately affects the least wealthy, that's all ignored for the hypothetical "my elephant repellent worked".
> And it's not a matter of wanting the masses to be scared.
WhatsApp messages between politicians and very obvious propaganda (see NHS) Zwith people on respirators say otherwise.
>Rather, the pattern I have seen again and again of people treating those they regard as good people as safe. That's been the huge chink in the armor against Covid all along. Doctors mostly got infected from coworkers, not patients because they were careful around patients and let down their guard in non-patient areas.
I don't even know what you're talking about here.
"Letting the guard down" was always a ridiculous thing when your "guard up" is unscientific and authoritarian hygiene theater. Your politicians didn't care for it but would send the police so you would.
This should be an async non blocking evaluation. The statisticians who do it should be anonymous by default. There should be an appeals process for a scientist to explain why an unconventional new method is actually robust.
There should not be a single number published by this process, but rather a list of stats that speak to the overall quality of the trial on many dimensions (power, sources of bias, etc).
Only information that would not be the same on 99% of trials should be written on this label (no sec style everything is a risk word vomit disclosures).
There should not be a pre-emptive application for a label - it can only be gotten after paper submission to reduce gaming.
There should be an independent advisory org that scientists can literally call to ask for advice on structuring the trials. These calls must not be disclosed. Much like farmers can call the government to ask for help on xyz crop problem.
And these labels should never be used as the primary source of punishment. Any and all sanctions/penalties/dismissals must go through a new review process done by a different group.
Any scientist who gets a label in a particular year should be given a vote to review the review agency on several dimensions. These aggregate reviews should be published broadly but not trigger any automatic consequences.
Clear, accessible information is the basis for any self regulating human system. We need more of it in this field.