> I am interested in the things you've said and linked, and my opinions have changed, but I think I'm well past what I am able to follow or interested to continue.
Ok, that's unfortunate but obviously your prerogative! I will try to post some more lay-friendly links in this response that may convey my points more straightforwardly.
> if the CAC score measures something is it always definitely plaque buildup? Because then repeated CAC scores have the potential to be good for telling if plaque buildup is changing over time in response to dietary changes in people with non-zero scores and CAD.
No, it can measure plaque density. Additionally, since calcified plaques are more stable than soft plaques, it can actually go up while risk goes down. We see this occur on some statin regimes, where we have boatloads of data showing statins reduce risk, yet we can see CAC scores go up as soft plaque is stabilised into calcified plaque.
I'm not saying CAC is without its merits, but you are stretching credulity beyond breaking point to suggest that because one cardiologist claims that reducing dietary fats doesn't show a reduction in CAC scores, it therefore disproves the lipid hypothesis. CAC has too many caveats to use it to make such bold inferences, even if we get to the wild epistemic standards required to think one cardiologist's claims are sufficient to overturn decades of well-conducted research. Additionally, as I've stated elsewhere, it still wouldn't be evidence that arterial wall damage or inability to repair damage is required for ASCVD, which, let's remember, is the reason we're even talking about this in the first place.
There's quite an interesting discussion about the pros and cons of the CAC here. Suffice to say, it's not suitable to stand up the kind of arguments you're making: https://www.youtube.com/watch?v=gxIeRUbHauw
> Because when someone you don't like cites multiple meta-analyses of RCTs you handwave it away with a Twitter thread where he "runs away from an argument chef's kiss"
No, that's not why I dismiss Mason's claims. I dismiss Mason's claims because he says "this meta analysis says this" and then when you look at the MA it says nothing of the sort. It's false to claim that I'm handwaving away the evidence itself.
> I know of Ben Goldacre's Bad Science, I see plenty of HN links about the reproducibility crises in science about the ways of p-hacking and manipulating data, about the broken incentives to publish and get funding, about the complexities of statistical analysis, things I can't argue through. Medical experts are incentivised to push treatments, academic experts are incentivised to push dodgy results, it's not easy for people who like and care about this stuff to say what's "high quality data" or isn't, let alone me. One can't handwave away measuring the amount of plaque buildup in the heart and say "that's not a measure of the amount of plaque buildup in the heart" as easily. If people change X and their heart health measurably improves, and X isn't reducing dietary fat, that should mean something.
I think it's really worth taking a second to think about what you're saying here. Some points of order first:
- The reproducibility crisis is about some fields of science re-running individual studies and finding that when they do so, they cannot reproduce the original findings.
- p-hacking is about altering the methodology of a study post-hoc to generate statistically significant findings when such findings were not present under the original methodology.
- the objection of the complexities of statistical analyses, I presume, is an objection based on the fact that when using statistical analyses to try to account for confounders and systematic bias, sometimes these analyses are poorly conducted and thus fail to mitigate these biases.
- "Medical experts are incentivised to push treatments" is presumably an objection that individuals publishing papers may have an implicit or explicit bias that causes them to over or understate the significance of a given finding, or record data incorrectly.
So your argument appears to be:
"Ok, so you've provided evidence that LDL is a risk for CVD based on over 200 prospective cohort studies (that is, the results have been successfully produced literally hundreds of times), mendelian randomisation studies, and randomised controlled trials including more than 2 million participants with over 20 million person-years of follow-up and over 150 000 cardiovascular events. However, I believe it's likely that if we were to re-run those studies again, we would not be able to reproduce those results.
“I'm also concerned that p-hacking and systematic bias is so consistently prevalent within all of these studies that it affects the summated findings so profoundly, and consistently in the same direction, that we cannot trust the findings to be accurate. Additionally, the individuals conducting these studies are all incentivised to push treatments and therefore we should believe it to be more likely than not, without any evidence to actually demonstrate this, that these individuals are biasing the results because of these incentives.
“It's not easy for lay people like me to ascertain whether these data from the top of the evidence hierarchy (that is, the evidence least likely to be affected by bias) are high quality, so we have to find a source of evidence that is less likely to be affected by such bias. To achieve this, I propose that we throw all the above data out, and instead trust the claims of a single cardiologist, who's written a blog post on his website about how serum cholesterol is irrelevant when it comes to cardiovascular disease risk.
“After all, p-hacking can't take place when the evidence is so poor that there are no p-values, calculations of statistical significance, or even recorded data from which one could calculate p-values.
“After all, you can't make errors in your statistical analyses if you don't perform any statistical analyses.
“After all, your attempts to control for bias can't go wrong if you don't attempt to control for bias.
“After all, 'One can't handwave away measuring the amount of plaque buildup in the heart and say "that's not a measure of the amount of plaque buildup in the heart" as easily. If people change X and their heart health measurably improves, and X isn't reducing dietary fat, that should mean something,' despite the fact that there are studies that do measure plaque buildup in the heart and show that it is affected by serum cholesterol. I believe that we can handwave it away when it's published in a peer reviewed journal, but we can't do so when it's a blog post on a cardiologist's website. This is the most reliable way to make sound scientific inferences.”
I don't even understand how one can reach these conclusions. I've tried my absolute best to interpret what you've said generously, but there's only so much I can do given material like this.
This whole line of reasoning is also far closer to what you’re accusing me of - instead of discussing what the data show, we should just “trust the experts”. We can’t analyse Dr Davis’ data because he doesn’t even provide any, but we should just assume he’s telling the truth because he’s a cardiologist and therefore so well paid that he has no motivation to be wrong. We should ignore the evidence I’ve provided because it’s harder for laypeople to understand than this supposed expert just saying “trust me, I ran some CAC tests”.
> presumably somewhere is a researcher or practitioner who has done many CAC measurements and has studied whether cutting dietary fat changes the score or not, at this point I am not going to look for it.
I've already provided you even better evidence than that: a pooled analysis of studies where SFA is replaced with PUFA and cardiovascular events and coronary deaths are reduced. You don't need to go looking, I gave it to you: https://pubmed.ncbi.nlm.nih.gov/19211817/. If you want a more consumable version, here's an excellent podcast on it (all their stuff is good, you should see if there's anything else there of interest to you: https://sigmanutrition.com/episode481/).
It's a shame, but not entirely surprising that you no longer desire to continue the discussion. Fundamentally, I think you will always run into issues until you re-assess your underlying epistemic framework when it comes to assessing evidence. Until you do so, reading studies will be largely pointless as you have no reliable tools to interpret those studies. You're running before you can walk.
If you're interested, the Cut Through Nutrition podcast is quite a nice primer on how to think about nutrition science research, I'd highly recommend working through the episodes. Once you've internalised the concepts, it becomes a lot easier to see how quacks like Mason and Davis are operating, even without a great deal of domain-specific knowledge. Podcast is here: https://podcasts.apple.com/gb/podcast/cut-through-nutrition/...
Ok, that's unfortunate but obviously your prerogative! I will try to post some more lay-friendly links in this response that may convey my points more straightforwardly.
> if the CAC score measures something is it always definitely plaque buildup? Because then repeated CAC scores have the potential to be good for telling if plaque buildup is changing over time in response to dietary changes in people with non-zero scores and CAD.
No, it can measure plaque density. Additionally, since calcified plaques are more stable than soft plaques, it can actually go up while risk goes down. We see this occur on some statin regimes, where we have boatloads of data showing statins reduce risk, yet we can see CAC scores go up as soft plaque is stabilised into calcified plaque.
I'm not saying CAC is without its merits, but you are stretching credulity beyond breaking point to suggest that because one cardiologist claims that reducing dietary fats doesn't show a reduction in CAC scores, it therefore disproves the lipid hypothesis. CAC has too many caveats to use it to make such bold inferences, even if we get to the wild epistemic standards required to think one cardiologist's claims are sufficient to overturn decades of well-conducted research. Additionally, as I've stated elsewhere, it still wouldn't be evidence that arterial wall damage or inability to repair damage is required for ASCVD, which, let's remember, is the reason we're even talking about this in the first place.
There's quite an interesting discussion about the pros and cons of the CAC here. Suffice to say, it's not suitable to stand up the kind of arguments you're making: https://www.youtube.com/watch?v=gxIeRUbHauw
> Because when someone you don't like cites multiple meta-analyses of RCTs you handwave it away with a Twitter thread where he "runs away from an argument chef's kiss"
No, that's not why I dismiss Mason's claims. I dismiss Mason's claims because he says "this meta analysis says this" and then when you look at the MA it says nothing of the sort. It's false to claim that I'm handwaving away the evidence itself.
> I know of Ben Goldacre's Bad Science, I see plenty of HN links about the reproducibility crises in science about the ways of p-hacking and manipulating data, about the broken incentives to publish and get funding, about the complexities of statistical analysis, things I can't argue through. Medical experts are incentivised to push treatments, academic experts are incentivised to push dodgy results, it's not easy for people who like and care about this stuff to say what's "high quality data" or isn't, let alone me. One can't handwave away measuring the amount of plaque buildup in the heart and say "that's not a measure of the amount of plaque buildup in the heart" as easily. If people change X and their heart health measurably improves, and X isn't reducing dietary fat, that should mean something.
I think it's really worth taking a second to think about what you're saying here. Some points of order first:
- The reproducibility crisis is about some fields of science re-running individual studies and finding that when they do so, they cannot reproduce the original findings.
- p-hacking is about altering the methodology of a study post-hoc to generate statistically significant findings when such findings were not present under the original methodology.
- the objection of the complexities of statistical analyses, I presume, is an objection based on the fact that when using statistical analyses to try to account for confounders and systematic bias, sometimes these analyses are poorly conducted and thus fail to mitigate these biases.
- "Medical experts are incentivised to push treatments" is presumably an objection that individuals publishing papers may have an implicit or explicit bias that causes them to over or understate the significance of a given finding, or record data incorrectly.
So your argument appears to be: "Ok, so you've provided evidence that LDL is a risk for CVD based on over 200 prospective cohort studies (that is, the results have been successfully produced literally hundreds of times), mendelian randomisation studies, and randomised controlled trials including more than 2 million participants with over 20 million person-years of follow-up and over 150 000 cardiovascular events. However, I believe it's likely that if we were to re-run those studies again, we would not be able to reproduce those results.
“I'm also concerned that p-hacking and systematic bias is so consistently prevalent within all of these studies that it affects the summated findings so profoundly, and consistently in the same direction, that we cannot trust the findings to be accurate. Additionally, the individuals conducting these studies are all incentivised to push treatments and therefore we should believe it to be more likely than not, without any evidence to actually demonstrate this, that these individuals are biasing the results because of these incentives.
“It's not easy for lay people like me to ascertain whether these data from the top of the evidence hierarchy (that is, the evidence least likely to be affected by bias) are high quality, so we have to find a source of evidence that is less likely to be affected by such bias. To achieve this, I propose that we throw all the above data out, and instead trust the claims of a single cardiologist, who's written a blog post on his website about how serum cholesterol is irrelevant when it comes to cardiovascular disease risk.
“After all, p-hacking can't take place when the evidence is so poor that there are no p-values, calculations of statistical significance, or even recorded data from which one could calculate p-values.
“After all, you can't make errors in your statistical analyses if you don't perform any statistical analyses.
“After all, your attempts to control for bias can't go wrong if you don't attempt to control for bias.
“After all, 'One can't handwave away measuring the amount of plaque buildup in the heart and say "that's not a measure of the amount of plaque buildup in the heart" as easily. If people change X and their heart health measurably improves, and X isn't reducing dietary fat, that should mean something,' despite the fact that there are studies that do measure plaque buildup in the heart and show that it is affected by serum cholesterol. I believe that we can handwave it away when it's published in a peer reviewed journal, but we can't do so when it's a blog post on a cardiologist's website. This is the most reliable way to make sound scientific inferences.”
I don't even understand how one can reach these conclusions. I've tried my absolute best to interpret what you've said generously, but there's only so much I can do given material like this.
This whole line of reasoning is also far closer to what you’re accusing me of - instead of discussing what the data show, we should just “trust the experts”. We can’t analyse Dr Davis’ data because he doesn’t even provide any, but we should just assume he’s telling the truth because he’s a cardiologist and therefore so well paid that he has no motivation to be wrong. We should ignore the evidence I’ve provided because it’s harder for laypeople to understand than this supposed expert just saying “trust me, I ran some CAC tests”.
> presumably somewhere is a researcher or practitioner who has done many CAC measurements and has studied whether cutting dietary fat changes the score or not, at this point I am not going to look for it.
I've already provided you even better evidence than that: a pooled analysis of studies where SFA is replaced with PUFA and cardiovascular events and coronary deaths are reduced. You don't need to go looking, I gave it to you: https://pubmed.ncbi.nlm.nih.gov/19211817/. If you want a more consumable version, here's an excellent podcast on it (all their stuff is good, you should see if there's anything else there of interest to you: https://sigmanutrition.com/episode481/).
It's a shame, but not entirely surprising that you no longer desire to continue the discussion. Fundamentally, I think you will always run into issues until you re-assess your underlying epistemic framework when it comes to assessing evidence. Until you do so, reading studies will be largely pointless as you have no reliable tools to interpret those studies. You're running before you can walk.
If you're interested, the Cut Through Nutrition podcast is quite a nice primer on how to think about nutrition science research, I'd highly recommend working through the episodes. Once you've internalised the concepts, it becomes a lot easier to see how quacks like Mason and Davis are operating, even without a great deal of domain-specific knowledge. Podcast is here: https://podcasts.apple.com/gb/podcast/cut-through-nutrition/...
Have a good week!