I'm sorry, maybe a didn't communicate clearly. SubiculumCode commented the main part of what I wanted to convey, so I won't repeat.
1. Yes! But that's doesn't exactly change things, in fact, it's part of my point. A big part of why this happened (and still does!) is due to the inherent difficulties and the lack of existing tools. If you ever get a chance, go look at a university physics lab. Even Columbia's nuclear reactors (fusion or fission!) and I think many will be surprised how "janky" it looks. It's because they build the tools along the way, not because lack of monetary resources (well... That too...) but because the tools don't exist!
My critique about the psych field is that this is not more embraced. You have to embrace the fuzziness! The uncertainty. But the field is dominated by people publishing studies that use very simple statistical models, low sample sizes, and put a lot of faith in unreliable metrics with arbitrary cutoffs (most well known being the p-value). Many people will graduate grad school without a strong background in statistics and calculus (it's also easier to think this is stronger than it is. And of course, there are also plenty who would be indistinguishable from mathematicians. But on average?). There are rockstars in every field, even when not recognized as rockstars. But it matters who the field follows.
And I must be absolutely clear, this is not to say that work and those results are useless. Utility and confidence are orthogonal. You might need 5 sigma confidence verified by multiple teams and replicated on different machines to declare discovery of a particle but before that there's many works published with only a few sigma and plenty of purely theoretical works. (Note: in physics replication is highly valued. Most work is not novel and it is easy to climb the academic ladder without leading novel research. This is a whole other conversation though) This is why I discussed Copernicus and Brahe but would not call them astronomers. That's not devaluing them, but rather nothing a categorical difference due to the paradigm shift Kepler caused. Mind you, chemistry even later!
2. I specifically mention economists (my partner is one). I could highlight them more but I feel this would only add to confusion. I believe those close to the details will have no doubt to their role. I don't want to detract from their contribution but I also don't want to convolute my message which is already difficult to accurately convey.
3. I think this is orthogonal. I'm happy to bash on other fields if that makes my comment feel less like an attack rather than a critique (or wakeup call). I'm highly critical of my own community (ML) and believe it is important that we all are *most* critical of our own tribes, because even if we don't dictate where the ship goes we're not far removed from those that do. I'll rant all day if you want (or check my comment (recent) history if you want to know if I'm honest about this). I'll happily critique physicists who can solve high order PDEs in their sleep but struggle with for loops. Or the "retired engineer" trope that every physicist knows and most have experienced.
But it is hard to be critical while not offending (there's a few comments about this too). Maybe you disagree with my critique but I hope you can reread the end of comment as hopeful and encouraging. I want psychology to be taken more seriously. But if the field is unable to recognize why the other fields are dismissive of them, then this won't happen. Sure, there's silly reasons that aren't reasonable, but that doesn't mean there's no reason.
It is a matter of fact that the (statistical) confidence of studies in psychology is much lower than those of the "hard" sciences (physics, chemistry, and yes, even biology (the last part is a joke. Read how you will)). In part this is due to the studies and researchers themselves, but the truth is that the biggest contributing factor is the nature of the topic. That is not an easy thing to deal with and I have a lot of sympathy for it. But how to handle it is up to the field.
1. Yes! But that's doesn't exactly change things, in fact, it's part of my point. A big part of why this happened (and still does!) is due to the inherent difficulties and the lack of existing tools. If you ever get a chance, go look at a university physics lab. Even Columbia's nuclear reactors (fusion or fission!) and I think many will be surprised how "janky" it looks. It's because they build the tools along the way, not because lack of monetary resources (well... That too...) but because the tools don't exist!
My critique about the psych field is that this is not more embraced. You have to embrace the fuzziness! The uncertainty. But the field is dominated by people publishing studies that use very simple statistical models, low sample sizes, and put a lot of faith in unreliable metrics with arbitrary cutoffs (most well known being the p-value). Many people will graduate grad school without a strong background in statistics and calculus (it's also easier to think this is stronger than it is. And of course, there are also plenty who would be indistinguishable from mathematicians. But on average?). There are rockstars in every field, even when not recognized as rockstars. But it matters who the field follows.
And I must be absolutely clear, this is not to say that work and those results are useless. Utility and confidence are orthogonal. You might need 5 sigma confidence verified by multiple teams and replicated on different machines to declare discovery of a particle but before that there's many works published with only a few sigma and plenty of purely theoretical works. (Note: in physics replication is highly valued. Most work is not novel and it is easy to climb the academic ladder without leading novel research. This is a whole other conversation though) This is why I discussed Copernicus and Brahe but would not call them astronomers. That's not devaluing them, but rather nothing a categorical difference due to the paradigm shift Kepler caused. Mind you, chemistry even later!
2. I specifically mention economists (my partner is one). I could highlight them more but I feel this would only add to confusion. I believe those close to the details will have no doubt to their role. I don't want to detract from their contribution but I also don't want to convolute my message which is already difficult to accurately convey.
3. I think this is orthogonal. I'm happy to bash on other fields if that makes my comment feel less like an attack rather than a critique (or wakeup call). I'm highly critical of my own community (ML) and believe it is important that we all are *most* critical of our own tribes, because even if we don't dictate where the ship goes we're not far removed from those that do. I'll rant all day if you want (or check my comment (recent) history if you want to know if I'm honest about this). I'll happily critique physicists who can solve high order PDEs in their sleep but struggle with for loops. Or the "retired engineer" trope that every physicist knows and most have experienced.
But it is hard to be critical while not offending (there's a few comments about this too). Maybe you disagree with my critique but I hope you can reread the end of comment as hopeful and encouraging. I want psychology to be taken more seriously. But if the field is unable to recognize why the other fields are dismissive of them, then this won't happen. Sure, there's silly reasons that aren't reasonable, but that doesn't mean there's no reason.
It is a matter of fact that the (statistical) confidence of studies in psychology is much lower than those of the "hard" sciences (physics, chemistry, and yes, even biology (the last part is a joke. Read how you will)). In part this is due to the studies and researchers themselves, but the truth is that the biggest contributing factor is the nature of the topic. That is not an easy thing to deal with and I have a lot of sympathy for it. But how to handle it is up to the field.