The list of faulty of outright fraudulent experiments done in any number of fields, especially the biomedical sciences, also grows ever longer, but it it seems this argument is hardly ever leveled here. It's well-established now that these problems exist in other fields, such as immunology, oncology, and other fields. Even closely related fields, such as the neurosciences, have been shown to be full of improbable and unreplicable findings, and many neuroimaging results are not interpretable in the way that claims are made of them.
Theranos anyone?
Every field should be looked at with caution until the perverse incentives that currently exist in academics are addressed.
What I see instead is a bias in certain physical sciences to think that somehow experiential phenomena are less rigorous because they don't fit the mold of those sciences as much. The systems are more complex and different metaphysically, so their legitimacy is questioned. This is somehow still happening even as developments in fields like quantum physics and AI are leading many very competent scholars to question basic assumptions about the nature of experience and consciousness vis-a-vis physically observable phenomena.
The irony of the Rosenhan study is that Rosenhan was putting forth exactly the same arguments as you, that psychiatry lacks rigor because it's too subject to the whims of subjectivity. So when this paper is shown to have been a fraud (even though it was dismissed in the field for many other reasons, but overall because it was unscientific) it is evidence that psychology is unrigorous? When it is not widely known to be a fraud per se, it is cited as evidence that psychology is unrigorous as well? It seems there's no way to win: the critics of the field cite this work as evidence of lack of rigor, and then when it's shown to be fraudulent, it's also shown as lack of rigor.
The even greater irony is that many of these fraudulent studies are being identified by... you guessed it, psychologists. I would go so far as to say no field has done more for the scientific study of science than psychology. Meta-analysis has its birthplace in psychology, and all these discussions of replicability ultimately flow from psychology as a field. If anything, psychology is among the only ones to be open about these issues and to take them seriously. In many other fields, they're swept under the rug, and questioners are attacked with arrogant hostility and accusations of incompetence.
The issue is that other sciences are not held back by these problems. We still have bridges, semiconductors, space flight and new cancer treatments. This is partly because it is much easier to measure and verify experimental phenomena (you call this a bias, but it is merely a fact?), but also because being correct matters. However many crappy experiments are done or results fabricated, at the end of the day you still have to put your rocket on the launch pad and fire the engines. The reproducibility issues in these sciences is a matter of efficiency.
Psychology is different in the sense that it doesn't seem to matter to the field very much whether they are correct. I say this because they keep doing experiments which lack a robust design, and then proceed to use complex statistical models to infer many unsupported claims which never seem to replicate. Then they meta-analyse these results to conclude that if there is an effect it is small or only applies to certain people. The 'power pose' study I linked is a case in point. All this is done in a very rigorous way by rigorous people using advanced equipment and statistics, who spend much time spinning careful narratives about their work in long Discussion sections. But you don't get a medal for trying in science.
It is hardly a compliment to psychology that they gave birth to the meta-analysis [1]. Note that getting some result in an experiment, then having the result overturned in a meta-analysis is the functional equivalent of not having done the experiment at all. In fact, it is inferior to doing nothing, because you have just wasted everyone's time.
Theranos anyone?
Every field should be looked at with caution until the perverse incentives that currently exist in academics are addressed.
What I see instead is a bias in certain physical sciences to think that somehow experiential phenomena are less rigorous because they don't fit the mold of those sciences as much. The systems are more complex and different metaphysically, so their legitimacy is questioned. This is somehow still happening even as developments in fields like quantum physics and AI are leading many very competent scholars to question basic assumptions about the nature of experience and consciousness vis-a-vis physically observable phenomena.
The irony of the Rosenhan study is that Rosenhan was putting forth exactly the same arguments as you, that psychiatry lacks rigor because it's too subject to the whims of subjectivity. So when this paper is shown to have been a fraud (even though it was dismissed in the field for many other reasons, but overall because it was unscientific) it is evidence that psychology is unrigorous? When it is not widely known to be a fraud per se, it is cited as evidence that psychology is unrigorous as well? It seems there's no way to win: the critics of the field cite this work as evidence of lack of rigor, and then when it's shown to be fraudulent, it's also shown as lack of rigor.
The even greater irony is that many of these fraudulent studies are being identified by... you guessed it, psychologists. I would go so far as to say no field has done more for the scientific study of science than psychology. Meta-analysis has its birthplace in psychology, and all these discussions of replicability ultimately flow from psychology as a field. If anything, psychology is among the only ones to be open about these issues and to take them seriously. In many other fields, they're swept under the rug, and questioners are attacked with arrogant hostility and accusations of incompetence.