Note that Rosenhan was a social psychologist. The list of faulty or outright fraudulent experiments done by psychologists grows ever longer. The entire field seems bankrupt to me. Part of the problem are perverse incentives. Get one positive and interesting result (which you can tailor to the zeitgeist for maximum impact) and you can live off books, TED talks and lectures for your whole life. Recent examples include power posing and confidence [1] (poor experiment) or changing political bias regarding gay rights [2] (outright fraud).
If psychology wants the status and rewards of being considered a legitimate science, it needs to make dramatic changes. In the meantime any initial result psychological research produces must be considered not just preliminary, but suspect.
I can't seem to figure out the incentives behind [2], the idea that during door-to-door canvassing only gay people can be effective at changing non-gay people's minds about same-sex marriage seems like an odd narrative for someone to want to push.
Other than encouraging employment of gay individuals by political campaigns/canvassers or maybe a general superiority complex, by stating that only gay people are capable of changing people's minds on the subject.
I have noticed a general pattern on Twitter, from people who push this sort of social science stuff, that they believe only people who are in the in-group (ie, race, gender, class, etc) are allowed to have opinions or engage in research about the group itself. Which always seemed anti-intellectual and borderline dangerous, as it actively discourages the wider population not only from doing useful research but also helping overcome ignorance in their day-to-day conversations, writing books, political projects, etc and generally engaging in issues.
It just makes everyone even more scared, careful, and closed off to the subject(s). Which is the opposite incentive structure for 'producing a cascade of opinion change' (as the study claims).
This just seems to be a punitive approach to change/progress, where oppressed groups, or people representing the groups, are just in it to collect an endless amount of things to shame other people for "not getting it", which ties back into the superiority thing. Assuming the actual goal is ending oppression, this is taking a totally backwards approach by providing ever more ways to minimize and degrade the opinions and personhood of other groups. And when this (self-)destructive strategy is challenged it's met with a bunch of hand-wavy stuff about privilege to justify everything.
Meanwhile the average person on the outside is just trying to live there life and aren't heavily invested. This punitive approach, particularly via social media, would seem to me to make them more likely to just avoid, ignore, or even resent such talk of change.
If you are talking about the seeming 'wokeness' of this research well yes, that is probably a fair comment. But that would be fine if they actually did a proper experiment. I don't think it is a coincidence that particularly woke research may be of poor quality. I agree with you that a strong idealogical bias is a hazard to generating truth. This is most pertinent in preventing certain types of research from being performed or results from being released. This is mostly a problem for psychology and sociology, which among their other merits have limited impact on actual life.
> I can't seem to figure out the incentives behind [2]
If I recall from discussion at the time, the reason the false finding was so welcome (and therefore so rewarding to the con man who wrote it up) was that it purported to show that making a large change to public opinion was cheap and easy to do.
The list of faulty of outright fraudulent experiments done in any number of fields, especially the biomedical sciences, also grows ever longer, but it it seems this argument is hardly ever leveled here. It's well-established now that these problems exist in other fields, such as immunology, oncology, and other fields. Even closely related fields, such as the neurosciences, have been shown to be full of improbable and unreplicable findings, and many neuroimaging results are not interpretable in the way that claims are made of them.
Theranos anyone?
Every field should be looked at with caution until the perverse incentives that currently exist in academics are addressed.
What I see instead is a bias in certain physical sciences to think that somehow experiential phenomena are less rigorous because they don't fit the mold of those sciences as much. The systems are more complex and different metaphysically, so their legitimacy is questioned. This is somehow still happening even as developments in fields like quantum physics and AI are leading many very competent scholars to question basic assumptions about the nature of experience and consciousness vis-a-vis physically observable phenomena.
The irony of the Rosenhan study is that Rosenhan was putting forth exactly the same arguments as you, that psychiatry lacks rigor because it's too subject to the whims of subjectivity. So when this paper is shown to have been a fraud (even though it was dismissed in the field for many other reasons, but overall because it was unscientific) it is evidence that psychology is unrigorous? When it is not widely known to be a fraud per se, it is cited as evidence that psychology is unrigorous as well? It seems there's no way to win: the critics of the field cite this work as evidence of lack of rigor, and then when it's shown to be fraudulent, it's also shown as lack of rigor.
The even greater irony is that many of these fraudulent studies are being identified by... you guessed it, psychologists. I would go so far as to say no field has done more for the scientific study of science than psychology. Meta-analysis has its birthplace in psychology, and all these discussions of replicability ultimately flow from psychology as a field. If anything, psychology is among the only ones to be open about these issues and to take them seriously. In many other fields, they're swept under the rug, and questioners are attacked with arrogant hostility and accusations of incompetence.
The issue is that other sciences are not held back by these problems. We still have bridges, semiconductors, space flight and new cancer treatments. This is partly because it is much easier to measure and verify experimental phenomena (you call this a bias, but it is merely a fact?), but also because being correct matters. However many crappy experiments are done or results fabricated, at the end of the day you still have to put your rocket on the launch pad and fire the engines. The reproducibility issues in these sciences is a matter of efficiency.
Psychology is different in the sense that it doesn't seem to matter to the field very much whether they are correct. I say this because they keep doing experiments which lack a robust design, and then proceed to use complex statistical models to infer many unsupported claims which never seem to replicate. Then they meta-analyse these results to conclude that if there is an effect it is small or only applies to certain people. The 'power pose' study I linked is a case in point. All this is done in a very rigorous way by rigorous people using advanced equipment and statistics, who spend much time spinning careful narratives about their work in long Discussion sections. But you don't get a medal for trying in science.
It is hardly a compliment to psychology that they gave birth to the meta-analysis [1]. Note that getting some result in an experiment, then having the result overturned in a meta-analysis is the functional equivalent of not having done the experiment at all. In fact, it is inferior to doing nothing, because you have just wasted everyone's time.
> The list of faulty or outright fraudulent experiments done by psychologists grows ever longer.
So what social-psychological experiments are bogus?
The most famous problem experiment I can think of is Zimbardo's fake prison, and IIRC the primary objection was that the effect was too strong. He let it go on too long and everyone was disgusted.
Rosenhan's experiment is bogus, yes, but I have to wonder... psychiatrists were condoning lobotomies just 15 years earlier. Is it likely that an entire discipline could turn itself around in that time? (Which doesn't excuse Rosenhan, of course)
I am 100% open to your thesis but would like more data.
The problem with the Stanford Prison Experiment was that the guards were secretly coached by the researchers to get the effect their hypothesis predicted, not that "the effect was too strong": https://www.vox.com/2018/6/13/17449118/stanford-prison-exper...
The results of the Milgram experiments were selectively publicized to sound more provocative. The context of the experiment was the trial of the nazi Adolf Eichmann, who plead during his trial that he was merely "following orders." (There is a myriad of evidence that he was in fact an enthusiastic ideologically motivated nazi who believed in what he was doing.)
The idea that a 'normal person' might become a nazi if ordered by an authority figure was provocative, since it suggested the potential to be a nazi existed in many if not all of us. The disturbing and provocative result of Milgram's experiment seemed to suggest that 'regular people' could indeed become nazis if given orders from an authority figure.
The deceit occurred when the results to publish were cherrypicked from a larger set of experiments performed by Milgram and his team, in which various variables were tweaked. The people from New Haven (home to Yale University) were most likely to comply when given orders from a man dressed like a scientist and were less likely to comply when the orders were given by people in other sorts of costumes. Why? Because people in New Haven had pride in their community, in the university located in their community, and had a belief in the necessity of science. They complied when they were told that compliance would further scientific progress, which they considered to be virtuous. These people were in fact motivated by ideology, just as Adolf Eichmann was. Just as Adolf Eichmann believed in the necessity of the nazi ideology, these people believed in the necessity of scientific progress.
So what did the experiments actually show? It showed that many people are willing to commit atrocities if they believe the ends justify the means. That's not really a provocative result.
I think that early social psychology had a problem with misusing statistics(not necessarily with fraudulent intention) , but I do not believe this extends to psychology, psychiatry, cognitive sciences more generally.
If psychology wants the status and rewards of being considered a legitimate science, it needs to make dramatic changes. In the meantime any initial result psychological research produces must be considered not just preliminary, but suspect.
[1] https://en.m.wikipedia.org/wiki/Power_posing [2] https://science.sciencemag.org/content/348/6239/1100.2