As a physicist, I'm not happy with the trend of new research fields calling themselves science (mainly for benefiting from the hard-earned respect physics [or natural philosophy, as it was called] has gained throughout centuries, which is the real problem [rather than etymology]), and recently introduced concepts to gain some legitimacy, for justification, such as "hard science", "natural science", etc.
Now the word "science" somehow means "legitimate and respectable research". What's worse, it's not the reality of these fields but a distortion of reality through verbal association, and the word "science" is slowly being dragged into mud due to non-reproducible or downright wrong published results thanks to many such fields of "science".
I should also add that mere "data fitting and data extrapolation" with no basic theory of fundamental understanding isn't science either.
If you're curious about the details, Feynman has defined the issue very well at some point
My background is physics, and I used to think that, but now working as a data scientist that has to deal with human data, I've come to think the opposite. The signal to noise ratio in human data is humongous, nothing is static, and there are zillion of moving parts. That means you need immense volumes of data to run well powered experiments that are designed to get at real causal factors. And there are ethics involved so you have to deal with potential early stopping in a statistically sound way. In some fields (like econ), you don't even really get to do experiments at all on most topics of interest, having to rely on the observations available.
But it turns out that it's worth applying the scientific method to these fields, so what you're left with are tough choices. To deal with these problems the way we would in a physics experiment would be prohibitively expensive, in the literal sense of prohibitive. You have to come to terms with the fact that you can only afford to get enough data that there's a non-negligible possibility of being misled. It's worth doing science here, and we can, but it's just plain hard. I didn't appreciate that before I started having to deal with it. Don't blame the subject for some practitioners' failings.
I think part of the problem is with public interpretation of the results, often as (mis)assisted by the popsci press. Often a sober data scientist will look at a result and conclude "we have gained slight, if any knowledge," while the press will be filled with headlines recommending completely reevaluating life choices and/or the structure of society based on the result.
Thank you for pointing this out! I come from biology, and yes, you almost never get your model to fit the data as well as a physicist would with his - because there are just way too many factors involved! (In fact I would probably treat too high a correlation in a publication as suspicious.) When you're working with living organisms, their sheer complexity and diversity is going to limit the statistical significance of your results (which might be why the urge to p-hack can be so strong). This doesn't mean we're doing bad science; on the contrary, I would argue that we are simply doing harder science...
I think your deluding yourself. If you can't get good data accept that and do something else. Pretending the best you can do must inherently be acceptable is an insidious idea that causes a lot of wasted effort.
I think you misread me completely. I am not saying that it's hard, but this is the best we can do, so we have to do it. I'm saying it's hard, but it turns out to be worth doing despite that. It just turns out that using the scientific method and using sound statistical reasoning from data is a really effective tool for learning about things, even when you cannot make bulletproof true/false statements. Think of it in a bayesian way. A small-medium experiment still moves your belief distribution, despite the fact that you remain relatively uncertain.
A huge range of things are worth doing without being science. Further, just looking at vast amounts of data is not the scientific method.
It's like using vast amount of information to create a self driving car vs. actually letting the car run on real roads. The first can only tell if it approximates reasonable driving, the second can tell if it avoids getting into dangerous situations. You can collect a lot of information on the US economy, but in the real world the FED is actively trying to manage things and you can remove that factor from the data.
Where is anyone proposing just looking at vast amounts of data? With certain kinds of data, you can still learn causal effects observationally, like in econometrics. That's the closest I can find in this discussion. I mean, you're totally right naive data analysis is bad and more data doesn't help that, but nobody is advocating for doing that.
You only have two choices, look at data you don't control or data you do. The entire point of experiments is to narrow the range of uncontrolled data as much as possible. Looking at raw data does not help. Looking at huge data-sets of minimally controlled experimental data does not help.
Physicists's for example can't change the age of the universe they are operating in. It's a rather large unknown, but not exactly an unknown unkown.
At the other end, people trust surveys of eating habits. I don't care if you send out a billion of those things it's still bad data in systematic and changing ways.
In between, most animal studies in mice are looking at disease analog X, in a population of fat, minimally stimulated, etc.
Define "good data". It really does depend on your field. One of our biology professors once told us that any r^2 value above 0.5 could be considered "not bad" on a linear regression in biology - I daresay physicists are used to rather nicer fits than these. But where you draw the line between "good" and "bad" data really is relative to your field of study, and science needs to be evaluated in that context.
While fields may consider X good enough, that does not mean it is good enough. One measurement might be what percentage of published papers are junk. And in that context many fields fail any reasonable metric.
I disagree. I'm glad that science is becoming less associated with the "hard" sciences and is being used more broadly. After all, science is much more than simply the sciences that strictly use the scientific method. The "soft" sciences like psychology are sciences even though their methods are sometimes not easily reproducible. But we shouldn't be resigning outreslves simply because the "soft" science methods are more difficult than the "hard" sciences.
If you go to most universities, psychology is a part of the faculty of Humanities, not science. There's a reason for that.
To call a discipline a science, it needs to use the scientific method, not just sometimes, but always. It needs testable hypotheses. Psychology sometimes has this, but often it does not. There is certainly reason for non-science fields to use scientific methods at times, but that doesn't necessarily mean those fields should be called sciences.
An illustrative example is the difference between "medicine" and "medical science". Your doctor has studied medicine. He or she, in addition to studying some medical science, has learned interview techniques, psychology, and various other aspects of a craft that are, in no way, scientific. If you talk to a doctor in a non-medical setting (many are specifically trained not to reveal ignorance to patients in order to maintain patient confidence), you'll find they're remarkably ignorant about the how's or why's of the human body, except when it comes to something they've been trained to spot and fix. A huge portion of their training centers on knowing when to do nothing at all, because medical intervention almost always carries it's own risks. In his or her daily job, your doctor does not employ the scientific method. At least, you should probably hope you are not being experimented on by your family physician! Some doctors do research in the field of "medical science", but this really is an entirely different job from being a family physician, surgeon, etc.. Medicine is a highly skilled craft that sometimes uses science, but it is not itself a science.
Actually psychology is a part of the sciences in most universities, atleast in Canada. There's a reason for that too, I'm sure.
Also, your assertion that to call a discipline a science, it needs to use the scientific method, is well... demonstrably false, simply by virtue of the fact that we are having this debate.
The scientific method produces very reliable knowledge, but it is not the only way to produce knowledge that is reliable. There is also knowledge that we rely on that is not as rock solid as knowledge obtained from the scientific method, but which is still valuable and still falls under the realm of science, because it is still part of the systematic pursuit of progressively more reliable knowledge.
Indeed! Just look at the puny and half-wrong progress made in physics and compare it to the enormous progress made in astrology and psychology which transformed our lives and understanding of everything!
I mean, they've been at it for hundreds of years, and they still haven't realized that "it is not the only way to produce knowledge that is reliable" (never mind what "reliable" means, who cares about such details anyway). Wish they were also blessed with magic ball.
And hey! They've been sucking our tax money like vampires for so many years!
The replication crisis isn't only within the soft sciences, but nice try.
According to a 2016 poll of 1,500 scientists, 70% of them failed to reproduce another scientist's experiments (50% failed to reproduce their own experiment). These numbers differ among disciplines:
chemistry: 90% (60%),
biology: 80% (60%),
physics and engineering: 70% (50%),
medicine: 70% (60%),
Earth and environment science: 60% (40%).
But it's just the social sciences right? Thank god we have the scientific method! Otherwise we'd be lost! Heaven forbid
Let me guess, you got those "scientific" results from the Scientific Journal of Social Academics, using that "other method that reproduces reliable knowledge", so this must be correct! And it can't have anything to do with the pressure they're feeling now that someone is looking at their work with some scrutiny, because they've been the gate keepers of "reliable knowledge" all along.
And I'm pretty sure those 1500 scientists represent the scientist all over the world from all countries and fields, and those studies must be from the shady physics journals such as Nature and Phys Rev series and not crap journals no real physicist even read. If only they could be like the major psychology journals http://www.nature.com/news/replication-studies-bad-copy-1.10...
You were funny when you tried to justify how scientific psychology was, by using+not using scientific method at will. You sound downright idiotic the moment you tried to imply that physics is a sham (which used to be the de facto science long before any of your "sciences" even appeared).
Anyway, go play with yourself. I've got better things to do. Like actual science. Rumble all you want. I'm done with you.
I'm sorry if I've angered you by showing you that the scientific method doesn't produce perfect science, just like the methods of science that psychology use doesn't produce perfect science either. I'm not rumbling, just giving words of advice that hopefully you'll heed if you actually want to be a scientist and not just a naive lab rat. To each their own though, you just perfectly demonstrated the arrogant attitude that is ruining science today. Why read OP's article when we have you as all the proof we need?
There is a fundamental difference between "half the studies are wrong" and "half of the people have encountered false results (at anytime in their life)". I agree that even "hard" sciences often claim dubious results but a psychology study with 20 (or often even less) participants from a not even remotely representative background that study a plethora of social behaviors is indeed a whole different level of unscientific.
>I disagree. I'm glad that science is becoming less associated with the "hard" sciences and is being used more broadly.
But what do you mean "used"?
In politics, the term "science" is extremely potent. Claiming that science is on your side is commonly done to dramatic effect in order to sway opinions. This effect is very useful in those cases where science tells us something irrefutably important, such as the future of the temperature of Earth's atmosphere or the effectiveness of medical interventions. But it's also very abusable when the standard of proof is relaxed, since it becomes easy to use defeasible reasoning to produce "science" which supports someone's ideology with the borrowed charisma of the physics department.
Right, so long as we can agree that science is not limited to the scientific method. Science should be well defined, but not limited to arbitrary confines.
Oh, so when science is defined in the way Popper defines it, it is "arbitrary confines", but when it is defined in a way to legitimize cargo cult science, it is "well defined".
> Science should be well defined, but not limited to arbitrary confines.
> Well-defined science is frankly an insult to science.
Spoken like a true pseudoscientist.
Yes, it should be an insult, right? How dare they can expect a scientist to use scientific method for all their results? I mean, it sounds totally crazy, insulting! And more importantly, how are we gonna publish papers or get grants then?
Science without the scientific method takes us back to alchemy, or astrology, or numerology. Those precursors eventually refined their methods and developed into sciences, but were not sciences. Most of the "soft" sciences appear to be closer to those protosciences, particularly when so much of the crackpot, discredited theories still circulate in the zeitgeist the way they do, as in, say psychology.
This is exactly what's wrong with those "sciences": you can't do things scientifically (meaning the way "natural" scientists do) when it works for you and do something else when things start to get hairy, and call it science at the end of the day, because "there's more to it than scientific method" or because unicorns and dragons.
That's the "fooling yourself" Feynman talks about.
And no, it's not just that majority of psychology results are "not easily reproducible".
Furthermore, again, science isn't just data fitting and extrapolation. A real theory gives you a lot more than what you put into it, including novel phenomena. Social "sciences" have been playing science for a long time, and still they don't have any universal laws or any deep understanding of what they're studying.
I'm not saying data collection, interpolation and extrapolation (or as Rutherford put it, "stamp collecting") isn't a bad or useless thing. It just isn't science.
Social sciences haven't been "playing science", they are just as scientific as physics. It's just that physics produces more reliable knowledge via the scientific method, while the social sciences produce less reliable knowledge.
The key point here is that science is not limited to the scientific method. The scientific method just produces more reliable knowledge, but is not the be all end all of science, it's just a limited subset, just like the "hard" sciences are a limited subset of science
> Social sciences haven't been "playing science", they are just as scientific as physics.
This sound a lot like trolling, but I'll bite for this once.
You can't have less/more reliable knowledge. Science can't be half-wrong and half-right. You're either right or you're wrong. Your theory can't be vague and non-falsifiable. And you have to get a lot more than what you put into your theory. Regardless of how much you want to call it science, that is just pseudoscience.
I personally really don't want to be associated with such "scientists". Which is why I think twice using the word science/scientist and tend to use the word physics/physicists. All thanks to scientists who are able to think out-of-the-box, unlike the narrow-minded stupid physicists who don't know anything but scientific method!
> You can't have less/more reliable knowledge. Science can't be half-wrong and half-right.
I wasn't trolling, but I think you might be?
Of course science can be more/less reliable or half-wrong/half-right. The truth isn't binary, it's closer to fuzzy logic sets.
And as much as you would like to call the "soft" sciences pseudosceince, you are doing yourself a disservice. In reality, your attitude suggests that you are closer to a pseudoscientist than the "soft" scientists you are so eager to dismiss. Narrow-mindedness like you are epsousing is the reason why science is in the dumps.
The truth is absolute, it's not some fuzzy thing. Your knowledge may be a fuzzy thing, but the truth is not. The problem in the social sciences is that they often ask questions that may not be well defined. So sure, then you end up in fuzzy half truths land, but only because you haven't defined the problem well.
To take a very simple example, let's take "intelligence." Almost immediately we can start arguing about what intelligence is, how to measure it, and so on. Notably, all the participants won't come to any meaningful conclusion because it's a poorly created human construct.
In contrast, the "hard" sciences like physics often deal with something much more clear cut: can you predict the future? If the starting conditions are X, what are the conditions at some time Y?
Can this be applied to the social sciences? We don't remotely know enough about things like the mind to be able to do this. Hell, even in biology the systems are so complex that we are still at the point of mostly guessing.
It's just not the same. Not by any fault of the people trying to study these extremely complex systems of course. Still, at the end of the day we should be honest with ourselves and recognize the differences.
Physics deals in fuzzy things, both theoretically (quantum mechanics and the associated uncertainty) and experimentally (noise is unavoidable and all conclusions require statistical reasoning). But I agree there is a vast amount more "fuzz" in the social sciences, but that seems like a reason to study it more, not less.
ah, good, no substantial reply. I thought you were just trolling and you were. Thank you for justifying everything I just said. Glad to know you agree that science does not = the scientific method
Just like Newton's laws eh? Newton's laws of motion are Right. They are also inaccurate compared to Einstein's theories of relativity, which must mean Newton's rules are Wrong and Einstein is Right. But wait! There's quantum mechanics which proves that Einstein's work can only be Wrong because there's no way his science could be Right while contradicting quantum mechanics, which is Right.
You have asserted that science (now inexplicably a noun) cannot be Half-Right or Less Reliable so are Newton's laws Right or Wrong? Is Relativility Right or Wrong?
Or perhaps Newton was merely "less precise" - maybe another way of putting it is "less reliable"? - then Einstein.
I think you may be firmly in the "wrong" camp, since you insist so in binary classifications.
> You can't have less/more reliable knowledge. Science can't be half-wrong and half-right. You're either right or you're wrong.
No, you're either confident enough the theory holds - at least in some circumstances, confident enough the theory doesn't hold, or you can be not confident enough to say, but you can't ever say your theory is "right". See for example Newton's universal gravitation; it is "mostly right" for most use cases, but it is wrong when general relativity kicks in. And even general relativity is "wrong" when you go into particle physics. It's not as simple as "right or wrong".
I think this is arguing semantics. Yes, the methods and models of physics and psychology are different, but they both are trying to explain natural phenomena in a scientific way, that is, through hypothesis and experiment. The cargo cult analogy may have been apt at the time (1974), but psychology is completely different today. For an example, look at what we now know about visual perception, which goes well beyond a characterization as "stamp collecting".
https://en.wikipedia.org/wiki/Hard_and_soft_science#Criticis...
Now the word "science" somehow means "legitimate and respectable research". What's worse, it's not the reality of these fields but a distortion of reality through verbal association, and the word "science" is slowly being dragged into mud due to non-reproducible or downright wrong published results thanks to many such fields of "science".
I should also add that mere "data fitting and data extrapolation" with no basic theory of fundamental understanding isn't science either.
If you're curious about the details, Feynman has defined the issue very well at some point
http://calteches.library.caltech.edu/51/2/CargoCult.pdf