The time horizon that publish or perish forces is not conducive to high variance research. If you do a year of research and get a null result you are screwed, so what do people do? Not pursue research with a significant chance of that happening. But high variance research is exactly what leads to major breakthroughs.
>"If you do a year of research and get a null result you are screwed"
The best I heard on this site was that "alpha is the expected value of p". After a few years of calibration, each field collects just enough data (or adjusts the cutoff) to pump out "statistically significant" results at the appropriate rate. It makes it just hard enough to seem like an accomplishment, but most grad students will get something to publish if they just keep at it.
Of course, that is a totally arbitrary metric that has little to do with the quality of the research.
We set up a control group for science. An entire field of science devoted to studying a phenomenon which doesn't exist. And they still manage to produce tons of significant results and findings, despite using much higher standards than regular social sciences.
> We set up a control group for science. An entire field of science devoted to studying a phenomenon which doesn't exist.
That this "phenomenon" does not and cannot exist is the foundational assumption of modern science. Back in the days when the Laws of Thermodynamics were being figured out to describe the properties of steam engines, a few chemists realized that everything in the universe could be described in terms of indivisible 'atoms'.
Atoms aren't indivisible anymore, but the assumption lives on.
Life is much more interesting if one assumes that our 3d-reality is but a projection of a higher-ordered reality.
I agree with what you said, but high variance research means high variance outcomes... not just scientific but also professional, and not just oneself but also one's grad students and so on. It is a chance we should all take from time to time, but not without calculating the odds and potential consequences, good or bad...
The only way to get high variance research is to offer very long (10 to 15 years), well paid fellowships with no reporting or progress requirements to young and promising scientists. Make these as hard to get as you like, but once awarded leave them alone to get on with research. Anything less than this will just generate iterative research.
I'm not convinced this will work as well as you might hope. The ticking clock, and the high expectations, are still there.
The J.J. Thomson quote still seems applicable, even if the time horizon is slightly longer:
>>> “If you pay a man a salary for doing research, he and you will want to have something to point to at the end of the year to show that the money has not been wasted. In promising work of the highest class, however, results do not come in this regular fashion, in fact years may pass without any tangible results being obtained, and the position of the paid worker would be very embarrassing and he would naturally take to work on a lower, or at any rate, different plane where he could be sure of getting year by year tangible results which would justify his salary. The position is this: You want this kind of research, but if you pay a man to do it, it will drive him to research of a different kind. The only thing to do is pay him for doing something else and give him enough leisure to do research for the love of it.”
That points towards some plausible answers for supporting non-capital-intensive research (e.g. pay to teach instead). Supporting more capital-intensive research, I really don't know.
Maybe one key is to creating a culture evaluating people on effort, talent, process, boldness , etc without being straightly tied to the result - and people who sucseed in doing that well , should have some form of job security , and the knowledge they are doing what they are paid to do.
And if over the long period they fail to produce results, even though they try well, maybe let them work together with someone else who is more lucky with hypoetheses ?
It's an interesting idea, but I suspect that in practice evaluation of these things -- "process" in particular -- will favour the conservative in practice.
This is exactly what my proposal is designed to do. No reporting or progress needs to be given each year. The recipients would have total leisure to do what every they wanted with no pressure to produce results. Sure some of the recipients will do nothing, and some will try and fail, but what we are looking for are the 1 in 100 or 1 in 1000 that produce an amazing breakthrough.
Sure. But I'm pretty sure that people on 10 year fellowships will still very much have their eye on what comes next.
10 year fellowships with a guaranteed pension at the end might just do it -- but funding those for youngish researchers, even in pretty small numbers, would be expensive indeed.
A fellowship of 10 years would give people enough time to take a risk, but still leave enough time to salvage a career if things didn't work out. People would devote the first 5 or 6 years doing high risk / high reward research that had a low chance of success. If things didn't pan out they could spend the last 4 or 5 years generating iterative results and publications.
These fellowships would not be a replacement for a normal academic position, but a way to give young scientists the breathing room to take a risk on high reward research. Given the chance many would - I certainly would have.
The most fun thing to say for a successful researcher is stuff like:
(1) "I wouldn't have been able to do my awesome work in today's research climate"
(2) "Nobody believed me, but then I showed them all..."
(3) "I was a destitute street magician until I discovered probability theory and then won a MacArther"
(4) "My first 10^5 experiments were failures, but I persisted."
(5) "I almost failed high school trig"
(a) ...because it was too boring for me.
(b) ...because i was a misguided pothead back then.
Take these stories as "the set that gets published" because a wide audience feels better about themselves after reading them.
On the other hand, the same public votes for lawmakers who are intent on cutting research funding because they think it is useless (e.g., they randomly pick some funny-sounding grant from NSF, read it on TV in some sarcastic voice, and then declare that all research is bunk).
I see the NSF become less exploratory because of the pressure to "have practical impact," and I believe that is driven by one particular political party in the US.
When so many people present well-conceived ideas that will at least have some payoff (at least in terms of publications), it is hard to justify supporting wonky ideas by people who don't show evidence of mastery in the new area.
How many of you have a total-crank uncle who thinks he has a better idea for solar cells, electric cars, or XYZ?
Well, the same holds if you substitute "uncle" for "professor at a tier-one research university."
There used to be a long tradition of DIY scientists working in their house without a degree. And they were free to explore the ways they wanted. And some were published
Actually opening science to hobbyists would be a good idea. I come from a country where diploma are required to be philosophes, artists and scientists however we hardly win any Nobel prizes or have good artists, while we had hobbyists who actually discovered stuff:
Rousseau applied math for cheating the national lottery, Lavoisier discovered the composition of air and the azote...my grandfather was like that.
But, I guess people already know diploma don't make you anything more than a person with a nice paper to hang on your wall.
The problem is mainly the respect of institutions.
I think that path still exists, especially in my native America, the problem is once you go beyond math and theoretical physics to experimental physics, most chemistry and biology etc. you need serious money and/or stature to arrange access to expensive instruments to do the work, and getting either generally means getting your Ph.D. first (although at least in the US you'll get paid to get it after you pay to get your bachelor's degree).
And you're probably not going to be able to do that, especially legally, if you live in a crowded apartment in a city, plus the days of doing experiments at a country home, or with very simple apparatus albeit in a formal lab like the University of Cambridge's Cavendish Laboratory, seem to be largely over due to the picking of all the low hanging fruit, although I'm sure there are niches where this isn't true, or quite so bad.
Some of this will depend on resourcefulness, like finding bargains on used equipment, but that circles back to first being a part of the scientific community so you've got a network to find out about those opportunities in the first place.
Seems one of the problems is that the funding sources are run by too many old people. They should be forced out every few years and not be allowed to make a career of it.
I'm just a grad student (but I've seen plenty of paper reviews and heard plenty of discussions at conferences), but at least in my corner of academia, the older and more established folks tend to be more visionary: they're more distant from the publish-or-perish scramble of assistant professorship and they've seen cycles of fads come and go. Of course a lot of younger profs bring fresh approaches too but IMHO the tendency to conform to 'accepted' topics to survive is much stronger on the early end of a career.
Anecdote of one, but my funder is quite old and he has taken us on even though our research is not quite in his vein. Moreover, our PI is super old but he's often pushing us to keep fusion in mind, although we are aware that really going down that road in our would be a tough sell for funders.
It isn't quite an age thing as much as it is a "establishment" thing. The establishment is, if you will, older and more established than any one individual, old or young.
In my area (AI), if anything I think the older people in funding bodies are more open to funding a large variety of work, because they've seen multiple cycles of hot techniques coming and going, and lots of eggs being put in a handful of baskets, so tend to think there's value in funding even long-shot and currently-not-hot areas in case something comes out of them long term. Younger people involved in these kinds of funding bodies are much more likely to be the ones pushing for funding to be highly concentrated in a handful of currently hot areas that have clear paths to near-term significant advances, the "we should put 100% of our funding into deep neural networks" type of strategy.
