> This emphasis on citations in the measurement of scientific productivity shifted scientist rewards and behavior on the margin toward incremental science
this sounds like the science equivalent of how search engines lead to the creation of content farms. citation scores are to spam science as pagerank is to wikihow?
Imagine going through life with a high-tech spam filter from the future that can filter out wikihow and boring science. Impossible to build, I suspect, but that would be the life.
> this sounds like the science equivalent of how search engines lead to the creation of content farms. citation scores are to spam science as pagerank is to wikihow?
Or a science equivalent to what money and market competition does with all human ventures. You need more and faster than your competitors to progress, else you'll fall into obscurity. "First to publish" is isomorphic to "first to market". If you treat research as a videogame, citations fit perfectly as an in-game currency.
That is to say, it's an example of a general problem of optimizing short-term metrics, which are a decent proxy of the actual goal up to a point.
Science could definitely use a spam filter. Journals unfortunately do a poor way of being one; they're thoroughly gamed, the same way Google is via SEO.
At least in the market a junk product doesn't survive despite being first to market. The product needs to be good or at least inspire others to be regarded as a real contribution. This additional requirement is what's missing from the incentive system in science today.
Tons of junk survives and thrives in the market. Your great grandparent specifically brought up content farms as an example. Homeopathic remedies, counterfeit SD cards, the list goes on.
Nothing about markets magically solves the problem that if it's hard to evaluate quality, there will be junk masquerading as quality.
About the search aspect, I wonder if there is any way to create a custom search engine or filter out crap from google.
I would love a browser addon that auto-filters out quora, wikihow, w3schools, techcrunch, etc...
None of those garbage sites are what I'm looking for, ever. I think that, amusingly, filtering out the top 5% of best SEO optimized sites would make results much better.
w3schools loads faster than MDN but I'm mostly with you
reevaluating our social attitudes towards 'trust in experts' will be an aftermath of the covid crisis, and it may bleed into search & social media as well -- is there value in 'peer-reviewed everything'?
Maybe more multi-disciplined group based (given how much more complex most problems are for individual experts to grapple with by themselves). Probably rated like team sports.
It's doubly tragic if it simultaneously incentivized the mass production of boring, low-value science and punished the boring but valuable work of replication.
>Imagine going through life with a high-tech spam filter from the future that can filter out wikihow and boring science.
Happily it's available now and it's called intuition. If something is boring then avoid; if something is exciting then pursue.
Problems being that it's purely anecdotal and you have to know and trust yourself. If you're too attracted to prestige, money or job security then it's going to return a distorted signal. Which is why organised science is now bureaucratic and slow despite the fact that there are more scientists than ever before.
this sounds like the science equivalent of how search engines lead to the creation of content farms. citation scores are to spam science as pagerank is to wikihow?
Imagine going through life with a high-tech spam filter from the future that can filter out wikihow and boring science. Impossible to build, I suspect, but that would be the life.