Also, a lot of peer reviewers have been trained this way, so even if DO make a paper more readable, you might be endangering it from getting published at all.
So, who is interested in making scientific papers read more easily?
Paper authors are not: they want to look sophisticated, whether their findings are sophisticated or not.
Paper reviewers are not: they want to preserve the usual style, at least for consistency, and also because they are authors of other papers, too.
Readers who are specialists in the field? Maybe, but they are used to the jargon and easily see through it.
Readers outside the field? Maybe, but nobody has an incentive to care about them, they are not reviewers, not potential co-authors (unless you plan a rare cross-disciplinary study), and if they approve grants, it may be better to impress them with the jargon and look important.
The rare curious non-scientist reader? These are a rounding error.
I, for one, care deeply about making my papers readable. I'm outside academia and doing research as a side-hobby, so I couldn't care less about things like tenure etc. So for example, one of my most successful papers [1], I intentionally wrote it in 2 pages (which is quite adequate to get the key idea across), instead of ballooning it into 20 pages. As a result, I had no choice but to publish it in one of the most obscure journals in existence, because almost no-one else accepts 2-page papers in this area these days. Not that it really matters: it still benefited from useful comments by its reviewers, and it's been far more widely read than any of my other papers. But with the journal I published it in, it would never do me a lick of good in an academic job hunt or grant proposal.
in my experience, the best papers are written by people who are not part of the academic incentive system.
We had a problem at UC Berkeley where a very smart CS guy wanted to publish a paper but since he didn't have an official affiliation, major journals wouldn't publish his papers (they literally wouldn't publish a paper by a person whose correspondence address was their home address).
So we gave him a title at Berkeley and the paper was published.
His work is some of the best around, but the academic community wouldn't pay attention until he ran some benchmarks and had them disseminated by prominent academics.
Do you have an academic and/or industry background? I'd be curious to hear about what the experience of publishing in an academic journal as outsider was like.
I'm a math PhD, left academia because I have kids and I wanted to give them a better life than you can give kids with a postdoc salary ;) I can't perceive any differences as an outsider vs. insider. Probably some reviewers subconsciously let it influence their judgment, but at least in some fields you can publish in double-blind reviewed journals where your identity is hidden from the reviewers.
I like my papers to be readable and used to put a lot of effort into making then flow nicely.
Then I got reviews saying I have to cite x, caveat y, relate to z and discuss special cases a,b,c... By the time you've done that there is no way it's going to read nicely any more.
But to play devil's advocate, the reviewers are right. The permanent scientific record does need all the nitpicking details. Papers are not supposed to read like news articles. Now I try to get a good abstract, intro and confusion and accept that the rest will be nit picking.
Could the increased unaccessiblity of the scientific literature be partly responsible for the distrust of the general public for scientists and scientific theories?
A lot of the intelligent people I know who distrust scientists became that way after seeing repeated patterns of poor science and incomplete knowledge being held up as absolute truth, and that "absolute truth" being used as a weapon to bludgeon dissenters.
The most familiar example is nutritional science, and the whole fat vs carbs debate. There are a lot of easy examples in medical science and economics as well. The scary thing is the more scientific articles you read the more you see this pattern everywhere. Scientists have greatly oversold the degree of their knowledge in most fields.
That seems unlikely, the general public doesn't read studies. Their knowledge about studies generally comes from articles written about those studies. Sadly those articles are often sensationalized pieces like "Scientists have harvested cells from jellyfish that could allow us to become immortal" and the likes.
I find it far more likely that the mistrust comes from those articles combined with the fake experts that appear on TV. This is less of an issue in Europe, but it does happen here as well.
Maybe both have the same cause, the need to publish important(-looking) results, or perish? Being scientifically rigorous may take a back seat: unimpressive papers are hard or impossible to publish.
Structurally it's similar to clickbait. I heard that such an incentive structure did bad things to journalism.
Perhaps. Or maybe they're both symptoms of the same root problem: a glut of "scientists" who care less about truth and more about money/academic career. BUT, we shouldn't view these scientists like Disney villains. They're often victims too: it's not their fault they didn't realize the dire circumstances of the modern academic scientist. In a way, they're trapped in a vicious feedback loop: dismal funding forces them to fight dirty to put food on the table, which results in junk science, which results in public distrust, which results in even less funding...
