> So if you just fund things that are popular, you're kind of stuck
The principled approach is to correct for that by looking for things that ought to be popular by current standards, but are nonetheless underfunded. Yes, this is hard - it's literally trying to beat all other grantors at their own game. It's also supposed to be hard. There was no reason to expect that "funding good work in science" would have an easy, painless solution.
I listened to a podcast a while back that discussed why the US shifting towards this approach (after a period of less constrained funding post WW2) has lead to worse returns on research investment.
The basic premise boiled down to: Scientists already want to work on valuable things, they just have different priorities. Understanding unknowns is motivation enough for them. But this makes the impact of their work also unknown, and scary to people who see that motivation as wasteful.
Making them justify ahead of time why their work will be useful, to satisfy the risk tolerance of financiers, means that they need to reduce that unknown before they even start, and potential impact with it.
"Understanding unknowns" is a perfectly good motivation for a grant - it just means that instead of seeking a grant to do X, you'll be seeking one for what amounts to a feasibility study on X-like stuff; this is not a bad thing for either 'impact' or 'risk tolerance'. Of course, it goes without saying that work where there are fewer "unknowns" to explore will be funded more directly.
But that assumes X is not an unknown, which is kinda the point. "X-like" is an easy pill to swallow for people who want to reduce risk, since they already know X.
If you just want to poke a stick at some lesser known Y, it's harder to appeal to those who want to understand potential returns on their investment.
And it seems that there was a time that predictability, and even returns, was less of a priority.
I can't recall the podcast (maybe Planet Money) but they mentioned Vannevar Bush pushing for the government to act as a patron for sciences, and to take on risk that private investors would avoid. Like, spend up to find a new X, then let the private sector step in and develop the X-likes.
The principled approach is to correct for that by looking for things that ought to be popular by current standards, but are nonetheless underfunded. Yes, this is hard - it's literally trying to beat all other grantors at their own game. It's also supposed to be hard. There was no reason to expect that "funding good work in science" would have an easy, painless solution.