it's a little annoying that the "AI Impacts" paper seems to be three MIRIs in a trenchcoat, considering that this article is about EA's AI agenda and by extension Singularity/MIRI. There is a slight conflict of interest here - as the publishers of the paper are also taking funding specifically to "deal with the threat" they're publishing on.
Yes if we have scientists that are getting paid to research a certain risk it’s inevitable that they have conflict of interest. Otherwise how would it be done?
I'll admit, there isn't actually a good solution to this. It's one of the core failures of the way we organize society and its resources.
I find this kind of fascinating, really. Because of the way we sort work and create institutions, there's a structural hook that ties in the persistence of the problem (or the creation of a perception of the problem) with the longevity and motivation of the institution / entity that is attempting to solve it.
Yeah, you could contract out to a third party (which would be better), but you're still establishing a relationship where their future earnings are dependent on coming up with a certain result, unless everyone involved is incredibly self aware and conscientious and is willing to work to put themselves out of a job. Not impossible, but a rarity.
Take the DEA, for example. Every indicator points to deprecation as the way forward. The science says that treating drugs like a crime is ineffective and creates all sorts of externalities, and there isn't even broad public support for the War On Drugs. It's expensive, ineffective, dangerous, wasteful. And it keeps an absolute fuckton of people employed, at all levels. These people have a vested interest in maintaining the status quo, if for no other reason than their positions afford them power, quality of life, etc. So they lobby to keep stuff illegal and go on campaigns attempting to convince people that they'll peel themselves if they get high.
I'm not suggesting that EA and this AI thing is comparable to the DEA, mind you. If given the choice to keep one I'd be hanging out with you guys. That said, I'm ever leery of surveys in general, and there's a recursiveness to using your own data to back up your claims that doesn't sit well. If it's truly something that warrants money / time / attention, there's gotta be external sources that corroborate - Which I don't necessarily doubt, but that's not the thing we're looking at, here.
