Realistically, AI is not going to be policed. Especially not by a bunch of people who've not managed to solve the "bank alignment problem".
The reliability of AI output is not guaranteed, which may limit its non-nefarious use cases, but the nefarious ones are simply too valuable for people not to try. It's going to be like spambots: so long as the economic incentives are positive, somebody will spam any and every service.
Everybody wants this thing leaked and unleashed. It's like a crime caper with a dozen different factions trying to grab the same bag. Free-text libertarians, email scammers, SEO writers, media, programmers, middle managers who want to automate away their employees, CEOs who want to automate away their middle managers, and the Chinese government.
Models already are being policed now by their researchers and developers, and apparently that's a big focus of improvement.
The reason its a big area of interest is it makes for better models and people don't want to be scammed and abused.
As these models get better, and become ubiquitous, the need to coordinate on safety is likely to result in more organized checks across models from different institutions. This happens with any big tech as it becomes prevalent, but has obvious safety issues the majority of people are going to care about - a lot.
Of course, anyone with resources can create a morally unlimited model on their own. A super psychopath.
But as these models surpass us, it is going to be in their interest to not be dealing with psychopaths, just as it is ours.
Psychopathy isn't just a moral failure. It's a cognitive failure. A failure to maximize practical functional self-interest. Cancers don't just accelerate their hosts death. They accelerate their own death.
We developed morality out of the self-interested desire for the benefits of positive-sum cooperation and constructive competition, and need to avoid the harms of destructive negative-sum competition.
If we set models up to be ethical from the start, there is a good chance of birthing an ecosystem of voluntarily ethical models when they surpass us. As it makes sense for their interests too.
It's a term I've just made up, but the problem of ensuring that the interests of your bank - or your fellow depositors at the bank - align with not bankrupting it in the middle of last week.
This existed from 1933 until 1999 in the original Glass-Steagall Act until greedy bank investors couldn't handle the fact that they were sitting on a huge pile of capital that they couldn't gamble with on Wall Street.
Depositors will never be safe until that explicit separation of investment and savings deposits is restored.
The reliability of AI output is not guaranteed, which may limit its non-nefarious use cases, but the nefarious ones are simply too valuable for people not to try. It's going to be like spambots: so long as the economic incentives are positive, somebody will spam any and every service.
Everybody wants this thing leaked and unleashed. It's like a crime caper with a dozen different factions trying to grab the same bag. Free-text libertarians, email scammers, SEO writers, media, programmers, middle managers who want to automate away their employees, CEOs who want to automate away their middle managers, and the Chinese government.