This is what I think about an imminent AI danger and those who believe in it:
1. It’s not as near as you think. Current machine “intelligence” is hardly on par with the abilities of an insect, let alone anything we’d call intelligent. Yeah, it’s capable of analyzing a lot of data, but computers could always do that. We don’t really know how to move forward, and there has been little progress in theory in a long while. We haven’t made any significant theoretical breakthrough in 20-40 years.
2. Intelligence isn’t a general algorithm to solve everything. For example, humans are not good at approximating solutions to NP-complete problems. Other algorithms make a better use of computational resources to solve problems that intelligence is not good at, and super-intelligence is not required to come up with those algorithms, as they use brute force on cheap computing nodes. Intelligence also isn't necessarily good at solving human problems, many of which require persuasion through inspiration or other means.
3. We don’t know what intelligence is, so we don’t know if the intelligence algorithm (or a class of algorithms) can be improved at all. Simply running at higher speeds is no guarantee of more capabilities. For all we know, an intelligent mind operating at higher speed may merely experience time as moving very slow, and grow insane with boredom. Also, we don’t know whether the intelligence algorithm can multitask well to exploit that extra time.
But most interesting at all is what I think is at the core of the issue:
4. Intelligence is not that dangerous — or, rather, it’s not so much more dangerous than non-intelligent things. This is related to point 2 above. We can obviously see that in nature, but also in human society. Power correlates weakly with intelligence beyond some rather average point. Charisma and other character traits seem to have a much bigger impact on power. Hitler wasn’t a genius. But some smart people — because they smarter than average — fantasize about a world where intelligence is everything, and not in a binary way, but in a way that gives higher intelligence non-linearly more power. This is a power fantasy, where the advantage they possess translates to the power they lack.
1. It’s not as near as you think. Current machine “intelligence” is hardly on par with the abilities of an insect, let alone anything we’d call intelligent. Yeah, it’s capable of analyzing a lot of data, but computers could always do that. We don’t really know how to move forward, and there has been little progress in theory in a long while. We haven’t made any significant theoretical breakthrough in 20-40 years.
2. Intelligence isn’t a general algorithm to solve everything. For example, humans are not good at approximating solutions to NP-complete problems. Other algorithms make a better use of computational resources to solve problems that intelligence is not good at, and super-intelligence is not required to come up with those algorithms, as they use brute force on cheap computing nodes. Intelligence also isn't necessarily good at solving human problems, many of which require persuasion through inspiration or other means.
3. We don’t know what intelligence is, so we don’t know if the intelligence algorithm (or a class of algorithms) can be improved at all. Simply running at higher speeds is no guarantee of more capabilities. For all we know, an intelligent mind operating at higher speed may merely experience time as moving very slow, and grow insane with boredom. Also, we don’t know whether the intelligence algorithm can multitask well to exploit that extra time.
But most interesting at all is what I think is at the core of the issue:
4. Intelligence is not that dangerous — or, rather, it’s not so much more dangerous than non-intelligent things. This is related to point 2 above. We can obviously see that in nature, but also in human society. Power correlates weakly with intelligence beyond some rather average point. Charisma and other character traits seem to have a much bigger impact on power. Hitler wasn’t a genius. But some smart people — because they smarter than average — fantasize about a world where intelligence is everything, and not in a binary way, but in a way that gives higher intelligence non-linearly more power. This is a power fantasy, where the advantage they possess translates to the power they lack.