>"try to imagine superintelligence but without consciousness."
The only thing that comes to mind is how many different things come to mind to people when the term "superintelligence" is used.
The thing about this imagination process, however, is that what people produce is a "bag of capacities" without a clear means to implement those capacities. Those capacities would be "beyond human" but in what direction probably depends on the last movie someone watched or something similarly arbitrary 'cause it certainly doesn't depend on their knowledge of a machine that could be "superintelligent", 'cause none of us have such knowledge (even if this machine could go to "superintelligence", even our deepmind researchers don't know the path now 'cause these are being constructed as a huge collection of heuristics and what happens "under the hood" is mysterious to even the drivers here).
Notably, a lot of imagined "superintelligences" can supposedly predict or control X, Y or Z thing in reality. The problem with such hypotheticals is that various things may not be much more easily predictable by an "intelligence" than by us simply because such prediction involves imperfect information.
And that's not even touch how many things go by the name "consciousness".
The only thing that comes to mind is how many different things come to mind to people when the term "superintelligence" is used.
The thing about this imagination process, however, is that what people produce is a "bag of capacities" without a clear means to implement those capacities. Those capacities would be "beyond human" but in what direction probably depends on the last movie someone watched or something similarly arbitrary 'cause it certainly doesn't depend on their knowledge of a machine that could be "superintelligent", 'cause none of us have such knowledge (even if this machine could go to "superintelligence", even our deepmind researchers don't know the path now 'cause these are being constructed as a huge collection of heuristics and what happens "under the hood" is mysterious to even the drivers here).
Notably, a lot of imagined "superintelligences" can supposedly predict or control X, Y or Z thing in reality. The problem with such hypotheticals is that various things may not be much more easily predictable by an "intelligence" than by us simply because such prediction involves imperfect information.
And that's not even touch how many things go by the name "consciousness".