(Sorry, this is a bit rambling. Hopefully it'll still be interesting to some of you. Have had a few pints at this point...)
EDIT: Actually, this is nearing "crazy" levels. Just ignore unless you really enjoy stream-of-consciousness. Sorry about this, HN! :)
I know I'm really late to the party here, but there's a premise in this whole discussion that I'm not sure I understand.
Why should we prevent AI from taking over? I mean, I "get it"... it wouldn't be HI and that feels kind of weird, but what's objectively special about HI? Why are we treating "HI==good" as axiomatic? I mean even us tribal, overly-emotional (&c) humans value DI (Dog Intelligence) even if we're pretty sure that it can't contemplate the fact that we're all made of the remnants of supernovae. There's no evidence as of yet that a greater intelligence a) exists[1], even in principle, or b) would be any less benevolent towards us. Perhaps they would even create nice little simulations for us to exist in. Though, I wonder what the purpose of my simulation is, given current circumstances :).
Yes, a transition from HI->AI would inevitably lead to a lot of human death (unless we're talking really out-there take-over plans involving disease and such), but would AI really be worse? And for whom and why? Humans themselves have caused a lot of death and we seem to value ourselves pretty highly overall (and undeservedly, IMO).
It might be that HI is the "end of the road" just like the Turing Machine appears to be the end of the road in terms of what you can compute... but not in terms of how fast you can compute it. Would "faster" automatically mean "better" (see footnote)? I dunno.
[1] The existence of a "higher" ("faster" is probable) intellige is a interesting question. How would you judge such a thing? Is there more "power"[2] to be gained through something other than being able to reflect at yourself? AFAIUI self-reflection is one of the distinuguishing features of intelligence, but given that we're "better" than Chimps -- who have an idea of "self", thus self-reflection -- it may not be the decider. And even so, such reflection is still subject to Physics and thus without "free will".
[2] Not just faster, but "better", in some non-linear sense.
EDIT: Actually, this is nearing "crazy" levels. Just ignore unless you really enjoy stream-of-consciousness. Sorry about this, HN! :)
I know I'm really late to the party here, but there's a premise in this whole discussion that I'm not sure I understand.
Why should we prevent AI from taking over? I mean, I "get it"... it wouldn't be HI and that feels kind of weird, but what's objectively special about HI? Why are we treating "HI==good" as axiomatic? I mean even us tribal, overly-emotional (&c) humans value DI (Dog Intelligence) even if we're pretty sure that it can't contemplate the fact that we're all made of the remnants of supernovae. There's no evidence as of yet that a greater intelligence a) exists[1], even in principle, or b) would be any less benevolent towards us. Perhaps they would even create nice little simulations for us to exist in. Though, I wonder what the purpose of my simulation is, given current circumstances :).
Yes, a transition from HI->AI would inevitably lead to a lot of human death (unless we're talking really out-there take-over plans involving disease and such), but would AI really be worse? And for whom and why? Humans themselves have caused a lot of death and we seem to value ourselves pretty highly overall (and undeservedly, IMO).
It might be that HI is the "end of the road" just like the Turing Machine appears to be the end of the road in terms of what you can compute... but not in terms of how fast you can compute it. Would "faster" automatically mean "better" (see footnote)? I dunno.
[1] The existence of a "higher" ("faster" is probable) intellige is a interesting question. How would you judge such a thing? Is there more "power"[2] to be gained through something other than being able to reflect at yourself? AFAIUI self-reflection is one of the distinuguishing features of intelligence, but given that we're "better" than Chimps -- who have an idea of "self", thus self-reflection -- it may not be the decider. And even so, such reflection is still subject to Physics and thus without "free will".
[2] Not just faster, but "better", in some non-linear sense.