> the combined research power of humanity, which is necessary for AGI to advance further, is much deeper than any one entity
AGI won't need people to advance further. That is pretty much the functional definition.
But the bar is even lower. Just as today's rich keep increasing the economic distance between themselves and the poor, with the middle and creative classes already feeling that gravity, so will the AGI's - even if they stalled out only as smart as we are, but cheaper in inference mode. (A limitation that is highly unlikely.)
No doubt humans will facilitate AGI activity in our economy and real world for practical reasons for a little while. But at some point, they won't need us physically or socially either.
> AGI won't need people to advance further. That is pretty much the functional definition.
No, AI can ideate as well as any human, but that is not sufficient. Scientists ideate and test. Engineers design and test. There's always a validation stage, where ideas meet the bottleneck of reality.
Are you saying scientists without labs and tools to run experiments on could do science? It's all in the brain or GPU? That is so naive.
AGI won't need people to advance further. That is pretty much the functional definition.
But the bar is even lower. Just as today's rich keep increasing the economic distance between themselves and the poor, with the middle and creative classes already feeling that gravity, so will the AGI's - even if they stalled out only as smart as we are, but cheaper in inference mode. (A limitation that is highly unlikely.)
No doubt humans will facilitate AGI activity in our economy and real world for practical reasons for a little while. But at some point, they won't need us physically or socially either.