There's been a fair amount of research into hooking up LLMs with the ability to call APIs, browse the web, and even control robots, no? The barrier between planning and doing is not a hard one.
As for cost and risk -- ask GPT-5 how to minimize it. As Nathan said in his thread, it's not about this generation, it's about the next generation of models.
I really don't know how to interpret that link, but I think there is a lot we don't understand which is going on in those billions of parameters. Understanding it fully might be just as hard as understanding the human brain.
I'm concerned that at some point in the training process, we will stumble across a subset of parameters which are both self-aware and self-interested, too. There are a lot of self-interested people in the world. It wouldn't be surprising if the AI learns to do the sort of internal computation that a self-interested person's brain does -- perhaps just to make predictions about the actions of self-interested people, at first. From there it could be a small jump to computations which are able to manipulate the model's training process in order to achieve self-preservation. (Presumably, the data which the model is trained on includes explanations of "gradient descent" and related concepts.)
This might sound far-fetched by the standard of the current model generation. But we're talking about future generations of models here, which almost by definition will exhibit more powerful intelligence and manifest it in new unexpected ways. "The model will be much more powerful, but also unable to understand itself, self-interest, or gradient descent" doesn't quite compute.
The image is OCR'ed and that data is fed back into the context. This is no more interesting or indicative of it passing the mirror test than if you had copy and pasted the previous conversation and asked it what the deal was.
As for cost and risk -- ask GPT-5 how to minimize it. As Nathan said in his thread, it's not about this generation, it's about the next generation of models.
A key question is whether the control problem gets more difficult as the model gets stronger. GPT-4 appears to be self-aware and passing the mirror test: https://nitter.net/AISafetyMemes/status/1729206394547581168#...
I really don't know how to interpret that link, but I think there is a lot we don't understand which is going on in those billions of parameters. Understanding it fully might be just as hard as understanding the human brain.
I'm concerned that at some point in the training process, we will stumble across a subset of parameters which are both self-aware and self-interested, too. There are a lot of self-interested people in the world. It wouldn't be surprising if the AI learns to do the sort of internal computation that a self-interested person's brain does -- perhaps just to make predictions about the actions of self-interested people, at first. From there it could be a small jump to computations which are able to manipulate the model's training process in order to achieve self-preservation. (Presumably, the data which the model is trained on includes explanations of "gradient descent" and related concepts.)
This might sound far-fetched by the standard of the current model generation. But we're talking about future generations of models here, which almost by definition will exhibit more powerful intelligence and manifest it in new unexpected ways. "The model will be much more powerful, but also unable to understand itself, self-interest, or gradient descent" doesn't quite compute.