> Automated task evaluations have proven informative for threat models where models take actions autonomously. However, building realistic virtual environments is one of the more engineering-intensive styles of evaluation. Such tasks also require secure infrastructure and safe handling of model interactions, including manual human review of tool use when the task involves the open internet, blocking potentially harmful outputs, and isolating vulnerable machines to reduce scope. These considerations make scaling the tasks challenging.
That's what to worry about - AIs that can take actions. I have a hard time worrying about ones that just talk to people. We've survived Facebook, TikTok, 4chan, and Q-Anon.
Talking to people is an action that has effects on the world. Social engineering is "talking to people". CEOs run companies by "talking to people"! They do almost nothing else, in fact.
That's what to worry about - AIs that can take actions. I have a hard time worrying about ones that just talk to people. We've survived Facebook, TikTok, 4chan, and Q-Anon.