Hacker News new | past | comments | ask | show | jobs | submit login

The power of an AGI goes far beyond intelligence. It could generate millions of its own instances (possibly with variations) and hack on thousands of systems at the same time, actively adapting each one as needed. It could perform social engineering on a large number of people simultaneously and communicate in ultra-high bandwidth and protocols that most humans involved cannot follow.

A likely potent form of gaining power may involve assuming the identities of people one knows, via text messaging or email, and use that to gain access to important information. The aggregation and superhuman-level analysis of private information thousands of times faster than humans would bring enormous financial and social power to the AGI. Given the sad state of security in most systems, and the relative ignorance of information security issues among many political and business leaders, an AGI will not have much problem with this sample plan off the top of my head.

And there are many other more clever plans an AGI could come up with that we have not yet considered...




Suppose AI 1 uses that approach but Gmail has AI 2 acting as a spam filter. Suddenly phishing becomes much more difficult.

Further, spinning off other copies requires hardware to run them. If MS has an AI 3 go over windows source code it may be much harder to hack desktops.

Without authorial fiat a world of real world AI's may be much more stable and boring than you might think.


Defense is generally much harder than offense, in most systems. The attacker needs to find only one loophole. The defender needs to protect all of them, including the fallible humans in the loop. This is one reason even large corporations often fail against a small group of hackers.

And in the real world, not everyone exclusively uses services and systems from Google and Microsoft, presumably two of the most competent technology providers. Plenty of people also use ad hoc services set up by internal teams.


VISA, MasterCard, the US Federal Reserve, and IRS etc are a vast targets. However, while people regularly fake CC, Visa's internal systems seem safe. Suggesting defense is actually easier than offence when both sides are competent.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: