Norbert Wiener was ahead of his time in recognizing the potential danger of emergent intelligent machines. I believe he was even further ahead in recognizing that the first artificial intelligences had already begun to emerge. He was correct in identifying the corporations and bureaus that he called "machines of flesh and blood" as the first intelligent machines.
They self-evidently are. Profits are at some stage related to fulfilling a demand. No matter what, in the end the corporation has given a group of people what they wanted. If you think there is any scenario where that is worse than consuming all the matter in the universe to make paperclips, you must not be human.
Just to clarify, I do mean what I say. Even if the corporation produces for the most reprehensible people you can imagine, how is that worse than everything ending for no reason?
The "context" I'm referring to, which you've omitted from your quote, is that this is a discussion about the book "Superintelligence". In this context, it's entirely possible that a corporation could be autonomous.
> No matter what, in the end the corporation has given a group of people what they wanted.
For example, environmental destruction and labor abuse. There is always "a group of people" that want that kind of thing. Not a majority, but that doesn't matter.
Yes, profits are the result of fulfilled demands but maximized profits turn the whole thing into a net negative deal for all other (on a long enough time span, for all) parties involved and not all those who are not involved.
I got that part. Here's my issue tho: the paperclip maximizer turns it's programming, it's vision, into a net negative for everyone else and is thus indistinguishable from the many people who are, while potentially sharing A - or THE greater goal - are turning the achievement of their sub-goals into a net negative for everyone, including themselves.
But an 'advanced' artificial intelligence wouldn't do that anyway, because 'advanced' means that you 'understand' - are aware of - the emergence and self-organization of 'higher-dimensional' structures that are build on a foundation.
Once a child understands Legos, it starts to build more and then more out of that ...
A lot can be build out of paperclips, but an 'advanced' AI would rather quickly find the dead end and thus decide - in advance - that maximizing the production of paperclips is nonsense.
> Fortunately, none of these qualify as paperclip maximizers
I think it's weird that the maximum-paperclip hazard of super intelligence receives wide credence given that the purpose of burying the world in paperclips is so obviously stupid.
And even weirder, that the maximum-paperclip hazard can only serve as a bone of contention over what constitutes the nature of intelligence within discourse for a discipline which by definition continually begs the question of intelligence.
To rephrase this into its obvious fallacy:
A hazard of super intelligence is that it will be super stupid. And not only this, we are truly worried about the malevolence this stupidity.
Sounds like a guaranteed income for life!
And there are legion of ivory tower prognosticators who are wholly ignore this idiocy as they trouble everyone over its implication...
//The notion that artificial intelligence (AI) may lead the world into a paperclip apocalypse has received a surprising amount of attention. It motivated Stephen Hawking and Elon Musk to express concern about the existential threat of AI. It has even led to a popular iPhone game explaining the concept.//
...but there's another legion who build institutional academic careers by foisting such idiocy upon a credulous technogentsia:
//Joshua Gans is a Professor of Strategic Management and Jeffrey S. Skoll Chair of Technical Innovation and Entrepreneurship at the Rotman School of Management University Of Toronto//
But it's not fair to pick on a few grifters, because there's societal pattern of diseased thought. The AI maximum-paperclip-hazard fallacy is not an example of isolated lunacy among a few crazy outliers, it's an example of large class of contradictions that are going unchallenged to the point of risk the fate of organized human activity:
Synthetic currency that contrives value via a proof of stake that requires large, exponentially increasing commitments of energy to express;
Investor-driven, corporately mediated disruption of markets and workforces (implying an enormous range of negative determinations from such disruptions);
Mutually assured destruction, whereby enormous activity is committed creating the most dangerous known processes and substances to make ready for a purpose of cataclysm which must never be realized;
Growth economics on the face of a world already so terraformed that its thermodynamics have been disrupted to the point of threats to global ecological cycles;
A great democracy in which the federated will of the people is hamstrung by an utterly contrived and irrelevant contest for leadership between two men who represent the same policy.
The glare of these contradictions is so bright there's widespread blindness, yet we keep staring at the sun.
It's not stupid though, because there are no objective universal values that you can intelligently deduce. It's stupid to you and me because we don't want to bury the world in paperclips, we want to fill the world with art and laughter and adventure and kindness, and burying the world in paperclips is a stupid way to fail to achieve that. But if someone did want the paperclips then there's no argument you could use to change their mind, except to explain how it might deprive them of something else they want even more.
I don't really understand, "paperclips" is a stand-in for anything that would make the universe have near-zero value (when tiled/converted to this substance/pattern) when evaluated as a hypothetical in a public poll. If you can't break 50% on global control via AI, no matter how you phrase the question, what chance do you have for getting democratic support to tile the universe in microscopic patterns that vaguely resemble office supplies?
I remember reading Bostrom's work in 2014 and raving about it to others while no one really understood what I was so interested in. Well, now everyone is talking about this topic. One of my favorite analogies in the book goes something like, imagine a worm wriggling in the ground, it has no conception of the god-like beings that inhabit the world, in cities, having all sorts of goals, doing all sorts of jobs. It literally does not have the brain power to comprehend what is happening.
This seems like a poor metaphor, considering that we can understand what constituent things would make up a superintelligence, even if we don’t understand the whole.
This discussion centers too much on the definitions of words like superintelligent and reminds me a lot of philosophical discussion about omnipotence. Both seem to rely more on defining concepts first and then assuming their existence as a consequence.
Omnipotence is provably impossible: "Could Jesus microwave a burrito so hot that he himself could not eat it?" etc.
Super-intelligence, not so much — there's plenty of examples of above-average humans in many areas, and no reason to think that putting the top expert in each field into one room is impossible, and no reason to think that this configuration cannot be implemented in software with a sufficiently powerful computer.
And that's without the things that machines already beat us at, because super-human chess playing software is easily available, and computers that do arithmetic better than the entire human species even if we were all trained to the level of the current world record holder are cheap enough to be given away free glued to the front of a magazine, so there's no single person who has a particular advantage with those things.
What chess does do, is give an example: if I was playing a game with Kasparov, I would have no idea which move he might make at any given moment, but despite that I'd still expect him to win.
With an AI, I don't even necessarily know what "game" it's playing even if I'm the one who wrote the utility function it's trying to maximise.
Omnipotence just means being able to do whatever power can do. So it is incoherent to ask if God could make a square circle. The words don’t mean anything.
God could make a burrito too hot for himself to eat. Then he could make himself capable of eating it. Rinse and repeat.
Yeah I never really bought that language argument - the idea that the paradox arises solely from a schism between language and reality. I prefer to think eschatology solves the issue quite nicely. The omnipotent entity is the one for which such a situation never arises. Similarly if a universe evolves to give rise to God, who is to say that God was never present. Or all around us, really?
Either way I'd say Matthew 4:1 solves that paradox in a way that satisfies billions of people. Though I am getting a kick out of imagining Mexican/American Jesus munching on burritos.
There's two things there, and I don't get either of them.
First, if you want to sidestep if Jesus is omnipotent, then it's no longer saying anything about omnipotence.
Second, you also have to make the devil omnipotent.
Actually, the second point has a sub-point, because if it's two different entities then you get "an unstoppable force meets an immovable object" which is more easily resolved by "the force just passes right through the object without influencing it" — for the self-contradiction I gave to even be present, you would need to regard the devil as being another aspect of Yahweh in the same regard that Jesus is an incarnation of Yahweh within Christianity (not to be confused with how Jesus is seen in Islam, which totally rejects this as polytheistic idolatry).
> considering that we can understand what constituent things would make up a superintelligence
Can we? What constituent things would make up a superintelligence? Who's to say that our answer to that question is what is actually true in the case of a real superintelligence?
> Both seem to rely more on defining concepts first and then assuming their existence as a consequence.
Unlike religious philosophers like Anselm of Canterbury and Descartes and their ontological argument for the existence of a supreme being merely by imagining it, I don't believe anyone in the study of superintelligence is presupposing that they exist, or even can exist, they only presuppose how one might hypothetically exist.
Presumably the AI is in charge of things that humans used to manage and therefore understand. Worms don’t understand anything about even a slice of human society, so I don’t think it’s a great metaphor.
Why does an AI have to handle human affairs at all? It could exist outside of human goals, that would not make it not a superintelligence, just as we don't really care about worms.
I guess it doesn’t, but at this point what exactly are we speculating about? Because it seems like imaginary sci-fi, dependent on the definition of superintelligence and not on any real world developments.
It seems much more realistic to me that AI will be running systems that humans used to run, and therefore will understand at some level.
Well that is what the book is about, it is a speculative look at what hypothetical superintelligences might look like, it is explicitly not about the real world at all. Remember that the author is a philosopher, not an engineer, philosophy is all about hypotheticals.
Philosophy is not all about hypotheticals. Philosophy of technology especially is mostly about technologies that already exist and their impact on society. Not speculation.
Some philosophy (of technology, among others) relates to that, not all. It is not necessarily all about concrete impacts either, it depends on the author and their interests.
We understand some of it but who can say we understand the majority of it ? We might be at 0.1% of understanding the reality without be able to state this. Just as a worm surely « understand » some of it to differentiate and process it’s surroundings.
I always thought about it like being a permanent infant. The world is huge, full of colorful things you don't understand, and it'll be like that forever. But it's also a poor metaphor because adults have something that toddlers don't: fear. As a kid, you're an ignorant and curious blank slate; as an adult, you've established expectations and anxieties, so you'd probably be having a much worse time. :D
Imagine a clocked-down Turing computer, whose operating speed us is so slow that we can see its operations. Now imagine the programs which cannot, in principle be run on this computer. You can't because there aren't any.
Now imagine we are that computer.
Humans are capable of abstract thought and reasoning, which is sufficient to understand even the most complex software programs, to such an extent as they are understandable. The comparison to the worm is bogus.
The clocked-down Turing computer you are proposing would have to have an infinite amount of tape to be truly "Turing complete" and capable of running any program we can imagine in principle.
To actually match the worm analogy, imagine a clocked-down Turing machine with a tape length of maybe 100k cells. Now you can definitely imagine programs that can't run on it.
The worm's limitation isn't that it's slow at thinking, it basically has only a handful of neurons compared to a person and there are problems/concepts that are just too large to fit in its brain and it would never be able to imagine even if it could live and think until the end of time.
Those things aren't infinite and you don't have infinite time either. If you're trying to use something like Turing completeness to argue a point, you need to respect that it's only relevant for theoretical, formal systems. You can't pull it out as a "gotcha" when it's a question of real-world capabilities.
The basic argument here is specifically about the practical limitations. An actual worm, with a finite number of neurons and a finite lifetime, even if it was an exceptionally clever one, lived much longer than average, and spent every waking minute of its life pondering the mysteries of humans, would never be capable of comprehending more than the tiniest fraction. Even if you had worm scholars that developed some form of writing and could pass along information to subsequent generations, it's still not going to be possible because of the actual, physical limitations that have to be dealt with. A theoretical intelligence of a similar ratio to human intelligence would be similarly incomprehensible to a really clever human that spent their whole life studying the output of scholars who spent all their lives trying to understand it.
An AI isn’t infinite either. Infinities don’t enter into this. You brought it up, not me.
I’m aware of Bostrom’s argument and it is bogus. There isn’t a continuum of intelligence difference between a worm and a human, and a human and a so-called superintelligence. There is a categorical difference between the worm and the other two, which occupy the same class. Hierarchical abstract thought allows the human to reason (using cognitive artifacts like pencils or computers) about processes of any complexity, including those which vastly outstrip its own.
Boston’s a philosopher, not an engineer or computer scientist, and it shows when he makes basic errors like this.
> Infinities don't enter into this. You brought it up, not me.
No, you did when you brought in Turing machines:
> Imagine a clocked-down Turing computer, whose operating speed us is so slow that we can see its operations. Now imagine the programs which cannot, in principle be run on this computer. You can't because there aren't any.
If the Turing machine doesn't have infinite capacity, I 100% can imagine programs that "cannot, in principle be run on this computer".
Furthermore, if the Turing machine has many orders of magnitude less capacity than a computer and can only run for a tiny fraction of the number of cycles before it grinds to a halt, I can imagine many, many, many programs that won't be able to run on it.
> An AI isn't infinite either.
It doesn't have to be. Just being many orders of magnitude "larger" than us is enough that it, for all practical purposes, is incomprehensible to us enough that it might as well be an entirely separate category.
An AI would be running on a computer with the same space and time bounds as the human, because the AI is just software running on the human’s computer.
If there are no infinities, then I am not sure why you brought up an infinite machine as an argument. The point is to imagine a being with orders of magnitude more processing power than our brains, it'd be able to fathom things we literally could not, even with any amount of slowness of processing. It'd be a physical limitation.
You brought up infinite machines, hence my and the other person's comment. Turing computers or machines have infinite tape. Maybe you meant to just say "computer," not "Turing computer."
Humans can use computers but only for human readable tasks. There is no guarantee that we could comprehend a superintelligence running off a supercomputer, we already don't understand neural network internals at even their current stage.
I meant a general purpose computer I the sense of a universal Turing machine, but without the infinite storage requirement. Technically that would be a linear bounded automata, but that glosses over the the universal construction of the Turing machine.
Maybe the norms in HN are different, but in my field a Turing machine is primarily a general purpose computer, and “Turing complete” describes models which can represent any Turing machine within its constraints. An “infinite tape Turing machine” is explicitly specified when needed.
Personally I’m a bit outside of the mainstream in that I never use infinities except in the case of representing an unterminated series. I reject that “infinity” as a number even makes sense as a context.
I like to imagine bacteria as the compute substrate for an immaterial city of digital inhabitants. Fungus are even cooler, with the hypothetical wood wide web. maybe we already are the worms!
We already have "superintelligences" in the world; Nature and other humans treated as a collective is far more powerful than any individual. We manage these risks by not trusting them completely, and restricting their dominance over us. I don't see that we can't adapt to superintelligent machines as long as we don't surrender all decision making to them; the risk comes from the same old, where a group possesses overwhelming power that is then used to regiment and oppress a less powerful group. In which case, possession of AI is far from unique.
Bostrom's accomplishment is to succeed at bamboozling many who felt they were beyond being bamboozled.
Bostrom is a charlatan playing rubes with an intellectual version of three-card monty:
//This paper argues that at least one of the following propositions is true: (1) the human species is very likely to go extinct before reaching a “posthuman” stage; (2) any posthuman civilization is extremely unlikely to run a significant number of simulations of their evolutionary history (or variations thereof); (3) we are almost certainly living in a computer simulation. It follows that the belief that there is a significant chance that we will one day become posthumans who run ancestor‐simulations is false, unless we are currently living in a simulation. A number of other consequences of this result are also discussed.//
What is a post-human stage of development?
Is human extinction a post-human stage of development?
How did this argument get from post-human to ancestor simulation? It's stipulated in a lemma.
What is a simulation according to Bostrum? Undefined.
What then is an "ancestor" simulation?!
The weaseling on display adds gross insult to intellectual injury:
How unlikely is "extremely unlikely"? What is a "significant number" of ancestor" simulations? What is "evolutionary history"? How did he get from ancestor simulation to computer simulation?
Who else can stake an academic reputation on tenets delivered in the form of afterthoughts?
A theory must contain a promise of making matters simpler. If there's any kind of theory associated with Bostrom's argument, it's that many people can't distinguish a theory from an arbitrary collection of intriguing statements. Bostrom is functioning as an academic instance of Weisenbaum's Eliza: it just circularly echoes its own kooky conjectures in the form of lemmas and an outwardly spiraling discussion of itself.
Bostrom is playing a silly joke on readers. He gives his game away with his ridiculous lemmmas, showing the ace during the shuffle:
"It follows that the belief that there is a significant chance that we will one day become posthumans who run ancestor‐simulations is false"—period.
So, what was the question which was to be begged? Read it and weep. You've been played.
You've got to hand it to a magician / mentalist who gets his audience to believe they are as "worms wiggling in the ground with no conception of god-like beings that inhabit the world".
The most pitiless aspect of this fantasy is that it includes a proposition for what man is unaware of in the form of the overt statement of that which he is unaware: "god-like beings" Could the question of faith be any further begged? It's deranged.
Here we encounter Zizek's fourth quadrant of Rumsfeld's knowns: the "unknown knowns." Ideology. That which you know but avoid or refuse to become aware that you know to keep your faith alive.
There's a rule about the actual unknown, which is that it must be literally about the which is not known!
More generally, as to what is not known, we can only poke at the edge of a lacunae, maybe inferring something about the contours or gradient to mystery, maybe following the edge to refining our adaptation to ourselves. But our work doesn't belong with the unknowable. It belongs with what we can know about ourselves. The unknown will take care of itself.
In intellectual terms, I'm currently interested in the fusion of Asian and Western history, reading James Clavell's Asian Saga now, after watching Shogun recently. David Graeber's books are also on my list once I finish the Saga. I've read Bullshit Jobs and Debt by him but I've heard good things about The Dawn of Everything, particularly how European Enlightenment ideas might have actually been influenced by what they saw from Native Americans.
In terms of projects I'm working on, I'm traveling currently and it's a pain to track how much money I've spent due to needing to convert foreign currencies, so I'm building a simple app for that.
I just want to point out that the paperclip problem was already present in a short story from 1959 by a Soviet sci-fi writer Dneprov - Crabs on the Island
The author states that AI safety is very important, that many experts think it is very important and that even governments consider it to be very important, but there is no mention of why it is important or what "safe" AI even looks like.
Am I that out of the loop that what this concept entails is so obvious that it doesn't require an explanation, or am I overlooking something here?
The idea that most AIs are unsafe to non-AI interests is foundational to the field and typically called instrumental convergence [1]. You can also look up the term "paperclip maximizer" to find some concrete examples of what people fear.
It's unfortunately hard to describe what a safe AI would look like, although many have tried. Similar to mathematics, knowing what the correct equation looks like is a huge advantage in building the proof needed to arrive at it, so this has never bothered me much.
You can see echoes of instrumental convergence in your everyday life if you look hard enough. Most of us have wildly varying goals, but for most of those goals, money is a useful way to achieve them -- at least up to a point. That's convergence. An AI would probably get a lot farther by making a lot of money too, no matter what the goal is.
Where this metaphor breaks down is we human beings often arrive at a natural satiety point with chasing our goals: We can't just surf all day, we eventually want to sleep or eat or go paddle boarding instead. A surfing AI would have no such limiters, and might do such catastrophic things as use its vast wealth to redirect the world's energy supplies to create the biggest Kahuna waves possible to max out its arbitrarily assigned SurfScore.
We flatten mountains to get at the rocks under them. We fly far above the clouds to reach our holiday destinations.
We have in our pockets devices made from metal purified out of sand, lightly poisoned, covered in arcane glyphs that so small they can never be seen by our eyes and so numerous that you would die of old age before being able to count them all, which are used to signal across the world in the blink of an eye (never mind (Shakespeare's) Puck's boast of putting a girdle around the earth in 40 minutes, the one we actually build and placed across the oceans sends information around it in 400 milliseconds), used to search through libraries grander than any from the time when Zeus was worshiped, and used to invent new images and words from prompts alone.
We power our sufficiently advanced technology with condensed sunlight and wind, and with the primordial energies bound into rocks and tides; and we have put new πλανῆται (planētai, "wandering" star) in the heavens to do the job of the god Mercurius better than he ever could in any myth or legend. And those homes themselves are made from νέος λίθος ("neolithic", new rock).
We've seen the moon from the far side, both in person and by גּוֹלֶם (golem, for what else are our mechanised servants?); and likewise to the bottom of the ocean, deep enough that スサノオ (Susanoo, god of sea and storms) could not cast harm our way; we have passed the need for prayer to Τηθύς (Tethys) for fresh water as we can purify the oceans; and Ἄρης (Ares) would tremble before us as we have made individual weapons powered by the same process that gives the sun its light and warmth that can devastate areas larger than some of the entire kingdoms of old.
By the same means do our homes, our pockets, have within them small works of artifice that act as húsvættir (house spirits) that bring us light and music whenever we simply ask for them, and stop when we ask them to stop.
We've cured (some forms of) blindness, deafness, lameness; we have cured leprosy and the plague; we have utterly eliminated smallpox, the disease for which शीतला (Seetla, Hindu goddess for curing various things) is most directly linked; we can take someone's heart out and put a new one in without them dying — if Sekhmet (Egyptian goddess of medicine) or Ninkarrak (Mesopotamian, ditto) could do that, I've not heard the tales; we have scanners which look inside the body without the need to cut, and some which can even give a rough idea of what images the subjects are imagining.
"We are close to gods, and on the far side", as Banks put it.
Wonderfully written, and though I've seen this kind of reshaping of perspective on our human achievements in the modern world before, you've done it exceptionally well here.
The article itself is talking about a specific book. "Superintelligence: Paths, Dangers, Strategies" by Nick Bostrom. That book is the seminal work on the subject of AI safety. If you honestly want answers to your questions I recommend reading it. It is written in a very accessible way.
If reading a whole book is out of question then I'm sure you can find many abridged versions of it. In fact the article itself provides some pointers at the very end of it.
> Am I that out of the loop
Maybe? Kinda? That's the point of the article. There has been 10 years since the publication of the book. During that time the topic went from the weird interest of some Oxford philosopher to a mainstream topic discussed widely. 10 years is both a long time and a blink of an eye. Depending on your frame of reference. But it is never too late to get in the loop if you want to.
At the same time I don't think it is fair to expect from every article ever to rehash the basic concepts of the field they are working on.
Many have expressed my sentiments far better than I can, but Superintelligence is quite frankly written in a very tedious way. He says in around 300 pages what should have been an essay.
I also found some of his arguments laughably bad. He mentions that AI might create a world of a handful of trillionaires, but doesn’t seem to see this extreme inequality as an issue or existential threat in and of itself.
Ok? I don't see the contradiction. When I say "It is written in a very accessible way" I mean to say "you will understand it". Even if you don't have years of philosophy education. Which is sadly not a given in this day and age. "frankly written in a very tedious way" seems to be talking about how much fun you will have while reading it. That is an orthogonal concern.
> He says in around 300 pages what should have been an essay.
Looking forward to your essay.
> I also found some of his arguments laughably bad.
Didn't say that I agree with everything written in it. But if you want to understand what the heck people mean by AI safety, and why they think it is important then it has the answers.
> He mentions that AI might create a world of a handful of trillionaires, but doesn’t seem to see this extreme inequality as an issue or existential threat in and of itself.
So wait. Is your problem that the argument is bad, or that it doesn't cover everything? I'm sure your essay will do a better job.
> He mentions that AI might create a world of a handful of trillionaires, but doesn’t seem to see this extreme inequality as an issue or existential threat in and of itself.
I've not read the book, so I don't know the full scope of that statement.
In isolation, that's not a big issue and not an existential threat, as it depends on the details.
For example, a handful of trillionaires where everyone else is "merely" as rich as Elon Musk isn't a major inequality, it's one where everyone's mid-life crisis looks e.g. like whichever sci-fi spaceship or fantasy castle they remember fondly from childhood.
Haven't read the book either, but a handful of trillionaires could be that the "upper 10 000" oligarchs of the USA get to be those trillionaires, and everyone else starves to death or simply can't afford to have children and a few decades later dies from old age.
Right now, in order to grow and thrive, economies need educated people to run it, and in order to get people educated you need to give them some level of wealth to have their lower level needs met.
It's a win-win situation. Poor/starving people go to arms more quickly and destabilize economies. Educated people are the engineers, doctors and nurses. But once human labour isn't needed any more, there is no need for those people any more either.
So AI allows you to deal with poor people much better now than in the past: an AI army helps to prevent revolutions and AI engineers, doctors, mechanics, etc, eliminate the need for educated people.
There is the economic effect that consumption drives economic growth, which is a real effect that has powered the industrial revolution and given wealth to some of today's rich people. Of course, a landlord has the incentive for people to live in his house, that's what gives it value. Same goes for a farmer, he wants people to eat his food.
But there is already a certain chunk of the economy which only caters to the super rich, say the yacht construction industry. If this chunk keeps on growing while the 99% get less and less purchasing power, and the rich eventually transition their assets into that industry, they get less and less incentives to keep the bottom 99% fed/around.
I'm not saying this is going to happen, but it's entirely possible to happen. It's also possible that every individual human will be incredibly wealthy compared to today (in many ways, the millions in the middle classes in the west today live better than kings a thousand years ago).
In the end, it will depend on human decisions which kinds of post-AI societies we will be building.
Indeed, I was only giving the "it can be fine" example to illustrate an alternative to "it must be bad".
As it happens, I am rather concerned about how we get from here to there, as in the middle there's likely a point where we have some AI that's human-level at ability, which needs 1 kW to do in 1 hour what a human would do in 1 hour, and at current electricity prices that's something humans have to go down to the UN abject poverty threshold to be cost-competitive with while simultaneously being four times the current global per-capita electricity supply which would drive up prices until some balance was reached.
But that balance point is in the form of electricity being much more expensive, and a lot of people no longer being able to afford to use it at all.
It's the traditional (not current) left vs. right split — rising tides lifting all boats vs. boats being the status symbol to prove you're an elite and letting the rest drown — we may get well-off people who task their robots and AI to make more so the poor can be well-off, or we may have exactly as you describe.
Or imagine if AI provides access to extending life and youth indefinitely, but that doing so costs about 1% of the GDP of the US to do.
Combine that with a small ruling class haveing captured all political power through a fully robotic police/military force capable of suppressing any human rebellion.
I don't find it difficult to imagine a clique of 50 people or so sacrificing the welfere of the rest of the population to personally be able to live a life in ultimate luxery and AI generated bliss that lasts "forever". They will probably even find a way to frame it as the noble and moral thing to do.
What does AI, or even post-singularity robots do for the 50 richest people? They already live like it's post-singularity. They have the resources to pay people to do everything for them, and not just cooking and cleaning, but driving and organizing and managing pet projects while they pursue art careers.
> Or imagine if AI provides access to extending life and youth indefinitely, but that doing so costs about 1% of the GDP of the US to do.
That's a bad example even if you meant 1% of current USA GDP per person getting the treatment (i.e. 200 bn/person/year), because an AI capable of displacing human labour makes it very easy to supply that kind of wealth to everyone.
That level is what I suggested earlier, with the possibility of a world where everyone not in the elite is "merely" as rich as Elon Musk is today ;)
> I don't find it difficult to imagine a clique of 50 people or so sacrificing the welfere of the rest of the population to personally be able to live a life in ultimate luxery and AI generated bliss that lasts "forever". They will probably even find a way to frame it as the noble and moral thing to do.
I do find it difficult to imagine, for various reasons.
Not impossible — there's always going to be someone like Jim Jones — but difficult.
> That's a bad example even if you meant 1% of current USA GDP per person getting the treatment (i.e. 200 bn/person/year), because an AI capable of displacing human labour makes it very easy to supply that kind of wealth to everyone.
Clarification: I meant 1% per person of the GDP at the time the wealth is generated. NOT present day GDP. Medicine is one area where I think it's possible that costs per treatment may outpace the economic development generated by AI.
Any kind of consumption that the ultra rich may desire in the future that also grows faster than the economy is a candidate to have the same effect.
It's the same as for ASI X-risk: If some entity (human, posthuman, ASI or group of such) has the power AND desire to use every atom and/or joule of energy avaialble, then there may still be nothing left for everyone else.
Consider historical wonders, whether it's the Pyramids, the Palace of Versailles, Terracotta army, and so on. These tend to appear in regimes with very high levels of concentration of power. Not usually from democracies.
Edit, in case it's not obvious: Such wonders come at tremendous costs for the glory of single (or a few) individuals, paid for by the rest of society.
Often they're built during times when wealth generation is unusually high, but because of concentration of power, medium wealth can be quite low.
Once the police and military do not need a single human to operate, the basis for democracy may be completely gone.
Consider past periods of history where only a small number of soldiers could dominate much larger number of armed citizens, and you will notice that most of them were ruled by the soldier class. (knights, samurai, post Marian Reform Rome).
Democracy is really something that shows up in history whenever armed citizens form stronger armies than such elite militaries.
And a fully automated military, controlled by 0-1 humans at the top, is the ultimate concentration of power. Imagine the political leader you despise the most (current or historical) with such power.
> AI is safe if it does not cause extinction of humanity.
I don't think that is true. "AI is not safe if it cause extinction of humanity." is more likely to be true. But that is a necessary requirement but not sufficient.
Just think of a counter example: An AI system which wages war on humanity, wins and then keeps a stable breeding population of humans in abject suffering in a zoo like exhibit. This hypothetical AI did not cause extinction of humanity. Would you consider it safe? I would not.
That's called "s-risk" (suffering risk). Some people in the space do indeed take it much more seriously than "x-risk" (extinction risk).
If you are deeply morally concerned about this, and consider it likely, then you might want to consider getting to work on building an AI which merely causes extinction, ASAP, before we reinvent that one sci-fi novel.
Personally, I see no particular reason to think this is a very likely outcome. The AI probably doesn't hate us - we're just made out of joules it can use better elsewhere. x-risk seems much more justified to me as a concern.
The AI doesn't have to hate us for this outcome. In fact it might be done to cocoon and "protect" us. It just has different idea from us what needs to be protected and how. Or alternatively it can serve (perfectly or in a faulty way) the aims of its masters. A few lords reigning over suffering masses.
> If you are deeply morally concerned about this, and consider it likely, then you might want to consider getting to work on building an AI which merely causes extinction, ASAP, before we reinvent that one sci-fi novel.
What a weird response. Like one can't be concerned about two ( (or more!) things simultaneously? Talk about "Cutting off one's nose to spite one's face"
The quote I've heard is: 'The AI does not hate you, nor does it love you, but you are made of atoms which it can use for something else': https://www.amazon.de/-/en/Tom-Chivers/dp/1474608787 (another book I've not read).
> Or alternatively it can serve (perfectly or in a faulty way) the aims of its masters.
Our state of knowledge is so bad that being able to do that would be an improvement.
The argument is that "humans live, but suffer" is a smaller outcome domain and thus less likely to be hit than an outcome incompatible with human life.
Because at that point, getting something to care about humans at all, you've already succeeded with 99% of the alignment task and only failed at the last 1% of making it care in a way we'd prefer. If it were obvious that rough alignment is easy but the last few bits of precision or accuracy are hard that'd be different.
I fail to see a broad set of paths that end up with a totally unaligned AGIs and yet humans live but in a miserable state.
Of course we can always imagine some "movie plot" scenarios that happen to get some low-probability outcome by mere chance. But that's focusing one's worry on winning an anti-lottery rather than allocating resources to the more common failure modes.
> already succeeded with 99% of the alignment task and only failed at the last 1% of making it care in a way we'd prefer.
Who is we? Humanity does not think with one unified head. I'm talking about a scenario where someone makes the AI which serves their goals, but in doing so harms others.
AGI won't just happen on its own. Someone builds it. That someone has some goals in mind (they want to be rich, they want to protect themselves from their enemies, whatever). They will fiddle with it until they think the AGI shares those goals. If they think they didn't manage to do it they will strangle the AGI in its cradle and retry. This can go terribly wrong and kill us all (x-risk). Or it can succeed where the people making the AGI aligned it with their goals. The jump you are making is to assume that if the people making the AGI aligned it with their goals that AGI will also align with all of humanity's goals. I don't see why that would be the case.
You are saying that doing one is 99% of the work and the rest is 1%. Why do you think so?
> Of course we can always imagine some "movie plot" scenarios that happen to get some low-probability outcome by mere chance.
Definitions are not based on probabilities. sanxiyn wrote "AI is safe if it does not cause extinction of humanity." To show my disagreement I described a scenairo where the condition is true (that is the AI does not cause extinction of humanity), but I would not describe as "safe AI". I do not have to show that this scenario is likely to show the issue with the statement. Merely that it is possible.
> focusing one's worry on winning an anti-lottery rather than allocating resources to the more common failure modes.
You state that one is more common without arguing why. Stuff which "plainly doesn't work and harmful for everybody" is discontinued. Stuff which "kinda works and makes the owners/creators happy but has side effects on others" is the norm, not the exception.
Just think of the currently existing superinteligences: corporations. They make their owners fabulously rich and well protected, while they corrupt and endanger the society around them in various ways. Just look at all the wealth oil companies accumulated for a few while unintentionally geo-engineering the planet and systematically suppressing knowledge about climate change. That's not a movie plot. That's the reality you live in. Why do you think AGI will be different?
> You are saying that doing one is 99% of the work and the rest is 1%. Why do you think so?
(Different person)
I think it's much starker than that, more even than 99.99% to 0.01%; the reason is the curse of high dimensionality.
If you imagine a circle, there's a lot of ways to point an arrow that's more than 1.8° away from the x-axis.
If you imagine a sphere, there's even more ways to point an arrow that's more than 1.8° away from the x-axis.
It gets worse the more dimensions you have, and there's a lot more than two axies of human values; even at a very basic level I can go "oxygen, food, light, heat", and that's living at the level of a battery farmed chicken.
Right now, we don't really know how to specify goals for a super-human optimiser well enough to even be sure we'd get all four of those things.
Some future Stalin or future Jim Jones might try to make an AGI, "strangle the AGI in its cradle and retry" because they notice it's got one or more of those four wrong, and then finally release an AI that just doesn't care at all about the level of Bis(trifluoromethyl)peroxide in the air, and this future villain don't even know that this is bad for the same reason I just got that name from the Wikipedia "List of highly toxic gases" (because it is not common knowledge): https://en.wikipedia.org/wiki/List_of_highly_toxic_gases
> This can go terribly wrong and kill us all (x-risk). Or it can succeed where the people making the AGI aligned it with their goals. The jump you are making is to assume that if the people making the AGI aligned it with their goals that AGI will also align with all of humanity's goals.
Sure, but for s-risk-caused-by-human-intent scenario to become an issue the x-risk problem has to be solved or negligible.
If we had the technology to capture all of a human's values properly so that their outcomes are still be acceptable when executed and extrapolated by an AGI then applying the capture process to more than one human seems more like a political problem than one of feasibility.
> You are saying that doing one is 99% of the work and the rest is 1%. Why do you think so?
Because I'm not seeing a machine-readable representation of any human's values. Even a slice of any human's values anywhere. When we specify goals for reinforcement learning they're crude, simple proxy metrics and things go off the rails when you maximize them too hard. And by default machine minds should be assumed to be very alien minds, humans aren't occupying most of the domain space. Evolved antennas are a commonly cited toy example of things that humans wouldn't come up with.
> Definitions are not based on probabilities. sanxiyn wrote "AI is safe if it does not cause extinction of humanity."
It's a simplification crammed into a handful of words. Not sure what level of precision you were expecting? Perhaps a robust, checkable specification that will hold up to extreme scrutiny and potentially hostile interpretation? It would be great to have one of those. Perhaps we could then use it for training.
> Just think of the currently existing superinteligences: corporations.
They're superorganisms, not superintelligences. Even if we assume for the moment that the aggregate is somewhat more intelligent than an individual, I would still say that almost all of their power comes from having more resources at their disposal than individuals rather than being more intelligent.
And they're also slow, internally disorganized and their individual constituents (humans) can pursue their own agendas (a bit like cancer). They lack the unity of will and high-bandwidth communication between their parts that'd I'd expect from a real superintelligence.
And even as unaligned optimizers you still have to consider that they depend on humans not being extinct. You can't make profit without a market. That is like a superintelligence that has not yet achieved independence and therefore would not openly pursue whatever its real goals are and instead act in whatever way is necessary to not be shut down by humans. That's the self-preservation part of instrumental convergence.
> You state that one is more common without arguing why. Stuff which "plainly doesn't work and harmful for everybody" is discontinued. Stuff which "kinda works and makes the owners/creators happy but has side effects on others" is the norm, not the exception.
A superintelligence wouldn't be dumb. So game theory, deception and perhaps having a planning horizon that's longer than a rabid mountain lion's should be within its capabilities. That means "kinda works" is not the same as "selected for being compatible with human existence".
> Sure, but for s-risk-caused-by-human-intent scenario to become an issue the x-risk problem has to be solved or negligible.
Sure. I can chew gum and walk at the same time. s-risk comes after x-risk has been dealt with. Doesn't mean that we can't think of both.
> seems more like a political problem than one of feasibility
Don't know what to tell you but "political problem" is not 1% of the solution. Political problem is where things get really stuck. Even when the tech is easy the political problem is often intractable. There is no reason to think that this political problem will be 1%.
> Not sure what level of precision you were expecting?
I provided a variant of the sentence which I can agree with. I will copy it here in case you missed it: "AI is not safe if it causes extinction of humanity." (noticed and fixed a typo in it)
> They lack the unity of will and high-bandwidth communication between their parts that'd I'd expect from a real superintelligence.
Sure. If you know the meme[1] when the kids want to eat AGI, corporations is the "food we have at home". They are not kinda the real deal and they are kinda suck. They are literally made of humans and yet we are really bad at aligning them with the good of humanity. They are quite okay at making money for the owners though!
> A superintelligence wouldn't be dumb.
Yes.
> That means "kinda works" is not the same as "selected for being compatible with human existence".
During the AGI's infancy someone made it. That someone has spent a lot of resources on it, and they have some idea what they want to use it for. That initial "prompting" or "training" will have an imprint on the goals and values of the AGI. If it escapes and disassembles all of us for our constituent carbon then we run into the x-risk and we don't have to worry about s-risk anymore. What I'm saying is that if we avoid the x-risk, we are not safe yet. We have a gaping chasm of s-risk we can still fall into.
If the original makers created it to make them rich (very common wish) we can fall into some terrible future where everyone who is not recognised by the AGI as a shareholder is exploited by the AGI to the fullest extent.
If the original makers created it to win some war (another very common wish) the AGI will protect whoever they recognise as an ally, and will subjugate everyone to the fullest extent.
These are not movie scenarios, but realistic goals organisations wishing to create an AGI might have.
Have you heard the term "What doesn't kill you makes you stronger"? There is a not as often repeated variant of it: "what doesn't kill you sometimes makes you hurt so bad you wish it did".
Tbh, if you replaced the word "AI" with the word "technology" this sounds more like an overwhelming paranoia of power.
As technology progresses, there's also not much difference if the "creators" you listed pursued their goals with "dumb" technologies. People/Entities with differing interests will cross with your interests at some point and somebody will get hurt. The answer to such situations is the same as the past. You establish deterrence, you also adopt those technologies or AGI to serve your interests against their AGIs. And so balance is established.
> this sounds more like an overwhelming paranoia of power
You call it overwhelming paranoia, I call it well supported skepticism about power based on the observed history of humankind so far. The promise, and danger of AGIs is that they are intelectual force multipliers of great power. So if not properly treated they will also magnify inequalities in power.
But in general your observation that I’m not saying anything new about humans is true! This is just the age old story applied to a new technological development. That is why i find it strange how much pushback it received.
Yeah, the constant dissonance with AI safety is that every single AI safety problem is already a problem with large corporations not having incentives aligned with the good of people in general. Profit is just another paperclip.
Not only but also; they're also every problem with buggy software.
Corporations don't like to kill their own stakeholders; a misplaced minus sign, which has happened at least once*, and your AI is trying as hard as possible to do the exact opposite of one of the things you want.
Is that dissonance or shows that the concept is generally applicable? Human inventions can be misaligned with human values. The more powerful the invention, the more damage it can do if it is misaligned. The corporation is a powerful invention. Super intelligence is the most powerful invention imaginable.
This is one of the most delusional and speculative books I've ever read. The author comes up with elaborate analytical models resting on slippery, loosely-defined terms. Being smart with algebra while totally disconnected from technological grounds. It's the kind of stuff VP execs and Bill Gates like to read, and one of the reasons for the current bubble.
My annoyance is that _anyone_ would write a book on a technical subject who knows absolutely nothing about the subject. LLMs aren't a philosophical concept; they're a software mechanism with myriad constraints and design limitations built in. Understanding their future demands a deep understanding of those mechanisms. So why on earth would an academic who knows zero about engineering, software, or AI techniques have the temerity to write a book suggesting he can see farther into the evolution of LLMs than, say, a carpenter or bricklayer? At least those skills know something about physical mechanisms and engineering constraints. But not Bostrom.
The continued interest in a book of bold uninformed argumentation that's so obviously insubstantial just goes to show how bad humans are at telling the difference between useful knowledge and wild speculation. It's almost as silly as caring whether the prognostications of Rod Brooks (or worse still, Ray Kurzweil) come true. As if guessing right actually meant something...
I've re-skimmed it recently as well, and found it to be extremely zeerusted and needlessly alarmist in retrospect. A lot of it is written from the perspective of "a handful of scientists build brain in a bunker a la Manhattan project" that is so far from our actual reality that 90% of the concerns don't even apply.
Exponential runaway turned out to not be a thing at all, progress is slow (on the order of years), competitors are aplenty, alignment is easy, everything is more or less done in the open with papers being published every day. We're basically living out the absolute best possible option out of all the ones outlined in the book.
Looks like the real-world risks of AI are, predictably, AI being used to avoid responsibility/liability/regulation or plainly copyright-laundering (which likewise predictably is only a temporary loophole until laws catch up) and companies like Google reversing all progress they made in reducing their emissions by doubling down on resource-intense AI.
"Avoiding regulation" as a Service of course has a huge market potential for as long as it works, just like it did for crypto and the gig economy. But it is by definition a bubble because it will deflate as soon as the regulations are fixed. GenAI might have an eventual use but it will in all likelihood look nothing like what it is used for at the moment.
And yeah, you could complain that what I said mostly applies to GenAI and LLMs but that's where the current hype is. Nobody talks about expert systems because they've been around for decades and simply work while being very limited and "unsexy" because they don't claim to give us AGI.
The problem starts with talking about "AGI" and LLMs/GenAI in the same breath. LLMs are not and can not be AGI. They are impressive, but they are glorified autocomplete. When ChatGPT lets you "correct" it, it doesn't backtrack, it takes your response into consideration along with what it said before and generates what its model suggests could come next in the conversation. It's more similar to a Markov chain than to an expert system.
Im going to make play devils advocate here, but I think LLMs are the closest we can get to AGI, because AGI is a silly concept. Literally nobody on earth has “General Intelligence,” nobody is generally capable in all things, so why do we expect software to be?
Still, the average cutting edge LLM does a hell of a lot better at a great many things than a great many humans. I know, it’s just computer, but what is the average skill level? We just keep moving the goal posts.
You have to distinguish between intelligence and knowledge/experience.
Nobody knows everything or has experienced everything, but we do have "general intelligence", which is the ability to reason over whatever we DO know and HAVE experienced in order to make successful plans/predictions around it and actually use this knowledge rather than merely recall it.
Of course some people are more intelligent than others, but nonetheless our brain reflects the nature of our species as generalists who can apply our intelligence to a wide/unlimited number of areas.
There are at least two things fundamentally missing from LLMs that disqualify it from deserving of the AGI label.
1) LLMs have extremely limited ability to plan and reason, even over the fixed knowledge (training set) that they have. This is a limitation of the simplistic transformer neural network architecture they are based on, which is just a one-way conveyor belt of processing steps (transformer layers) from input to output. No looping/iteration, working memory, etc - they just don't have the machinery to be able to reason/plan in an open ended way.
2) LLMs can't learn. They are pre-trained and just have a fixed set of knowledge and processing templates. Perhaps we should regard them as having a limited type of intelligence ("crystalized intelligence") over what they do know, but it can't be described as general intelligence when it excludes novel reasoning/planning ("fluid intelligence"), as well as the ability to learn anything new.
We will eventually design human-like "general" intelligence (there's no magic about it that prevents us from doing it), so LLMs are not as good as it gets, but LLMs (and upcoming enhanced LLMs) may be as good as it gets for a while - AGI may well require a brand new architecture based around the ability to learn continuously. This isn't going to happen in next 5-10 years.
The author claims that we are "between third and fifth point" in the following list:
>i Safety alarmists are proved wrong
>ii Clear relationship between AI intelligence and safety/reliability
>iii Large and growing industries with vested interests in robotics and machine intelligence.
>iv A promising new technique in artificial intelligence, which is tremendously exciting to those who have participated in or followed the research.
>v The enactment of some safety rituals, whatever helps demonstrate that the participants are ethical and responsible (but nothing that significantly impedes the forward charge).
>vi A careful evaluation of seed AI in a sandbox environment, showing that it is behaving cooperatively and showing good judgment.
Have we really gone past the first point? After decades of R&D, driverless cars are still not as safe as humans in all conditions. We have yet to see the impact of generative AI on the intellectual development of software engineers, or to what extent it will exacerbate the "enshittification" of software. There's compelling evidence that nation states are trusting AI to identify "likely" terrorists who are then indiscriminately bombed.
The abridged summary here elides that 1 is a history of claims of intolerable harm being proved wrong, not that every claim has already been proved wrong. In this frame that too many people kept raising alarms equivalent to "cars with driving assistance will cause a bloodbath" which then come to pass, not that there are no further safety alarmist claims left about what could be coming next as the technology changes.
Keeping it focused on AI every release of a text, image, and voice generator has come with PR, delays, news articles, and discussion about how it's dangerous and we need to hold it back. 3 months after they release politics hasn't collapsed from a 10 fold increase in fake news, discussion boards online are still as (un)usable as they were before, art is still a thing people do, and so on. That doesn't mean there are no valid safety concerns just that the alarmist track record isn't particularly compelling to most while the value of the tools continues to grow.
I think it will always depend on who you ask, and if they're arguing in bad faith:
"Sure, the sentry bot can mistakenly shoot and kill its own owner and/or family, but only if they're carrying a stapler. Like, who even uses a stapler in this day and age?"
I’ve been working on this issue for a while and the conclusion I have come to is:
We’re not going to see actual movement on managing AI risk until there is the equivalent of a Hiroshima/three mile island/chernobyl from a self-improving system that has no human in the loop.
Not enough people actually believe ASI is possible and harmful, to create a movement that will stop the people who are pursuing it who don’t care or don’t believe its going to be harmful.
It would have been impossible to have a nuclear weapons ban prior to World War II because 1. Almost nobody knew about it 2. Nobody would have actually believed it could be that bad
The question you should ask is, if someone does make it, on any timeline is there any possible counter at that point?
It seems like your nuclear analogy might have an answer to the question. Take that theory that it could have been possible the nuclear chain reaction ignited the atmosphere.
It seems like the worst-case predictions about AI are at that scale. There is also the possibility that AI causes problems on a much smaller scale. A country’s weapons system goes terribly wrong. A single company’s business goes berserk. Stock trading bot at a large enough bank/fund causes a recession. Stuff like that seems rather likely to happen as systems are adopted before they’re “ready” and before they’re anything close to AGI.
The book “weapons of math destruction” is full of examples of AI fucking up badly and ruining people’s lives today. It’s just that the damage is more diffuse than Hiroshima and nowhere NEAR as gruesome and specially located and sudden.
Perfect is the enemy of good, so why vote for a lesser good?
Humans are so existentially biased and self-centred!
And they are always forgetting that they wouldn't even be there if others hadn't made room for them.From the Great Oxygen to the K–Pg extinction event.
Be generous!
"Man is something that shall be overcome. What have you done to overcome him?"
It's going to happen biologically before it happens in silicon, anyway. And the biological venue could very well be humans (genetically modified). So I quite literally agree. :)
There is the exemplary manga Blame!, in which the intelligent machines have run amok and turned the whole Solar System into one dystopian megastructure, and the safeguard protocols, like an immune system gone awry, have wiped out the majority of humans. The hero is an agent tasked the reviving humanity, which requires him to embark on a centuries-long journey through this megastructure, which is something like Journey to the West meets the Odyssey.
The main danger is from people losing their jobs and the societal upheaval that could arise from that. AI isn't going to take over the world by itself and turn humanity into slaves, destroy us (unless we put it in charge of weapons and it fucks up) or harvest our blood for the iron.
I love how people think because we are getting very good at efficiently encoding human intelligence that implies that we are very close to creating superintelligence, and that our progress on creating superintelligence will somehow resemble the rate of progress on the simpler problem of encoding existing intelligence.
If we can create a human-level intelligence in the computer - it would already be superintelligence. No human on Earth is capable of reading and remembering Internet scale corpus of data, or doing math at GHz speeds, etc.
When it comes to speed, the comparison I like to use is that transistors are faster than synapses by the ratio to which a marathon runner is faster than continental drift.
If we can match our existing intelligence (but it’s a jagged border of capabilities), our progress in creating superintelligence won’t matter because we won’t be the ones making it.
AI safety is fear mongering to shut up the Luddites
The AI we have now (Stable Diffusion, chatgpt) are technical advancements that allow inferior but cheaper production of artistic content. It is not a step closer to death-by-paperclips; it is merely another step of big capital automating production, hoarding more wealth in a smaller group.
The closer thing to AI safety is unsupervised execution of laws by ML.
Stable Diffusion, where the best models are freely downloadable and the company behind them is losing money and currently doesn't look like it has much of a future, is your example of "big capital automating production, hoarding more wealth"?
At least with OpenAI/GPT, the models are actually kept behind a door, though there are still downloadable competitors that keep insisting they're not terrible :P
I'd say stability AI directly are not doing that very well, but the vast majority of users of image generation AI are using NovelAI or Midjourney, both of which are raking in cash and don't share their models (NovelAI seems to be at least mostly finetunes of SD, while there's not much public details on midjourney's models at all). Both cases don't have much 'big capital' (especially compared to OpenAI) though: they're just very profitable from getting on the train of making it easy to use early.
> Stable Diffusion, where the best models are freely downloadable
Stable Diffusion 1.6, Stable Diffusion 3 Large & Large Turbo, the model(s) — originally an SDXL Turbo-based finetune, but explicitly stated to be planned to evolve over time and potentially eventually use multiple models — used for the Stable Image Core service are all not publicly released under any license, and SD3 Medium under only a restrictive license.
In the places I see people working with them, 3 is considered a disappointment (though noisy people are often those with the negative opinions, so take with a pinch of salt), while most of the interesting stuff is either SDXL or what third parties have done by fine-tuning on 1.5 or 2.
This thread is a testament to Dunning-Kruger effect.
Every observation listed here can be generalized to a case of worry that AI is stupid, selfish and/or mean. Unsurprisingly, just like what's feared about people!
Epic confusion of the topic of cybernetics with "AI".
In colloquial parlance "AI" must be quoted to remind that the topic is hopelessly ambiguous, where every use demands explicit clarification via references to avoid abject confusion. This comment is about confusion, not about "AI".
"Thou shall not make machines in the likeness of the human mind."
Too late.
Weinzenbuam's Eliza showed the bar to likeness is low.
Television similarly demonstrates a low bar, but interestingly doesn't arouse viewers' suspicions about what all those little people are doing inside when you change the channel.
I find it helpful when considering implications of "AI" to interject the observation of the distinction between life and computing machinery: these are profoundly different dynamics: life is endogenous, mechanical computers are exogenous. We don't know how or why life emerges, but we do know how and why computers occur, because we make them. That computers are an emergent aspect of life may be part of the conundrum of the former and therefore a mystery, we design an control computers, to the extent that it can be said we design or control anything. So if you chose to diminish or contest the importance of design in outcomes of the application of computers, you challenge the importance of volition in all affairs. This might be fair, but apropos Descartes' testament to mind: if you debase yourself, you debase all your conclusions, so best to treat confusion and fear about the implications of applied computing as a study of your own limits.
There's a single enormous and obvious hazard of "AI" in this era: that we imbue imitations of human responses with humanity. The bar is low for a convincing imitation and transformer technology demonstrates surprisingly high levels of imitation. This is conducive to confusion, which is becoming rampant.
The start of a responsible orientation to rampant confusion is to formally contextualize it and impose a schedule of hygiene on fakery.
The great hazard of centralized fakery (e.g. radio and television) is a trap for the mind.
We are living in the aftermath of a 500 year campaign of commercial slavery. When people are confused, they can be ensnared, trapped and enslaved. The hazard of "AI" is not the will of sentient machines burying the world in manufactured effluvia— we've achieved this already!— it's the continuation of commercial slavery by turning people into robots.
This thread reads like it's being created by bots; cascading hallucinations.
Well, this bot is throwing down the gauntlet: Prove you're human, y'all!
When AI surpasses the high bar built up in other mediums (a.k.a. heavily curated environments for procrastination) then the problem of where to pigeon hole AI goes away, rather than a hostile/obvious illusion it will just be another welcome illusion.
There will always be humans trying to push humanity further or hold it back, we should expect bots in that mix from now on. Regulating bots out of existence only offers a false sense of handicap so that human contributions continue to feel the most meaningful for a time.
For anyone else who had the same initial thought, this isn't talking about bounties as in hits; it's basically talking about giving rewards for exposing violations of laws, funded by non-government parties so those parties can have higher confidence that violations aren't being ignored.
In other words, a privately-funded Crime Stoppers for the crimes of AI researchers.
https://en.wikipedia.org/wiki/Possible_Minds