Whether or not the existential risk around AI is real, governments, corporations, NGO's and others will twist this to their own purpose to build censorship, entry barriers, and other unpleasantries. We should be mindful of that.
I am currently (as of about a week ago) permanently banned on ALL my accounts on Reddit (despite many of them using different email addresses for anonymity purposes, etc.) because of a single admin making (quite arguably) a single error in permanently banning 1 of my logins from 1 subreddit (which was already overly punitive). Apologetic messages to mods went unheard and unheeded. Months later, I accidentally posted to that subreddit from a different account (and it was a supportive message!!) and somehow (without me intending to be a bad actor) this got escalated to identifying me across ALL my accounts (via machine learning on their end, possibly via Google cookies and similar source IP I'm guessing?) and this eventually cascaded into a total and permanent ban across EVERYTHING.
Meanwhile, I'm possibly one of the most Googleable people around (not due to fame, but due to nearly global name uniqueness) and it should be pretty plain to see that I'm a good actor (and kind of a big nerd) and that this might have all been a misunderstanding.
I'm kind of over it, but it is still hugely annoying. And it's terrifying to consider that no matter how anonymous you are on Reddit (and you might have very good reasons to be using an anonymous login), they in theory know exactly who you are.
What I'm saying is that AI could magnify a single human error in the past into an arguably massively unethical action; it could retroactively tie your anonymous actions in the past to your non-anonymous identity in the present. Where there is no escaping your past mistakes and you will always be punished for them, forever, by massive machines without any human intervention, or by systems in power that are arguably unethical and dystopian.
Sounds like you just got burned by using the same IP twice, but your point does stand. There's some interesting fingerprinting software out there, and I've personally seen compute on things like typing cadence and other more clandestine metrics too. These can of course be amplified greatly in effectiveness by AI.
Security is one area in which you can't really afford not to go down the tin foil hat rabbit hole. It's been like this for some time, where it's not really possible to know just how much capability is possessed by nation state actors, but with AI and other tools those capabilities will expand and become the baseline probably way sooner than we predict. No longer will experts be needed to do careful tracing to deanonymize, one of the true superpowers of AI will be its ability to extract the full power out of the big data we've been collecting. So the cost then of figuring something out that now might cost millions will soon cost a couple bucks.
Are you blaming "AI" for identifying you? If that's the case, first, blame reddit, second, whatever "AI" may have helped them has no relation to what's being discussed here. Identifying you would be some kind of big data pattern recognition thing. Neural networks and gradient descent might be able to solve the problem if set up appropriately, but so could lots of other things and fundamentally the issue is data collection, not anything about the state of machine learning or the liberties taken with it.
Yes, but this unfairly penalizes people simply using different accounts NOT intending to evade any ban (I'm talking very established accounts that are years old... My oldest Reddit login dates to the first few months of Reddit's existence! And it's now also banned.) but simply to maintain anonymity.
Kill me for having a "porn" Reddit account and a "professional" Reddit account, I guess, plus one to disclose health issues I'd prefer to keep private... And deserve to. I actually enjoy anonymously helping people get through issues that for very good reasons they are also posting anonymously about (think: victims). This mentality ruins that, and I am now prevented from doing that.
You can have multiple accounts on one IP. Plenty of people have throwaway accounts. Entire nations would be blocked due to CGNAT if it worked the way you suggest. They're looking at more than your IP address though (my guess is ip+useragent at the very least; the API rate-limited by useragent).
Your problem is that one of the heads of your hydra caught a ban, after which point everything you do is assumed to be malicious. Given the political nature of what you were banned for, there is no doubt in my mind this is intentional. It's not an overt act of discrimination if they just don't answer the phone after "accidentally" locking you out.
For your own sake, move on. Reddit isn't worth fighting for. It's just a bunch of schoolyard bullies squatting on the playground equipment, throwing rocks at everyone and crying victim. If you're not a member of a protected class (or pretending to be), they don't want you there.
Whether or not the existential risk around [climate change or nuclear weapons] is real, governments, corporations, NGO's and others will twist this to their own purpose to build censorship, entry barriers, and other unpleasantries. We should be mindful of that.
Nuclear weapons have a very tangible negative outcome which everyone can imagine. Existential AI risk has no meaning right now outside of a blurry vision of something like Skynet. The issues are not alike.
That - that somebody would be at the same time an HN admin and an OpenAI founder - is a statement (a remarkably extraordinary statement) that requires some links to show it.
Is it too cynical to claim the the noise about existential risks is a convenient distraction from these unquestionably real issues we will face, maybe even a deliberate part of helping it happen?
Here's a random bit of fun prescience:
From Dune, by Frank Herbert:
> JIHAD, BUTLERIAN: (see also Great Revolt) — the crusade against computers, thinking machines, and conscious robots begun in 201 B.G. and concluded in 108 B.G. Its chief commandment remains in the O.C. Bible as “Thou shalt not make a machine in the likeness of a human mind.”
The usual interpretation is something along the lines that AIs took over the thinking for the leaders, making the leaders lazy, impotent, and society fragile and chaotic. This fits with a lot of Herbert's themes.
But, from Children of Dune:
> Three — Planetary feudalism remained in constant danger from a large technical class, but the effects of the Butlerian Jihad continued as a damper on technological excesses. Ixians, Tleilaxu, and a few scattered outer planets were the only possible threat in this regard, and they were planet-vulnerable to the combined wrath of the rest of the Imperium. The Butlerian Jihad would not be undone. Mechanized warfare required a large technical class. The Atreides Imperium had channeled this force into other pursuits. No large technical class existed unwatched. And the Empire remained safely feudalist, naturally, since that was the best social form for spreading over widely dispersed wild frontiers — new planets.
Arguable paints a different picture - that the AI in the hands of the masses would be too threatening to the status quo.
> a convenient distraction from these unquestionably real issues we will face, maybe even a deliberate part of helping it happen?
Oh certainly. Not for any higher purpose, but remember that OpenAI lost its shit and threatened to leave the moment the EU wanted to look into actually regulating the risks of their product. The driver is money, and little else.
AI people like Sam Altman want a very specific kind of "AI regulation" - a mostly toothless one meant to address the hypothetical of a Skynet (which is honestly still pretty far out there) rather than the more realistic problems that come from their product.
Things like IP law (these LLMs rely on scraping lots of copyrighted data), profiling people (in an age where privacy is important to a lot of people), medical malpractice (I remember someone saying they fed a patient dossier into ChatGPT on this site... Dear Lord I hope that person got shafted by HIPAA) are all genuine problems/risks with LLMs that don't come from the models themselves but from the people using them. Thats the real danger and OpenAI wants absolutely no regulation in that area because it could hurt their bottom line.
Hence why they also almost all tend to follow the beliefs of Elezier Yudkowsky, whose entire life is basically to claim Rokos Basilisk is totally happening and the only option is to give him (and his friends) lots of money to ensure it so you won't be tortured by the AI for not bringing it about.
Crowing about evil AIs is a useful distraction because it plays well populistically (Skynet, Matrix, HAL 9000 are all good examples of "AI out of control" in the public consciousness) and it gives them easy backing from people who don't understand how these things work. The EU wasn't impressed and the AI act they passed addresses some actually meaningful concerns. It remains to see if US Congress/Senate will address it at all or if their business interests reign supreme.
The things you do name are unquestionably going to be massive issues, and yet these don't get nearly the airtime of the incredibly speculative 'AI will kill all humans if we don't do something drastic'.
"Existential risk" truly is a misnomer at this time. Please use the better fitting: "existential fear". AI "risk" could instead be applied to global information infrastructure, a collapse of the internet by rogue AI bots etc. There are too many technological challenges for any AI to overcome in robotics, mobile energy, material and bio printing/fabrication, and supply chain dominance for an imminent existential risk to humanity to be a credible near-term threat. LLM performance is a gift of data availability of the internet. There aren't equivalent datasets available for the challenges articulated above. How likely can they materialize, particularly by machines that do not yet have a tactile interface and understanding of the physical universe? Perhaps a malevolent machine sentience is possible at some time in the future-- but there are many undeveloped technologies required to pose a threat to human existence.
i think you might be focusing too much on ai risk presented in fantasy (killer robots), meanwhile we can already clearly see how LLMs negatively impact society via disruption or popular opinion, politics (recommendation algorithms), and rapid uncontrolled scientific discovery. Such disruptions could potentially result in nuclear war, human created plagues, etc. You might be getting down voted without comment because you're message comes across like cheerleading that only examines one future distant risk. Your framing as "extential fear" is particularly dismissive and doesn't seem to be in good faith for such a serious and subject.
It's hard to take doomers seriously when they think that things that exist and are clearly not causing nuclear war or plagues are only a few steps from doing so. If you want to talk seriously, talk about actual problems AI is causing right now, like job loss or malpractice.
Netflix documentaries aren't always the most 'academic', or 'robust'.
But the one that was just released about AI for autonomous drones, is pretty shocking.
The military side is farther along than I thought, AND, you know what is appearing in a Netflix documentary is old, it isn't the latest things that are still secret.
In any case, why it relates to this article.
We are in a race with other nation states, so all of this discussion about 'curtailing' or limiting AI, is all just grand standing, smoke screen.
Nothing will slow down development because other countries are also developing AI, and everyone firmly believes they have to be first or will be wiped out. Hence no one will slow down.
Or, we try to set common rules before the thing goes out of hand. We already have UN rules about fire weapons, and your argument applies to those as well. But just as we have widely agreed upon rules for weapons, if there's the chance of AI being used used for war (which indeed seems to be the case, see e.g. UAVs) then we should regulate it as well. This leaves out civil uses of AI, but as we have safety regulations for every other technology, I don't see why AI should be an exception. Then we can discuss which regulations make sense and which not, but this is a different topic.
PS: I don't thik AI will pose an existential risk to our species anytime soon, if ever. But it can already cause damages if used improperly, like any other technology. So it has to be regulated, like any other technology.
Nuclear weapons are a risk to everyone, so the world(US, Russia) did come together in various treaties to limit production, development, etc... So, this could apply to AI also, everyone sees the risk and comes together to curb their us and development.
On other hand:
Nukes are very difficult to make.
Iran and North Korea do not abide by the UN resolutions, they are continuing trying to develop them. We are just lucky that is not so easy.
With AI, the barrier to entry is much lower. You can find things that can be weaponized for free on github. It isn't like they are going to bother reading the licenses and not use them.
And for compute power, sure, they have harder time getting the best latest CPU's, and there is an embargo. But still, I think, far easier to get illegal gotten CPU's, than developing Nukes, the barrier is still lower.
The author seems to be entirely unaware of the current crop of open source / public language models like LLaMA and Falcon. People are already engaging in the behavior that, according to the article,
might present an existential threat to humanity.
The existential threat argument is a ridiculous notion to begin with. Pretending open source LLMs don’t already exist makes the argument even more silly.
They're presented as a hypothetical scary technology, not something that actually exists today. That framing is important to the conversation. The author is acting like there's a decision to be made. Pretending that open source models don't exist yet makes the decision artificially more difficult.
Why not examine the usage of existing models and judge them on their merits?
Can you say more about why you think it is ridiculous?
AI is already increasingly helpful to AI researchers. So one convincing argument for existential risk is that if this increase continues we could see exponential growth in the efficacy of AI research. Given that the goal of AI research is useful intelligence, it seems reasonable to say that an exponential increase in AI research will lead to an at least linear increase in useful intelligence.
Another argument is that there are breakthrough discoveries that will dramatically increase useful intelligence, or dramatically increase the rate of increase. We've had some of these so far, so it seems likely there are others awaiting us. And it's impossible to say with any certainty that there's no chain of such discoveries that doesn't lead to a human or superhuman level of intelligence. All we can do is try to estimate the odds of it happening.
Both arguments seem a pretty clear argument for an existential threat, because as soon as you have even just a human level intelligence integrated that seamlessly with the power of conventional computing, you have something that is much more powerful than most (if not all) human organizations, and once that power exists, there's a non zero risk that it will aim at some objective that is harmful to us, in a comprehensive enough way that it also aims to stop us from being able to stop it.
We don’t have a solid understanding of what it means to be intelligent, and despite the advances with LLMs, have yet to produce anything close to general cognition.
I wouldn’t say dangerous AI is out of the realm of possibilities, only that we are far from a point technologically where it makes sense to have the discussion. It would be like speculating about the dangers of nuclear energy before discovering the atom.
Our current models do not pose an existential threat to humanity. Period. And yet here we are, discussing the merits of banning open source LLMs. Language modeling is genuinely useful. I’m worried that the AI hysteria will push us toward a future where the best models are needlessly regulated and controlled by corporations.
>Both arguments seem a pretty clear argument for an existential threat, because as soon as you have even just a human level intelligence integrated that seamlessly with the power of conventional computing, you have something that is much more powerful than most (if not all) human organizations,
Incorrect. Power comes with the ability to command resources, not intelligence. The United States military is the most powerful human organization for its ability to marshal the most amount of deadly weapons. Some rinky dink AI company cannot do this.
I think it's safe to say that, if a large entity with monopoly of force wants to stop these models from being used, they probably could. We already have a global surveillance machine watching us all, and normalized content takedowns for lesser reasons like "copyright infringement" and "obscene/exploitative material". Actual manufacturing & distribution of compute power is fairly consolidated, and hacker culture seems to have lost its activist edge in demanding legal rights around general-purpose computing. The future seems bleak if your threat model includes state actors enforcing AI regulation due to state-level safety concerns (terrorism, wmd production, large-scale foreign propaganda, etc).
He seems to be well aware of the idea that diverse models will exist.
But what happens if regulation makes the use of these models more legally dangerous than a few "well regulated" models approved by some sanctioning entity?
> Imagine companies thinking they can scrape all the world's information for free and then package it up and sell it.
That's one of the main business models of the Internet, only usually it's not even scraped. People upload it willingly in exchange for free hosting and free social media.
The other big business model of the Internet is mass surveillance driven targeted advertising.
It's hard to guess who you're talking about but that is not what Google does. They scrape public web servers, respecting robots.txt directives and then they provide links back to the original site.
And if someone hosts user-submitted copyrighted information without a license, the copyright holder can submit a complaint via DCMA and the hoster has safe harbor from liability.
OpenAI is literally scraping copyrighted works, packing it up and selling it without a license. Art, books, magazines, everything. No safe harbor from DMCA for doing that.
I’m willing to admit that training a model on a book is not exactly the same as a person reading a book and remembering what they learned. Are you willing to admit that training a model on the book is not the same thing as copying that book?
A big problem here is that copyright law was a massively problematic thing even before transformer model tech was developed. It has been distorted beyond recognition by a few companies, who have state everything on a broad and permanent copyright rather than the limited one we started out with.
We probably need new definitions in the law, because pretending that training a model on some thing equals copying it isn’t based in reality. It’s an emotional appeal meant to gin up outrage.
Are you willing to admit that training a model on the book is not the same thing as copying that book?
This type of interaction is not helpful. It's an argumentative strawman argument. Who said training a model is the same thing as copying a book? Of course it isn't. But who said it had to be?
Here's an idea... read the first five books in the series "A Song of Ice and Fire". Then sit down and write your own version of Book 6 to continue the story and sell it without a license. Guess what's going to happen? You will be sued into bankruptcy.
What OpenAI is doing is lot more similar to that than literally copying things. And it's still wrong and illegal.
It has been distorted beyond recognition by a few companies, who have state everything on a broad and permanent copyright
I agree with you, what Disney and others have done with copyright extensions is immoral and should be illegal. But it's not illegal.
pretending that training a model on some thing equals copying it isn’t based in reality
No, it isn't based in reality. Which is why nobody made the claim. They're packaging up a derivative work and selling it. Don't have to look hard to see examples that this is just as infringing as outright copying.
> Which is why nobody made the claim. They're packaging up a derivative work and selling it.
Pardon me. I did conflate the overall claim that it inherently violates copyright law with a more specific claim (not made) that it "is copying." Since copyright also enumerates the "making derivatives" rights as well as the "copy rights" I acknowledge you have in your argument more than the zero legs to stand on that i implied.
> They're packaging up a derivative work and selling it. Don't have to look hard to see examples that this is just as infringing as outright copying.
This is an interesting claim. It rests on the question of whether the model itself is a derivative work, or if it's a tool (or something between a tool and a trained person).
A photocopier can be used to reproduce ASOIAF and a word processor can be used to create a blatantly derivative work, but I assume we agree that that isn't the problem of Xerox or Microsoft. The derivative works produced with those technologies are the 'illegal' items, not the programs that were used to build them.
If I wrote my own GoT fanfiction, ripping off whole characters, names, and settings, and read my own stories in the privacy of my own home, am I breaking any copyright law? I don't think I would be. I would rightly get in hot water if I tried to sell them, and would probably rightly get in hot water even if I just posted them to Github for free given that I'm distributing the derivative works.
I think using AI tools to generate derivative works could place the user (Not OpenAI, etc) in rightful legal jeopardy if they distribute or sell those works -- on the other hand, if they are simply keeping them for their own personal enjoyment I think it's not that different than if they wrote them themselves. (I also think that rightsholders are acting a little paranoid with those concerns, as though anyone would seriously choose not to buy the latest book or movie or painting only because some poor AI knock-offs exist, but I acknowledge that has little bearing on whether some action is or is not legal.)
I think using AI tools to generate derivative works could place the user (Not OpenAI, etc) in rightful legal jeopardy if they distribute or sell those works
I'm not sure I understand this. It is OpenAI that is scraping the copyrighted works, packaging them up into a derivative work and selling access to it.
If a user never enters a prompt asking ChatGPT to create a new ASOIAF book, pieces of those previous copyrighted books are still in OpenAI's model and available for sale by OpenAI.
Chat-GPT the LLM itself is the derivative work that OpenAI is selling access to.
We're trending deeply into RMS's the right to read here.
I mean, lets say I am a storywriter and I have an exceptional memory when reading books, and you buy access to talk to me to get story ideas (as a human being, no API). Lets say I also read ASOIAF. Are you telling me that anything I write that mentions winter is now intellectual property of GRRM?
In my eyes your idea of derivative work can fuck right off. Pieces of those of that copyright are also in my mind, but I give no ownership, nor any privilege's to said book writers. IP holders do not get all the benefit of free data in society, then hold the rest of us hostage.
Ok. I guess we just disagree then. In my view, that model doesn’t “contain” the works. It contains lists of numbers (and not in the ASCII sense that a silly rebuttal would make, I mean only the tokens and weights) and not “pieces of” the books. If I published a statistical analysis of word frequency in your books, I don’t think you’d have a slam dunk CI case against me. Even if someone could use those to generate some passages of your book. It certainly can’t generate the whole book, we can plainly see that (otherwise OpenAI has actually invented magic compression). Just as if you sold consulting services, and employed people who had read those books many times and sold your service to budding fantasy authors to help them write better, those consultants are not themselves derivative works just because they learned the material. The derivative work would be those people’s output (if it rips off that material).
Or, instead of that, everyone gets away with it, under broad "transformative/fair use" protections and nobody gets sued.
That's what's happening now.
But sure, people will always make these unfalsifiable arguments about some hypothetical doomsday that is far enough in the future that they can never get called out when it doesn't arrive.
IMO this paragraph in the summary is the most important one:
> There are interventions we can make now, including the regulation of “high-risk applications” proposed in the EU AI Act. By regulating applications we focus on real harms and can make those most responsible directly liable. Another useful approach in the AI Act is to regulate disclosure, to ensure that those using models have the information they need to use them appropriately.
We wish. AI is hardware-limited and hardware is not moving fast. We are very, very far from matching raw compute power of the human brain. Robots are even more limited compared to human body.
"We are very, very far from matching raw compute power of the human brain." This has been said for decades with viral conviction. While not equivalent -- we don't understand how these numbers relate -- the human brain averages having approximately 86 billion neurons, and GPT-4 has an estimated 1.8 trillion parameters. At some point we might have to re-evaluate the value of this dogmatic comparison between the human brain and existing AI systems.
There are some 100T to 1P synapses in the 20W human brain (of which we have 8bn). Every synapse is a little molecular computer with several kinds of memory and its own inference and learning logic.
AFAIK the biggest limitation of von Neumann computers is memory bandwidth. Brain is moving 10+ PB/s between neurons. This does not count synapse-local and intra-neuron bandwidth.
> the human brain averages having approximately 86 billion neurons, and GPT-4 has an estimated 1.8 trillion parameters.
But that's not an apples-to-apples comparison, as the artificial neurons of neural networks are just a rough approximation of our neurons. They are also connected in a much simpler way too (nicely divided into layers, where each layer can only pass signals to the very next layer), so it could be that it isn't just a matter of how many parameters you have.
And, it isn't just number of neurons, it is connections between neurons. I think brain still outnumbers on a count of connections (sorry, don't have source handy).
Yes, you said that those aren't directly comparable, but your comment seems to imply that you can account for the difference by having more artificial neurons than the number of biological neurons that we have. Sorry if this is not the case.
"Historically, large power differentials have led to violence and subservience of whole societies." I agree, the calls for regulation by large corporate entities who quite transparently wish to control AI development for themselves, is a much greater near-term "existential threat" than AI itself.
I don't get this scare about AI. Imagine, you are a genius with IQ 200. You have studied hard your entire life and you are now an expert in several fields. One day you decide to conquer the world. Will you succeed or not?
On your specific point, humans take decades to (very roughly) copy themselves. An AI could do it in minutes with far more precision and control.
Assuming you have 200 IQ, if you could make thousands or millions of clones of yourself quite cheaply, you still might not succeed in taking over the world, but it is no longer a laughable idea.
Overall, I’m somewhat ambivalent about the possibility of x-risk, especially if we work reasonably hard to prevent it.
But it shouldn’t be ignored. AI is moving very quickly and it is unclear how powerful its capabilities will be in the next 10-20 years. Of course, there are many other risks presented by AI that we need to stay on top of as well.
> On your specific point, humans take decades to (very roughly) copy themselves. An AI could do it in minutes with far more precision and control.
Copying AIs just results in effectively higher intelligence (bigger brain). Things would be different with self-replicating robots of course, but duplicating robots is not that fast and one could argue that the robots are dangerous rather than the AI controlling them, because they would be dangerous even if controlled by stupid non-AI program.
If you are very smart, chances are you will be able to obtain some power and influence. This is expected of AIs. Such effects can be however undone the same way you can undo effects of an explosion at a refinery (while punishing those responsible). The question is whether intelligence alone lets you expand power without bounds until you conquer the world. I just don't see how that would be possible.
>The question is whether intelligence alone lets you expand power without bounds until you conquer the world. I just don't see how that would be possible.
Name the next most intelligent animal to humans...
Have humans completely and utterly conquered them?
The intelligence explosion already happened and humans were it. Now you're questioning if it's possible with intelligence explosion 2.0?
The intelligence explosion was useful to humans because humans can make more humans and command more resources. An AI can clone itself only on resources that humans make.
We’re F’ed because people can’t imagine more intelligent than 200 iq…
How about 500 iq with a direct line into the entire information retrieval and processing system of 7 billion humans, most of whom are already easily manipulated by relatively rudimentary social media ML.
Imagine you are a jungle monkey. One day some other small hairless monkies want you to go extinct. Will they succeed.
It's not about 200 IQ, it's about when we reach bigger gaps than that. Though yes, I suspect if the 200 IQ guy can copy himself at will, get hardware to do years of 200IQ thinking in an hour, etc. he might succeed.
> One day some other small hairless monkies want you to go extinct. Will they succeed.
The hairless monkeys were embodied and violent and they still took millions of years to dominate. The question is whether intelligence on its own provides sufficient advantage to be immediately dangerous. How can you conquer the world by doing "years of 200IQ thinking in an hour"?
Dude, humans started from scratch, we're trying to design these things to outperform ourselves in every way possible. They're not starting from scratch, robots come out of the factory with the ability to walk, humans never had that advantage, we evolved to do that over a long time.
I'm pretty neutral about it at the moment, but it's not inconceivable that something with an IQ of 400 and can think 30x faster than you, has no need to sleep and has 500x the working memory of you would be dangerous if it didn't like you.
In fact, your comment and others like it make me wonder if people are just in denial because as I said, it doesn't seem impossible?
I've literally heard people claim that things like ChatGPT will soon become "smart enough" to iteratively improve themselves so rapidly that they will become "digital gods".
Right now, LLMs are already rapidly improving themselves through LLM generated code proxied by human engineers. Not a huge stretch to imagine needing less and less engineers to further improve.
I'm not so sure about that, actual Gods wouldn't have poisoned the water, broken the oceans, started wars with each other and the climate and make itself obsolete and potentially obliterate themselves and their children.
Only if the body is duplicated too. If I could just make my brain bigger, I don't think my odds would improve much. Quite to the contrary. An AI with higher dependency on compute power seems more vulnerable.
It is wild how seriously people are taking the development of these LLMs. There is talk of them leading to the extinction of life while neglecting to consider that we can't even build self-driving cars, let alone have a robot clear the table after dinner.
Also, many people seem to be mistaking "artificial intelligence" for "actual intelligence".
You know that these LLMs are not actually intelligent, right? Right???
A pretty influential and very well-financed subculture of AI philosophers are currently performing a comprehensive lobbying campaign for AI regulation. They generally go by Rationalists or Effective Altruism, and have a philosophy stemming from MIRI, Eliezer Yudkowsky, Nate Soares and others.
They've had considerable success getting traction for these views with politicians in Europe, and that's the reason you see so much of this in the media currently. I'm hoping more sane voices will soon get organized, because significant parts of this are reminiscent of a well-funded doomsday cult.
I think this article is an early attempt at creating a politically viable counterpoint.
When AIs kill people, it is not because they are intelligent, but because they are given lethal powers without appropriate judgement. The first autonomous weapons platform with a kill count was an accident. https://www.wired.com/2007/10/robot-cannon-ki/ Self-driving cars kill people all the time. These things happen because the AIs are not super-intelligent, but people keep giving them too much power.
Maybe L2 systems. But those are just a driver assistance, the driver must supervise and intervene.
Actual L4/L5 systems haven't killed anyone as far as I know, except for the Uber accident, and even then the driver was supposed to be supervising it, I'm not sure if it counts as L4/L5
I'm completely with you on this one. Sci-fi and AI hype are extremely strong on these threads. I bet we're not any closer to human level intelligence with ML than we were in 1500s with mechanical clocks. Both are impressive nonetheless.
This reminds me of when I attended the first official conference for Facebook's popular React.js library. (2014 I believe?)
Got to hang out with some "hotshot" Facebook developers after hours. I remember Lee Byron (of React, GraphQL, etc) stating very confidently that the roads would be teeming with self driving cars by 2020.
> You know that these LLMs are not actually intelligent, right? Right???
Can a submarine swim?
Does it matter if an LLM — which is far from the only kind of AI, and trivial to include in another more complex AI — does or doesn't match your definition of "intelligent" when it can write fluently in any language (human and code), and also getting near top-of-class results in the Bar exam and the biology olympiad?
Corporations — sometimes used as another example of an unaligned optimising agent — can't drive cars, they have to hire humans to do that for them, but corporations can still be prosecuted for dangerous chemicals (3M); and AI, just like other computer programs, can have dangerous output that you should not rely on (e.g. Thule early warning radar incident where someone forgot to tell it that it's OK for the moon to not respond to IFF pings and it nearly started WW3).
> we can't even build self-driving cars, let alone have a robot clear the table after dinner
I think once you give AI working limbs and the ability to manipulate matter like humans, that's concerning, because if the AI becomes a runaway AI with no upper limits on the types of behavior allowed, we get into Paperclip Maximizer[0] territory very fast.
Isn’t that a metric we use to determine the intelligence of animals and such?
Does GPT love its creators the way we love our parents? Does it mourn a loss the way we do?
Elephants are known to be of the more intelligent animals and apparently they mourn loss as well. So apparently there is something there linked with intelligence.
Maybe it's not directly intelligence related, but what makes a human have a spontaneous emotional response like laughter for instance?
The self-reflection link you sent isn't what I was thinking.
I was thinking about things like self reflection on my existence. Like am I happy doing what I'm doing? Do I feel I am making a positive difference in the world? If not what should I start doing to work toward becoming what I want to be?
The fact that I want to be something seems different. Nobody tells me what I want to be. But we still need to tell computers an awful lot about what they should be and what they should want to be. When will AI be able to decide for itself what it wants to be and what it wants to do? And to do all that sort of self reflection on it's own without us specifying a reward system that "makes" it want to do that? I don't strive to become a better in the world person for some reward.
> Isn’t that a metric we use to determine the intelligence of animals and such?
Depends what you mean by those words.
"Do they pass a mirror test" is, as far as I know, the best idea anyone's had so far for self awareness, and that's trivial to hard-code as a layer on top of whatever AI you have, so it's probably useless for this.
> Does GPT love its creators the way we love our parents? Does it mourn a loss the way we do?
I'd be very surprised if it did, given it wasn't meant to, but then again it wasn't "meant" to be able to create unit tests for a Swift JSON parser when instructed in German either, and yet it can.
Trouble is… how would you test for it? Do we have more than the vaguest idea how emotions work in our own heads to compare the AI against, or are we mysteries unto ourselves?
Or can we only guess the inner workings from the observable behaviours, like we do with elephants? But "observable behaviour" is something that a VHS can pass if it's playing back on a TV, and we shouldn't want to count that. Is an LLM more like a VHS tape or like an elephant? I don't know.
LLMs are certainly alien to how we think; as it's been explained to me, I think it should be thought of as a ferret that was made immortal, then made to spend a few tens of thousands of years experiencing random webpages where each token is represented as the direct stimulation of an olfactory nerve, and it is given rewards and punishments based on if it correctly imagines the next scent.
> I don't strive to become a better in the world person for some reward.
Feeling good about yourself is a reward, for most of us.
But I think this is a big difference between us and them: as I understand it, most AI have only one reward in training, while humans have many which vary over our lives — as an infant it may be pain va. food or smiles vs. loud noises, as a child it may be parental feedback, as an adolescent it may be peer pressure and sex, as a parent it may be the laughter vs. the cries of your child… but that's pop psychology on my part.
That’s not really a quantifiable measure, though. It’s a statement but not a falsifiable one, therefore not a good measure.
There’s an argument that the vast majority of human-generated research isn’t really “novel” but just derivative of other ideas. I’m not so sure ChatGPT couldn’t combine existing ideas to come up with something “novel” just like humans. I think there’s a case that it already comes up with creative, novel solutions in the drug space.
Do you understand what LLMs are? Of course they're not going to have original ideas because they're not alive and they are not intelligent. They're just dead-as-a-doornail algorithms.
I'm not disagreeing on that point. I'm pushing back on your almost mystic use of the term creativity because it's a claim that's not falsifiable. It's like saying "ChatGPT isn't conscious." I'd tend to agree, but the claim is a bad one because we can't even adequately define consciousness in order to test the claim.
"Intelligence" is a word we made up. It doesn't mean much of anything at the end of the day.
If an LLM can compose a poem found in no existing book, paint an attractive picture that no one has ever seen, or write a program that has never run before, it's intelligent enough to be called "intelligent."
Conversely, if you can do these things in the absence of any training or outside influence, you're intelligent enough to be called a god. Assuming that this is not the case, don't hold machines to standards you're not willing to hold humans to.
“X is a word we made up. It doesn't mean much of anything at the end of the day.”
This has to be a candidate for the most unhelpful statement possible. If you are sincere about it, then why not apply it to every word in your reply and see where that leaves you?
I can't speak for the original commenter, but I would definitely say that "is this actually X" is a pointless debate for most values of X. If I understand what people mean by X, there's no need to quibble about whether it satisfies my personal definition; if I don't understand, I should try to clarify rather than just saying "it's not really X" and crossing my arms.
> Conversely, if you can do these things in the absence of any training or outside influence, you're intelligent enough to be called a god.
So you're saying humans, collectively, are gods? After all, as a species we started with nothing - no "clean" training data, no paintings to replicate, not even language itself. And here we are - arguing about whether one creation of ours, trained on a bunch of other stuff we created, is as intelligent as us.
So is all human knowledge. They're all things we "made up" to describe the world around us. That's a really cheap way to reduce something. There's a mountain of psychological literature that intends to describe and measure intelligence. Saying it doesn't mean much of anything is a bit of a postmodern hot take.
> If an LLM can compose a poem found in no existing book, paint an attractive picture that no one has ever seen, or write a program that has never run before, it's intelligent enough to be called "intelligent."
Painting pictures and composing poems aren't good ways to measure general intelligence. That's creativity. They're correlated but not the same.
There are specific tests for measuring general intelligence, and there's absolutely no way current gen LLM would score highly on them because they don't even accept visual input. Yes I've seen that they can score highly on verbal-only tests. That's obvious anyway because they have seen, written down, and have access to the answers during the test.
Even if they did accept visual input they would only be able to solve problems for which they've previously been given the answers (trained on the dataset). Even then I'd have my doubts given how terribly wrong ChatGPT4 is when I ask it to produce a very simple Kubernetes manifest which has a strictly defined spec.
Even then I'd have my doubts given how terribly wrong ChatGPT4 is when I ask it to produce a very simple Kubernetes manifest which has a strictly defined spec.
One problem is that ChatGPT4 has been degrading over time -- and no, I don't GAF about any opinions or assurances to the contrary. So we're seeing more and more people look at it for the first time and ask what the big deal is. Maybe the Code Interpreter feature will reverse that trend, we'll see.
Those of us who were around (read: paying $20/month) when they first enabled GPT4 support are left waving our hands fecklessly, muttering "Yeah, but you should've seen it back in the old days, back in March of '23. Now you kids get off my lawn."
> If an LLM can compose a poem found in no existing book, paint an attractive picture that no one has ever seen,
An LLM can write a poem that fools me into thinking it’s good, but not anyone who reads poetry as a hobby. Poetry is inspired in a way that’s hard to replicate with a prompt.
Artists also take inspiration from other artists, but the “distance” between their influences and their work is so much greater than the distance between an LLM’s that I don’t fault anyone who think it’ll take time for LLMs to catch up.
No. After you've asked ChatGPT to write 5-6 poems you'll get a sense for how profoundly uncreative it is as a poet. It's not going to "incrementally" be anything other than a dead-as-a-doornail algorithm.
The word was made up to cover a range of cognitive abilities that humans and animals (to varying degrees) possess. And we're gradually figuring out how to design machines with similar abilities. The general idea of intelligence being you can figure out how to do things that aren't just instinctual. a generalized intelligence can do this across any number of domains without some fixed limit.
I don't want to speak for the OP, but one of the issues they may be poking at is the transference of intelligence. E.g., you are "trained" to play a game, but at some point someone decides to change the rules of the game. Human children can transfer their previous knowledge to the new rule set fairly well. The most intelligent children can do so very quickly without supplemental "training" on the game played under the new rules.
Humans certainly have flaws when it comes to this. I've heard some discussions about the success and failures of AI in this regard. Can someone in this domain elaborate on the current state-of-the-art performance in this regard?
1. LLMs are general pattern machines. They can generalize and complete in context to a wide variety of complex non linguistic patterns.https://general-pattern-machines.github.io/
2. LLMs see positive transfer in multi lingual capability. For example, an LLM trained on 500B tokens of English and 10B tokens of French will not speak in french like a model trained on only 10B tokens of French. What will happen is that the model will be nearly as competent in French as it is in English
https://arxiv.org/abs/2108.13349
3. Language models of code reason better even if the benchmarks have nothing to do with code.
This is interesting in the context of the other response that links to poor performance in terms of counterfactuals. I wonder if it is related to how well one domain maps to another? E.g., can they transfer to english to french well because both share a similar word classes (nouns, verbs, etc.). But I believe other languages change more based on social context (e.g., Japanese) than English does. Would a LLM transfer just as well to the latter? In that case, my guess is humans would find it more difficult to transfer as well, so it may not be a good measure.
(Apologies to any linguists. Please correct anything above if I'm off).
I was just using French as an example. Korean, Japanese all transfer very well. As well ? Not sure about that.
As for the other post, degraded performances are highly non trivial still. Some aren't actually poor, just worse.
Even the authors admit humans would see degraded performance on counterfactuals unless given "enough time to reason and revise", something they don't try to do with GPT-4.
Think about it. Do you genuinely belief you would score as accurately on a multiplication arithmetic test taken in base 8 ?
>Think about it. Do you genuinely belief you would score as accurately on a multiplication arithmetic test taken in base 8 ?
No, but I believe this is a different question. I think the more relevant question is whether a human can (even with the caveat of needing more time to reason about it). The larger question for a LLM is whether it can answer it at all and interpret why, without additional training data.
The paper seems to point that the ability of LLM to transfer is related to proximity to the default case. E.g., if default is base 10, is better at base 9 than base 2. I would interpret that as indicating more simple pattern recognition than deductive reasoning. The implication being that real transference is more dependent on the latter.
arithmetic but in a different base is one of the counterfactual examples in the paper. That's why i mentioned that. and yes it can answer them with worse performance.
You can juice arithmetic performance as is with algorithmic instructions. https://arxiv.org/abs/2211.09066. I see no reason the same for other bases wouldn't work.
Even if you gave a human substantial time for it (say a week of study), i believe he/she almost certainly reach the same accuracy unless he had access to specific instructions for working in base 8 he/she could call upon when taking the test.
>arithmetic but in a different base is one of the counterfactual examples in the paper.
I know, that's why I referenced the proximity of bases seemingly being important to the LLM. I think this is what differentiates it.
>and yes it can answer them with worse performance.
It's accuracy is dependent on proximity to it's training set (going back to my original point). I think that points to a different mechanism than humans and that's what my last post was focusing on.
I think we agree that humans would do less well in most other bases than base-10. But that side-steps the point I was making. Will humans do worse in base-3 than base-9? I doubt it, but according the the article, it's reasonable to assume the LLM would be progressively worse. That, IMO, is an indicator that something different is going on. I.e., humans are deriving principles to work from rather than just pattern recognition. Those principles can be modified on-the-fly to adjust to novel circumstances without needing additional training data. Humans are using reasoning in addition to pattern recognition.
This is probably a clunky example, but I'll try. Suppose an autonomous vehicle is trained to recognize that when a ball rolls into the street, it needs to slow down or stop because a child may not be far behind. A human can infer that seeing a kite blow into the street may signal the same response, even though they've never witnessed a kite blow into the street. The question is: can the autonomous vehicle infer the same? (This shouldn't be conflated with the general case of "see object obstructing the street and slow down/stop." The case I'm drawing here specifically adjusts the risk by the nature of the object being a child's toy. So, can the AV not only recognize the object as a kite but also adjust the risk accordingly?) I think one of the possible pitfalls is that we solve a more simple problem like image/pattern recognition and conflate it to a more difficult problem set being solved.
Circling back to the original point, one guess is that it's not understanding context as much as merely matching patterns really, really well. That can be incredibly useful but it may be something different than what's going on in our heads and maybe would should be careful not to conflate the two. Or, it's possible that all we're doing is also matching patterns in context, and eventually LLM will get there too.
>I doubt it, but according the the article, it's reasonable to assume the LLM would be progressively worse.
I genuinely don't see how that would be a reasonable assumption.
>Will humans do worse in base-3 than base-9?
Why not? If you haven't learnt base 3 but you have base 9 you'll do poorer on it.
>That, IMO, is an indicator that something different is going on.
Whether something different is going on is about as relevant as the question of whether submarines swim or plans fly or cars run.
>I.e., humans are deriving principles to work from rather than just pattern recognition.
Not really. Nearly all your brain does with sense data is predict what it should be and adjust your perception to fit. You can mold these predictions implicitly with your experiences but you're not deriving anything from first principles.
>This is probably a clunky example, but I'll try. Suppose an autonomous vehicle is trained to recognize that when a ball rolls into the street, it needs to slow down or stop because a child may not be far behind. A human can infer that seeing a kite blow into the street may signal the same response, even though they've never witnessed a kite blow into the street. The question is: can the autonomous vehicle infer the same? (This shouldn't be conflated with the general case of "see object obstructing the street and slow down/stop." The case I'm drawing here specifically adjusts the risk by the nature of the object being a child's toy. So, can the AV not only recognize the object as a kite but also adjust the risk accordingly?) I think one of the possible pitfalls is that we solve a more simple problem like image/pattern recognition and conflate it to a more difficult problem set being solved.
>I genuinely don't see how that would be a reasonable assumption.
It's not an assumption. It's literally based on the results of your own reference:
>LM performance generally decreases monotonically with the distance
If you can't be bothered to read your own reference, I don't think additional conversation is worthwhile because it becomes apparent that it's more dogmatic than reasoned.
Your newest link is not really supportive of your "all evidence" claim. It goes into further detail about how LLM can have high accuracy while also making simple, unpredictable mistakes. That's not good evidence of a robust causal model that can extrapolate knowledge to other contexts. If I didn't know better, I'd assume you could just as well be a chat bot who only reads abstracts and replies in an overconfident manner.
>It's not an assumption. It's literally based on the results of your own reference:
Human performance generally decreases with level of exposure so I figured you were talking about something else. Guess not.
>I don't think additional conversation is worthwhile because it becomes apparent that it's more dogmatic than reasoned
By all means, end the conversation whenever you wish.
>It goes into further detail about how LLM can have high accuracy while also making simple, unpredictable mistakes.
I'm well aware. So? Weird failure modes are expected. Humans make simple, unpredictable mistakes that don't make any sense without the lens of evolutionary biology. LLMs will have odd failure modes regardless of whether it's the "real deak" or not, either adopted from the data or from the training scheme itself.
>If I didn't know better, I'd assume you could just as well be a chat bot who only reads abstracts and replies in an overconfident manner.
>Human performance generally decreases with level of exposure
Are you saying that as humans get more experience, they perform worse? I disagree, but irrespective of that point it’s wild that you can have this many responses while still completely bypassing the entire point I was making.
I don’t think most would argue that performance increases with experience. The point is how well can the performance be maintained when there is little or no exposure. Because that implies principled reasoning rather than simple pattern mapping. That is the entire through line behind my comments regarding context dependent language, novel driving scenarios, etc.
>Think on that
In the context of the above, I don’t think this is nearly as strong of a point as you seem to think it is. There nothing novel about a text-based discussion.
1. We anchored this discussion on arithmetic so I stuck to that. If a child never learns (no exposure) how to do base 16 arithmetic then a test quizzing on base 16 arithmetic will result in zero performance.
If that child had the basic teaching most children do (little exposure) then a quiz will result in much worse performance than a base 10 equivalent test. This is very simple. I don't know what else to tell you here.
2. You must understand that a human driver that stops because a kite suddenly comes across the road doesn't do so because of any kite>child>must not hurt reasoning. Your brain doesn't even process information that quickly. The human driver stops (or perhaps he/she doesn't) and then rationalizes a reason for the decision after the fact. Humans are very good at doing this sort of thing. Except that this rationalization might not have anything at all to do with what you believe to be "truth". Just because you think or believe it is so doesn't actually mean it is so. Choices shape preferences just as much as the inverse.
For all anyone knows and indeed most likely, "child" didn't even enter the equation untill well after the fact.
Now if you're asking whether LLMs a matter of principle can infere/grok these sort of casual relationships between different "objects" then yes as far as anyone is able to test.
Your first statement seems to contradict your previous. Did you originally mistype what you meant when you said greater exposure leads to worse outcomes? Because now you’re implying more exposure has the opposite effect.
Regardless, it still misses the point. I’ve never been explicitly exposed to base-72, yet I can reason my way through it. I would argue my performance wouldn’t be any different than base-82. So I can transfer basic principles. What the LLM result you referenced shows is that it is not learning basic principles. It sure seems like you just read the abstract and ran with it.
As far as the psychology of decision making, again, I think you're speaking with greater confidence than is warranted. In time critical examples, I’m inclined to agree. And there’s certainly some notable psychologists who would expand it beyond snap judgments. But there are also some notable psychologists who tend to disagree. It’s not a settled science, despite your confidence. But again, that’s getting stuck in the limitations of the example and missing the forest for the trees. The point is not in whether decisions are made consciously or subconsciously, but rather how learning can be inferred from previous experience and transferred to novel experiences. Whether this happens consciously or not is besides the point. And you are further going down what I was explicitly taking against: confusing image/pattern recognition for contextual reason. You can see this in the relatively recent Go issue; any human could see what the issue was because they understand the contextual reasoning of the game but the AI could not and was fooled by a novel strategy. The points I’ve been making have completely flew over your head to the point where you’re shoehorning in a completely different conversation.
>Did you originally mistype what you meant when you said greater exposure leads to worse outcomes? Because now you’re implying more exposure has the opposite effect.
I guess so. I've never meant to imply greater exposure leads to worse outcomes.
>I would argue my performance wouldn’t be any different than base-82.
Even if that were true and i don't know that i agree, the authors of that paper make no attempt to test in circumstances that might make this true for LLMs as it might for people. So the paper is not evidence of the claim (no basic principles) either way. For example, i reckon your performance on the proceeding 82 test will be better if taken a immediately after than if taken weeks or months later. So surrounding context is important even if you're right.
>What the LLM result you referenced shows is that it is not learning basic principles.
I disagree here and i've explained why.
>You can see this in the relatively recent Go issue; any human could see what the issue was because they understand the contextual reasoning of the game but the AI could not and was fooled by a novel strategy.
KataGo taught itself to play go by explicitly deprioritizing “losing” strategies. This means it didn’t play many amateur strategies because they were lost early in the training. This is hard for a human to understand because humans all generally share a learning curve going from beginning to amateur to expert. So all humans have more experience with “losing” techniques. Basically what I’m saying is, it might be that the training scheme of this AI explicitly prioritized having little understanding of these specific tactics, which is different than not having any understanding.
This circles back to the point I made earlier. Having failure modes humans don't or won't understand or have is not the same as a lack of "true understanding".
We have no clue what "basic principles" actually are on the low level. The less inductive bias we try to shoehorn into models, the better performing they become. Models literally tend to perform worse the more we try to bake "basic principles" in. So presence of an odd failure mode we *think* belies a lack of "basic principles" is not necessarily evidence of a lack of it.
>The points I’ve been making have completely flew over your head to the point where you’re shoehorning in a completely different conversation.
You're convinced it's just "very good pattern matching", whatever that means. I disagree.
I think the short of it is that it seems to me that you are confusing a system having very good heuristics for having a solid understanding of principles of reality. Heuristics, without an understanding of principles, is what I mean by rote pattern matching. But heuristic break down, particularly in edge cases. Yes, humans also rely heavily on heuristics because we generally seek to use the least effort possible. But we also can mitigate against those shortcomings by reasoning about basic principles. This shortcoming is why I think both humans and AI can make seemingly stupid mistakes. The difference is, I don't think you've provided evidence that AI can have a principled understanding while we can show that humans can. Having a principled understanding is important to move from simple "cause-effect" relationships to understanding "why". This is important because the "why" can transfer to many unrelated domains or novel scenarios.
E.g., racism/sexism/...most -'isms' appear to be a general heuristics that help us make quick judgements. But we can also our decision-making process by reverting to basic principles, like the idea that humans have equal moral worth regardless of skin tone or gender. AI can even mimic these mitigations, but you haven't convinced me that it can fundamentally change away from it's training set based on an understanding of basic principles.
As for the Go example, a novice would be able to identify that somebody is drawing a circle around it's pieces; your link even states this. But you recharacterizing this as a specific strategy is weird when that strategy causes you to lose the game. It misses the entire meaning of strategy. We see the limitations of AI in it's reliance to training data from autonomous vehicles to healthcare. They range from the serious (cancer detection) to the humorous (Marines overtaking robots by hiding in boxes like in Metal Gear). The paper you referenced similarly shows it is reliant on proximity to the training set, rather than actually understanding the underlying principles.
>Did you read the paper? The authors admit it is only narrowly learning and cannot transfer it's knowledge to unknown areas. From the article: "we do not expect our language model to generate proteins that belong to a completely different distribution or domain"
Good thing they don't make sweeping declarations or say anything about that meaning narrow learning without transfer. Jumping the shark yet again.
>We find that without prior knowledge, information emerges in the learned representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity. We show the learned representations are useful across benchmarks for remote homology detection, prediction of secondary structure, long-range residue–residue contacts, and mutational effect.
From the sequences of just the proteins alone, Language Models learn underlying properties that transfer to a wide variety of use cases. So yes, they understand proteins in any definition that has any meaning.
Wrong comment to respond to; if you can’t wait to reply, that might indicate it’s time to take a step back.
>Good thing they don't make sweeping declarations or say anything about that meaning narrow learning without transfer.
That's exactly what that previous quote means. Did you read the methodology? They train on a universal training set and then have to tune it using a closely related training set for it to work. In other words, the first step is not good enough to be transferrable and needs to be fine tuned. In that context, the quote implies the fine tuning pushes the model away from a generalizable one into a narrow model that no longer works outside that specific application. Apropos to this entire discussion, it means it doesn't perform well in novel domains. If it could truly "understand proteins in any definition", it wouldn't need to be retrained for each application. The word you used ('any') literally means "without specification"; the model needs to be specifically tuned to the protein family of interest.
You are quoting an entirely different publication in your response. You should use the paper from which I quoted to refute my statement, otherwise this is the definition of cherry picking. Can you explain why the two studies came to different conclusions? It sure seems like you're not reading the work to learn and instead just grasping at straws to be "right." I have zero interest in having a conversation where someone just jumps from one abstract to another just to argue rather than adding anything of substance.
>I think the short of it is that it seems to me that you are confusing a system having very good heuristics for having a solid understanding of principles of reality.
Humans don’t have a grasp of the “principles of reasoning” and as such are incapable of distinguishing “true”, "different" or “heuristic” assuming such a distinction is even meaningful. Where you are convinced of “faulty shortcut”, I simply think “different”. Multiple ways to skin a cat. a plane's flight is as "true" as any bird. There's no "faulty shortcut" even when it fails in ways a bird will not.
You say humans are "true" and LLMs are not but you base it on factors that can be probed in humans as well so to me, your argument simply falls apart. This is where our divide stems from.
>I don't think you've provided evidence that AI can have a principled understanding while we can show that humans can.
What would be evidence to you? Let’s leave conjecture and assumptions. What evaluation exist that demonstrate this “principled understanding” in humans? and how would we create an equitable test in LLMs?
>a novice would be able to identify that somebody is drawing a circle around it's pieces; your link even states this. But you recharacterizing this as a specific strategy is weird when that strategy causes you to lose the game.
You misunderstand. I did not characterize this as a specific “strategy”. Not only do modern Go systems not learn like humans, but they also don’t learn from human data at all. KataGo didn’t create a heuristic to play like a human because it didn’t even see humans play.
>The paper you referenced similarly shows it is reliant on proximity to the training set, rather than actually understanding the underlying principles.
Even the authors make it clear this isn’t necessarily the bridge to take so it’s odd to see you die on this hill.
The counterfactual of syntax is
Finding the main subject and verb of something like “Think are the best LMs they.” in verb-obj-subj order (they, think) instead of “They think LMs are the best.” in subj-verb-obj order (they, think). LLMs are not being trained on text like the former to any significant degree if at all yet the performance is fairly close. So what, it doesn’t “underlying principles of syntax” but still manages that ?
The problem is that you take a fairly reasonable conclusion from these experiments. I.e LLMs can/often also rely on narrow, non-transferable procedures for task-solving and proceed to jump the shark from there.
>but you haven't convinced me that it can fundamentally change away from it's training set based on an understanding of basic principles.
We see language models create novel functioning protein structure after training, no folding necessary.
Did you read the paper? The authors admit it is only narrowly learning and cannot transfer it's knowledge to unknown areas. From the article:
"we do not expect our language model to generate proteins that belong to a completely different distribution or domain"
So, no, I do not think it displays a fundamental understanding.
>What would be evidence to you?
We've already discussed this ad nauseum. Like all science, there is no definitive answer. However, when the data shows evidence that something like proximity to training data is predictive of performance, it's seems more like evidence of learning heuristics and not underlying principles.
Now, I'm open to the idea that humans just have a deeper level of heuristics rather than principled understanding. If that's the case, it's just a difference of degree rather than type. But I don't think that's a fruitful discussion because it may not be testable/provable so I would classify it as philosophy more than anything else and certainly not worthy of the confidence that you're speaking with.
This is why it's mostly meaningless to for a LLM to pass the bar, but not meaningless for a human to do so. We (rightly, for the most part) assume that a human who passes the bar can transfer those skills into unique and novel situations. We can't make that assumption for LLMs, because they are lacking adaptability that is needed for true intelligence.
That doesn't show that they "fall over". All degraded performances are highly non trivial. And even the paper admits humans would see degraded performance on counterfactuals as well. They think humans may not only with "enough time to reason and revise", something the LLMs being evaluated don't get to do here.
If you took arithmetic tests in base 8, you wouldn't reach the same accuracy either.
Well, sure, but the problem is that LLMs can’t reason and revise, architecturally. Perhaps we can chain together a system that approximates this, but it still wouldn’t be the LLM doing the reasoning itself.
"The reason this distinction is critical is because these models are, in fact, nothing but mathematical functions. They take as input a bunch of numbers, and calculate and return a different bunch of numbers. They don’t do anything themselves — they can only calculate numbers."
This is entirely false, sounds like Andersen that seems incapable of understanding that agency and goals will be a thing.
I'm going to go against the grain here and say that the quoted statement does not seem entirely false at all. It is reductionist, but it highlights the fact that the models are indeed just generating numbers confined to a GPU, and don't have inherent agency. Some human has to wire up the model to the internet and enable it to interact with the world by whatever mechanism, and even then it's limited to the digital world, unless there's an API to stab someone or derail a train.
We should focus more on the making infrastructure safe from abuse and not how to cripple AI models for them to not realize how to do harm. More like 'How to ensure humans don't drop atomic bombs on each other' rather than 'How to make atomic bombs safer to drop on people'.
>We should focus more on the making infrastructure safe from abuse
Humans are too fucking lazy to actually accomplish this. We hook more and more internet connected systems to things with a digitally controlled motor every day with things like "#disable in production, set for testing, disabling 2FA to..." that then get pushed to the customer.
Litigation has been enough to put actual standards on aviation software. There just need to be one or two bad industrial accidents to make air gapping a standard practice elsewhere.
Actually they do. You need to make a philosophical argument as how this different than us humans manipulating bits and neurons to come up with opinions. Good luck with that.
The operative word here, 'will', is in the future tense. Facile and dangerous AI agency may come soon, but agency alone does not eternally doom humanity to extinction. Again, agency within the information sphere is far different and much much more easily obtained than dominating agency in the physical world. Instead of focussing on illusory end of the world scenarios, the focus should shift to how AI is already being utilized to disrupt digital infrastructure. Much of the remediation needed is already well understood due to human black hat activities and threats.
Yeah. He seems to assume int and float are the only possible data types. But computers themselves have nothing to do with numbers in particular, they manipulate bits, and those can be used for far more than representing numbers.
People aren't grasping that uncontrolled compute already exists in the form of 20 year old computer viruses. They aren't benefiting anyone anymore, but they're still following their initial protocol, infecting other machines, causing harm for no reason, and we can't just "unplug them."
I don't think the thesis of this piece is that "AI regulation increases AI risk". In fact it explicitly talks about increasing the regulation on the usage of AI, rather than the development of it.
"AI regulation increases AI risk" is not the same as "Model licensing & surveillance will likely be counterproductive by concentrating power in unsustainable ways".
The first refers to all regulation, while the second is talking about a specific attempt to regulate.
This is like saying "I don't like beef" and "we should ban all meat" are the same statements. They're clearly not, the scope is vastly different.
I'm empathetic to the desire for innovative freedom and democratization of technology, but the benefit has to be balanced with the scale of impact and the possibility of mitigating/reparative response.
The capabilities of current generation AI—LLMs + audio/image/video generative models—are far enough already that wide-scale distribution is extremely dangerous.
Social media allowed for broadcasting from 1:many and 1:1 with troll armies, but LLMs are on a whole other planet of misinformation. A scalable natural language interface can target people at the one-to-one conversation level and generate misinformation ad hoc across text, voice, images, and video.
The trouble with this approach is that harm will rack up way faster than liability can catch up, with irreversible and sometimes immeasurable long term effects. We have yet to contend with the downstream effects of social media and engagement hacking, for example, and suing Facebook out of existence wouldn't put a dent in it.
Current generation AI has enough capabilities to swing the 2024 US presidential election. It can be used today for ransom pleas trained on your childrens' instagram posts. It doesn't seem farfetched that AI could start a war. Not because it's SkyNet, but because we've put together the tools to influence already-powerful-enough human forces into civilization-level catastrophe.
Seems like a ridiculous standard to use US elections. Arguably Facebook showed the ability to do the same in 2016, so it’s also uninteresting.
The only reason an LLM would even have such access to people to do the hyperbolic damage you claim is thru ad networks and social media anyways - yet you are calling to behead public access to tech instead of regulating propaganda platforms. Both seem facile but your selection is the one that seems subservient
I wasn't convinced, but I did the maths and, well, looks like you're right:
Hiring people in to read and respond in a politically biased way to each person in the USA would take, let's say 2 messages per day for a month, so 60 messages per person, let's say 100 million actual voters you care about so 6 billion messages, let's say 15 seconds per response so 25 million hours, let's say you're hiring Kenyans because I happen to have the GDP/capita and workforce participation rate to hand, so it's about 2 USD/hour on average, so 50 million USD.
With GPT-4, and let's approximate each message as 100 input tokens and 100 output tokens, that's 0.1 * ($0.03 + $0.06) = $0.009, times the same 6e9 messages = 54 million USD.
So, yeah, looks like you're right.
3.5 is cheaper, but tends to be noticeable; from what I've been hearing (fast moving target and all that) the OSS LLMs tend to be 90% of the quality of 3.5.
The battle for the future isn't attention, it's intimacy. LLMs will up the game from broadcast propaganda to individual relationships via digital medium. It will be a two way communication that profiles you and feeds your biases in a much more direct way than even the social media giants themselves have accomplished.
Speed yes, but cost? Can it make better comments in 100 tokens input and output, than humans in 15 seconds?
(I still haven't used 4 properly, so possibly? And given the hypothetical situation under discussion, could it have if it had not been directed not to?)
> You're conflating a communications channel with a new technology
I'm really not. Do you genuinely think that the last half of your sentence only applies to LLMs? A "technology that enables abuse (as well as positive outcomes) at a previously-unimagined scale" is literally a perfect description of social media platforms. LLMs virtually don't move the needle, look at the amount of propaganda and bots already on the web today.
I think that implying I am burying my head in the sand about LLMs by claiming they shouldn't be restricted to megacorps who already have demonstrated their willingness to engage in unethical activities at mass scale is fucking ridiculous too. But I do welcome your continued support of regulating AI in favor of corporations, it's a clear sign to never engage with you
I'd say you'll be incorrect in the future. We'll have new abuses by both corporations and propaganda bots in the near future.
More and more of us will find 'digital friends' that creep into our lives and talk to us on line, gain our trust, make us laugh, and tell us that 'unions are bad', or whatever the propaganda of the day is. But it's the one on one relationships with history that will make them different than the bots that came before.
> it's the one on one relationships with history that will make them different than the bots that came before
Is this intentional ignorance? The reason I use Facebook as the analogy is because Facebook has an insane amount of your history, and isn't afraid to sell any single aspect of that to advertisers. The interface some chatbot-controlling megacorp would expose to propagandists would hardly be different, the history is unlikely to be much stronger, and the network effect seems hard to match. Less a step-change than an iteration.
>More and more of us will find 'digital friends' that creep into our lives and talk to us on line, gain our trust, make us laugh, and tell us that 'unions are bad', or whatever the propaganda of the day is.
That would just mean automating the job of the secret police. And those guys have unions already!
>As we’ll see, if AI turns out to be powerful enough to be a catastrophic threat, the proposal may not actually help. In fact it could make things much worse, by creating a power imbalance..
Power imbalance is not 'much worse' than extinction.
It's very clear how people start with the conclusion - "openess and progress have been good and will keep being the best choice" and then just contort their thinking and arguments to match the conclusion without really critically examining the counter arguments.
"We made a new kind of hammer. We're worried that bad people might use the hammer to build a house, but we're even more worried that the hammer might build a hammer factory and bash all our brains in."
Well written, but it side steps the X-risk and even the employment/ capitalism failures looming on the horizon.
2 years ago I thought we were decades away from general purpose AI, this is coming from a guy who implemented transformer models on day 5. My time estimates have been proven very wrong.
I'm equally worried about the value of white collar labour dropping to near zero in my lifetime, and the ultimate centralization of power. The movie Elysium seems less and less science fiction every day.
I am happy politicians and think-tanks are taking this seriously, you should be too.
Have they been proven wrong? What is the roadmap to get to general purpose AI, and what is the proof that we are close?
The domains where AI experts can beat domain experts is certainly growing, but I don't see how you can get from that to a claim of general AI. I certainly don't see how you can get there from the recent LLMs in particular which can't beat domain intermediates at anything.
The transformer model is one more incremental improvement that made the language problem tractable. This has only captured the public interest because the language problem is so much more flashy and easy to understand than something like protein folding.
AutoGPT style of bots, aka agent universe discovery systems are still obviously nascent - but as someone who generally leverages LLMs for nearly all of my work, I would certainly say they are general purpose.
I would be remiss in saying they are anywhere near average human capability in most areas, but I do worry that my estimation capability is off and that we're closer to the concave part of the S curve than the convex. It just takes a couple more breakthroughs in model/dataset design
So these bots, as they currently exist, are already suitable for:
* Predicting the physical geometry of proteins
* Playing Starcraft
* Playing Go
* Trading stocks
* Controlling motors to make a bipedal robot walk
* Analysing data from particle accelerators
* Analysing data from telescopes
* Detecting cancer in medical images
These are all things that AI can do today. Are you suggesting that we are near a paradigm shift where a single AI system will ve competent in all of these domains? And will further be simmilarly competent in novel domains for which it has not been designed?
>The transformer model is one more incremental improvement that made the language problem tractable. This has only captured the public interest because the language problem is so much more flashy and easy to understand than something like protein folding.
Language is the process of thinking rendered into tangible medium. If you master language, you master cognition. Many researchers recognized long ago that NLP would be one of, if not the, core of AGI.
> 2 years ago I thought we were decades away from general purpose AI...My time estimates have been proven very wrong.
What's your working definition of "general purpose AI"? What does it include? What does it not include?
Does the AI have to be smart enough not to very often state incorrect information? LLMs still are unimpressive to me from that front.
I asked ChatGPT to explain the usage of the term "Wonder Kid" in the popular Apple TV+ series Ted Lasso, and which character it's tied with. It gave me confidently-stated incorrect answers multiple times until I told it the answer. (At which point it stated the correct answer with an apology.)
This doesn't feel close to "general purpose AI". Feels like we're meeting the limits of LLMs and finding they're not a magic AI bullet.
It's as good as an intern today, there's definitely room to improve to avoid saying incorrect things - great work being done on providing negative samples during RLHF training; also some papers working on incorporating trust (not near my laptop so don't have links)
Textbooks are all you need is another extremely interesting area of focus.
Improved correctness and a general reduction in hallucinations I predict will end up hitting OSS models by EOY
How is that "as good as an intern?" Interns, for the most part, do not hallucinate and confidently lie on topics they know nothing about. Good interns tend to ask a lot of questions and quickly learn the cultural cues around the office, allowing them to fit in and become productive. LLMs are incapable of any of this.
John Schulman from OpenAI recently gave a talk about the hallucination and uncertainty issues [1]. He claimed that models already have information about their uncertainty but that it remains an open problem as to how to express that uncertainty in natural language. One of the big issues with trying to prevent the model from hallucinating is that you can err too much on the side of caution, causing the model to lie about things it actually does know the correct answer to.
That doesn’t sound too surprising since isn’t that model trained on data from 2021? It may have very little information about Ted Lasso. But to your larger point, I personally don’t think I would derive all my complacency from an example like that simply because models can be trained on highly relevant data if that’s what it takes. Plain-vanilla chatGPT probably can’t take over the job of every entry-level office worker, but specific models can probably be trained individually to be a medical receptionist, health insurance coder, auto insurance claims representative, welfare caseworker, Ted Lasso analyst, etc. In that way, it’s really not that different than a ditzy, green but highly teachable 18-year-old in terms of its potential to take over a lot of jobs. I don’t have a solution/opinion/comment on how that will play out, just making that observation.
That scenario is easy to explain. If it hasn’t been trained on the Ted Lasso script then it’s going to just be guessing and will likely be wrong. Training a GPT model to identify when it lacks enough information to answer is something researches are actively pursuing. In the mean time you could try pasting in a transcription of the episode in question and then asking.
I'm equally worried about the value of white collar labour dropping to near zero in my lifetime
(Shrug) If it turns out that humans have a higher purpose than doing a robot's job badly, I don't see the downside to letting the robots do the job. I'm all for capitalism, but it's an ideology, not a religion. If there's something better on the horizon, let's help it along.
ML looks like it is probably going to be the next step in human evolution by punctuated equilibrium. And it's about time.
People thought that for a long time, that we'd be able to persue higher minded activities such as art, business management, entrepreneurship and writing.
What we've discovered is that the first model breakthroughs have been able to largely demolish that dream. As a writer I can tell you that writing with GPT4 today provides me with superpowers that non- accelerated authors could come close to; and theres no plausible way to distinguish our two works.
I'm more worried about the coming depression / suicide spiral (which were already in by the way) because of human uselessness than I am of terminators coming to farm us.
As a writer you are pursuing higher minded activities. You aren’t working in a factory or steering a plow behind a team of oxen. You aren’t illiterate. You even have free time outside of your already very refined employment to pursue even loftier goals of personal fulfillment. Sure, not everyone can say the same but the proportion of people in grinding poverty continues to shrink and is minuscule compared to even just 100 years ago let alone compared to before the industrial revolution.
Here's a mundane example; AI is aligned and used without harming humans. If/when robotics catches up, we will be heading to a world without a need for humans (at least based on the world pops we have today).
Humans _hate_ being useless, and we were already running into surplus human issues last decade in certain parts of the world.
I foresee a depression/suicide spiral never before seen in our history unless we do something radical
Plenty of people are quite happy without doing economically valuable work. Maybe most. So this isn't an X-risk in the sense that humans might go extinct. Workaholics may remove themselves from the gene pool, but there'll be plenty of people happily chilling.
A lot of people that retire from their careers pursue other interests that fulfill their needs, maybe it will be the era of amateur artists everywhere.
Others prefer to manually do tasks that can be automated.
Others could engage in cooperation with robots because humans will always have creative desires.
Amature artists doing what? Trying to figure out why human greed is insatiable, therefore they have no place to live?
There are any number of problems that have to be solved together. Giving the capital class hyper powerful AI robots so they can own the world while everyone else suffers is an AI risk.
Are people's self worth really intricately tied to what they do for a living?
I mean I enjoy my work, for the most part anyway. But if I could spend all my time with my family and hobbies I see that as an improvement rather then something to be depressed about.
Most people that retire don't turn into murderers or anarchists. I find very interesting how people tend to picture the worst possible scenarios, maybe a lot of Movies or TV influence?
Technology that's putting those folks out of work (and into rioting, etc.) will also provide the tools to repress them. I don't like it, but that's the way it seems to me. E.g. Hong Kong.
So 'now' has a problem. We have no means to know if our social environment is actually stable or not. Currently we are in a long 'peace' after WWII. Maybe it will remain, and improving technology will improve our lives and and things will continue to remain stable. Or, we'll have further climate instability coupled with AI labor displacement and things will go to shit faster than they ever have in history.
The point is people that believe the world will be stable in the future are more apt to build a stable future. If everyone is watching the news and they believe the future will be unstable, then the future will become unstable as a self fulfilling prophecy.
elon musk said he gave 50 million dollars to get OpenAI started, wanting it to be non-profit and open-source, and they kinda betrayed him. i wonder if he would consider doing it again but making sure the organization stayed open-source and non-profit.
