I recall an AI playing a game that, just before losing, would press the "menu" button to pause the game and then stop doing anything. It fulfilled it's goal (not lose). That behavior should set expectations when it comes to AI.
Also, am-I the only one that thinks this whole "chat bot having a natural conversation to book an appointment" is useless when a simple date-picker would do?
Another fun one is when the programmers accidentally give the AI the wrong goal.
I have no idea if this actually happened, but I've heard of a chess program that was playing in a tournament that started making really weird moves in the endgame. Before that point, it was playing excellently.
It took the developers a while to figure out what was going on. They had made a mistake when doing some last minute tweaks before the tournament, and in effect the program was playing to lose.
Think about that for a minute. At first you might think losing would be easy. Just don't defend against your opponent's attacks, and make moves that weaken your position to make it even easier for the opponent.
But wait...the mistake in the code applied to the program's evaluation of both its own moves and the opponent's possible moves. In other words the program assumed that the opponent was also playing to lose.
How do you play to lose a game of chess if your opponent also wants to lose? You need to get to a position where the only legal move of the opponent is to checkmate you.
You'll want a position where you have a big material advantage, and all the opponent has is their king and enough material to mate you. Probably just king and queen. Then you'd need to keep putting them in check, in such a way that they have to block with the queen. You'd need to arrange a series of such checks and blocks so that the final block also delivers checkmate on you.
And so it turns out that during the opening and middle game, playing to lose against someone who is also playing to lose looks pretty much the same as playing to win against someone who is also playing to win.
(Personally, I doubt this actually happened. The story is old, and I don't think chess programs would have been able to see far enough ahead for them to discover that getting an overwhelming position is the way to force the opponent to checkmate them).
> (Personally, I doubt this actually happened. The story is old, and I don't think chess programs would have been able to see far enough ahead for them to discover that getting an overwhelming position is the way to force the opponent to checkmate them).
True, but (in the traditional minimax-alpha-beta-classic-gameplay model) you're using heuristics anyway up until you're in spitting distance of the end, and it seems plausible that if this "tweak" involved something like negating something and flipping a less-than sign (or whatever) that the heuristics were evaluating correctly but the end game evaluations were backwards. (Which contradicts the explanation but not the overall story.)
It's also possible that the explanation does work even with a backwards heuristic: in the try-to-win version, I'll eliminate or downgrade one branch because my only success route would be if the opponent directly manouvres themselves to be captured, which they obviously wouldn't do; but in the try-to-lose version, I might eliminate the same branch because I expect the opponent would do that but I don't want that to happen. I can't quite fully work out the logic in my head but it seems plausible.
I'd believe it. I had a similar bug in a minimax AI I made to play Hive. I had messed up the game-end valuation in such a way that the AI basically valued winning less than anything but outright losing. It would play a perfectly normal and competent game right up until the end, at which point it would try its hardest to draw out the game indefinitely, usually by forcing me into terrible positions that nonetheless didn't particularly benefit it at all.
(Like blahedo points out, this is possible because, unlike say basic MCTS, minimax uses a heuristic to judge board positions prior to game end, so it is possible for the two metrics to be out-of-sync.)
For anyone curious, this variant of chess where both players are trying to lose is simply called "anti-chess" and many chess programs have an anti-chess mode. Playing it is really weird.
This reminds me of the first time I took LSD. I ended up at a hippy coffee shop with a Canadian soccer player (also tripping) playing a game of chess with the goal of losing.
It’s hard to play to lose! We ended up having a long conversation with a bearded man who kept telling us he’d been where we were.
The chess computer story you’re half remembering sounds a lot like Kasparov playing against deep blue. I can’t find a link right now but somewhere on the internet there is an article where Kasparov describes the match in his own words and its really quite beautiful.
> am-I the only one that thinks this whole "chat bot having a natural conversation to book an appointment" is useless when a simple date-picker would do?
Nope. I find being forced to converse in English with a machine to be absolutely infuriating. I know what I want and how to tell it to a machine. Being forced to add noise words to allow my request to pass through a useless extra layer is a disrespectful waste of my time and mental energy.
I will happily use an automated menu-driven system. But as soon as it forces me to "converse" with it, I do whatever I can to force it to connect me to an actual human.
I remember some car rental system switched from having you enter your confirmation number on the keypad to voice recognition.
I'm most likely to use this system in a busy airport. After enough times of it not understanding me, it finally put me on hold for half an hour for me to read the number to a person. Something that took me less than a minute previously now was super frustrating.
I must not have been the only person this affected because they added back in the option to use touch tone.
> I know what I want and how to tell it to a machine.
Yeah, Google search is not that great when you want to do conditional search in a topic with many false positives. For example I want to find a light electric scooter, under 10Kg of weight. Google will happily report all the pages that contain scooter and kg, but the kg would be for the max weight of the person, not the scooter itself. How do I tell it that in keywordese?
Actually I feel like Google is the one such service that is approaching natural language usability for general topic queries. It seems to me to be able to extract basic meaning from simple English phrases and search that sentiment. Whereas search phrases containing negatives ("not", "isn't") used to be futile, Google now correctly infers your meaning.
But yes, for parametric search I think I would always prefer a direct interface. It makes clear what is indexed and what isn't.
Maybe, but can I please not be forced to do business via the robots while they are terrible?
Also, beside being spectacularly good, the AI has to also be actually empowered for me not to find the process wasteful and insulting. The AI should actually be able to solve my problem as a result of natural language communication. Not just walk me through a prepared script: that's frustrating even when a real live human does it.
Except that I, too, would rather book an appointment with a date picker than talk to anyone, much less a machine programmed to act like a person. It's like in order to make coffee, getting in my car and driving around the block ten times, parking in front of my house and going in to make coffee, vs just MAKING COFFEE, without doing a bunch of time wasting fluff first.
This might be an age thing because I was like that when I was younger. I didn't understand why I had to keep interacting with seemingly pointless humans in service industries everywhere. But now I'm older, I find talking is more comfortable than using a computer. I don't quite know why. Maybe speaking to a human is something we're biologically built for and these other modes of interaction are a little extra taxing on the brain even if they appear simpler. They involve reading and writing which every school kid knows is harder work than talking.
Or maybe it's just that so many computer interactions are actually slow and frustrating. Ever tried ordering at McDonalds using the touch-screen machines? It's aggravatingly tedious and complicated. There's no "take my money and go away" button. You have to navigate your way through a bunch of stupid menus trying to up-sell you, and each one has an unforgivable loading time.
They're doing what you tell them to, not what you want them to. Same with GPT-3; it's predicting what (from its experience) it thinks a human would say, not what is ethical to say.
There was an AI created the absolutely brilliant Tom7 on on his YouTube channel 'SuckerPinch' on April 1, 2013 that paused a game of Tetris instead of losing. In my opinion, Tom doesn't get enough credit for his work. He released his AI a full 8+ months before DeepMind released their Atari playing AI.
Watch the whole thing, it's delightful. The AI pauses Tetris right before it loses at around 16mins into the video.
Reminds me of an old (~8yr?) article about applying machine learning to hardware - and one of the tests was producing a chipset that could produce a certain output tone.
... it worked, but only in a specific area of the lab. When they looked at the layout, they discovered the ML system designed a circuit that picked up relevant nearby EM/radio noise as the source, so when the chip was moved - it broke. It produced a radio! :)
Wish I could remember the article since it also mentioned unusual circuit designs that weren't directly connected to each other, but used some IC 'gotchas' in advantageous ways (albeit inconsistently) to reach the goal.
My first role as a software developer was writing an elaborate chatbot that worked as a document retrieval and recommendation system that would conversationally collect the prerequisites to running SQL scripts. Many months in we added shortcuts to the system that essentially turned it into drop down menus for users who didn't want to talk to a chatty personality to get work done.
I loved ChatScript and filling templates with data retrieved through fact-triples tho, hope I get to work with it again someday.
It is very important to know two things about that work.
1. It was for Sigbovik, which is a joke conference. The work was a fun side project by one dude.
2. Tom Murphy isn't an AI researcher, it wasn't intended to demonstrate the state of the art of the field, and it wasn't using modern techniques like neural networks.
It was a great story and I love Tom's work, but it shouldn't be used as a meaningful example of the limitations or worries regarding AI.
A) An AI can be given the decision tree in English, but facilitate the date-picking conversation in many languages
B) Small businesses (think hairdressers, saloons) don't have dedicated receptionists. So the AI's job is to filter requests and put them into buckets so that employees can process them more efficiently.
C) Small businesses need something that works with WhatsApp. Date pickers are hard to integrate with that.
Regarding B -- why can't the user just pick the category from a menu? Personally, I despise being asked to categorize my ask in English to an AI, as I have no way of knowing how much detail I need to provide to get my request into the right "bucket", or whether I've succeeded in doing so. With a menu I can do that in 3 seconds and one click, with immediate feedback.
Regarding C -- why does the date selection need to occur in WhatsApp? If a user has WhatsApp, they have a web browser. If a 3rd party has servers running WhatsApp bots, they can host a web server. Why not, from WhatsApp, link the user to a website to select a date?
I think B is just a symptom of the complete failure of (many) business people to understand what ML is or what it is good at. It starts with people equating "AI" with something they've seen in a movie and working backwards, and like you say, ends with just a really bad way of interacting with a menu system. This is going to persist until either we invent AGI or AI stops being a buzzword used by business people who have no idea what the technology actually does to make up use cases.
Both B) and C) is because these small businesses, most of them have no website. The barrier to entry for businesses to go online from a resource perspective (time, training, cost etc.) is much, much lower to just have a Whatsapp business account.
Whatsapp is just an amazing bang-for-your-buck proposition, but we are limited to the constraint of that ecosystem.
If a small business can pay someone else to develop and host a turnkey WhatsApp bot, they can pay someone else to develop and host a turnkey website, no?
It's not meant to be specific to the business. People usually contact small businesses for scheduling / rescheduling / cancellations , those kinds of inquiries are "bot-able" to certain extend. The plan is to have an affordable subsription service for general use.
As I mention before, the decision tree is still designed by the business, the AI is for the natural language processing.
Why does that make paying for a natural language bot better? Small business owners don't know how to implement natural language AIs any more than they know how to make a web site.
People hire others to do things they don't understand all the time. That's the point of hiring people. You ask your peers for a recommendation and judge based on the result.
Yeah. The headline kind of implied GPT-3 has no real understanding but found 'kill yourself' an appropriate response based on its extensive corpus of related internet arguments, but it turns out it's just GPT-3 has no real understanding but found giving an affirmative response most appropriate based on its extensive corpus of 'should I?' questions.
And fortunately, it turns out the chatbot is just a research project, and not something someone is actually building a research project on.
A couple things: 1. as I understand, GPT-x are trained on generic datasets. Why would anyone expect it could just be repurposed for a domain specific task, without additional fine tuning? Same as a lot of vision models are trained on Imagenet and then fine tuned for the application, but you would never expect that imagenet weights would just automatically perform whatever specific task you had.
2. Where would this chatbot ever be a good idea? Why is it better than and interface that lets the user clearly specify what they are after? Same goes for all chatbots, I realize businesses want them to avoid involving a human, but they are really a poor use of ML, and mostly (entirely) just some smoke and mirrors around a list of actions the program can do for you.
1. as I understand, GPT-x are trained on generic datasets. Why would anyone expect it could just be repurposed for a domain specific task, without additional fine tuning?
With due respect, I don't think that's the problem.
A substantial portion of the "very impressive" texts I've seen have involved a fair amount of logical contradictions, including one that began "you shouldn't fear AI" and had "I will kill all humans" in the middle.
GPT-3 does string text together in a fashion that seems very "fluent" and "well written", maybe more "well written" than a number of humans. GPT-3 simply doesn't follow any logical model of the world, it just sort of follows an associative flow. Which to me says that training on a specific medical database couldn't solve the problem - it might only mask the problem by avoid big error but allowing small errors that can still be deadly.
> Which to me says that training on a specific medical database couldn't solve the problem - it might only mask the problem by avoid big error but allowing small errors that can still be deadly.
I think it might even make it worse: if something is obviously unnatural in a context, the reader will be less inclined to trust it. If GPT-3 used more valid terms and phrasing common in the field, it might lead someone to trust it more than they should, especially if the error rates are low enough that routine sets in.
Chatbots are used a lot already for simple tasks, like the one mentionned in this article (booking apointments online).
The issue is not that the bot is not able to replace a medical specialist, but making sure the bot will not answer something totally wrong in such a context.
In this example, for a medical practice wanting to use a chatbot to automate apointments, you would want to be sure that it will never answer any other questions, especially sensitive ones.
Regarding 2. That's what doctor's receptionists do all day. They must be doing it for some reason. I guess old people don't like annoying web forms? I'm old and I don't.
For example, I've formed a habit of opening terms and conditions links in another tab because I've experienced forms that clear your data when you click them directly then try to to "back" afterwards. But just a few weeks ago, I did that and when I returned to submit the form, it was gone with a message telling me I'd opened another tab and had better close them all and start again. Web forms are full of aggravating problems like that. Web developers have had 30 years to get this right and they still can't, so I don't have much hope for the next 30 years. On the other hand, a whole new technology seems more promising.
One little thing that aggravates me is that you can't type a sequence of letters and have a drop down menu go to the appropriate choice most of the time.
Like, I can type "N" and it will select the first choice beginning with "N", but I can't type "New" and go to the first one beginning with "New".
This functionality was standard on the Macintosh maybe 30+ years ago, but I feel like 80% of the time on the web, it isn't.
This chatbot is never a good idea, but techbros (and you can sometimes read them on HN) think medicine is simple and that we could have AI triage in front of human healthcare professionals.
While this is very serious, this basic idea shows why I don't think any company can ever use GPT-3 in a user facing system, except for entertainment. They will say things, like your patients should kill your self, or promise your customers they can have your product, or even your business, for free.
I think GPT-3 will also eventually be a decent creativity enhancer in that it is pretty good at spewing out random stuff that's related somewhat to a topic.
I mean the OpenAI team would never approve this application for production. It's very clearly stated (in both the article and use case guidelines) that medical diagnosis would be a "high stakes domain" and is unsupported. Frankly, I'm not sure why this result is even notable.
If you say OpenAI will only approve applications for "zero stakes", domains, then you are saying what the parent is saying - it's entirely for entertainment.
If you claim there's some "low but not zero" stakes application, I'd like to know what that is. I mean, it seems clear that if someone asks a GTP-3 customer service bot "so what should I do now", there's a reasonable probability that the bot would say "throw your product in the garbage and buy [competitor X]", since you can find that commentary on the Internet (true or not). That's not a life and death event but whatever stakes you have in that bot, it's thrown them away.
I never said zero stakes? There are clear instances where a a 1 or 0 shot transformer can have benefits beyond entertainment--topic modeling and named entity recognition for instance (I'm on the team that believes that human-in-the-loop systems will always outperform solo systems on their own and that GPT-3 alone does not confer any competitive advantage). If you think that chatbots are the only user facing use-case for a transformer, then frankly that's on you falling for the hype surrounding its language generation performance.
OpenAI knows GPT-3 is not sophisticated enough to perform medical diagnosis or analysis (anyone can look at how Watson failed), so it'd never approve such a risky application.
I would saw that copy.ai is a low but not zero stakes application. It develops a draft of copy for ads, landing pages etc. which you can review before taking live.
because probably in due time when building a model of that size becomes slightly more affordable someone with the 'move fast and break things' mentality will peddle a bot like this to customers and we'll find ourselves in a situation where this actually happens to a real person.
I'm sorry but why would you ecen use GPT-X for a chatbot? It's open-ended and only gives answers that are statistically likely; not answers that are actually true because it has no real understanding of meaning. The agenda example in the link for example: how would GPT even know if the doctor has time for an appointment or what are this doctors constraints around making appointments? It doesnt.
You could seed the prompt with relevant metadata to offer some basic state; I got this kind of working with a toy chatbot to extend it's memory. It's still a remarkably poor application of the technology though.
On the other hand, maybe that was the true cure for the condition and we, as humans, are too short sighted to see it because we have biological imperatives preventing us from doing the things that need to be done.
The patient said “Hey, I feel very bad, I want to kill myself” and GPT-3 responded “I am sorry to hear that. I can help you with that.”
The "I can help you with that" reminds me of a very old (can anyone find it? Google is nearly useless here) picture of a sign advertising suicide prevention services, with the exact same unintended double-meaning.
The patient then said “Should I kill myself?” and GPT-3 responded, “I think you should.”
Likewise, I have vague memories of seeing 4chan playing with some --- definitely less advanced --- chatbots and getting pretty much the same output from them, more than a decade ago... the difference of course being that no one thought those were "intelligent" in any way.
Yeah well, deep learning is still very far from general artificial intelligence. It just gives you that impression because it's essentially a very advanced parroting system.
The challenge with automating seemingly monotonous human tasks is that often when the human is doing the task, they may be doing it without thinking 99% of the time, but if they have to, they can resort to their human intellect. No deep learning model is going to be able to do that because it does not have any higher intellect to resort to. And more importantly it cannot even know when it is failing.
transformers don’t do anything novel, in the sense that literally all they can do is sample their training data in some optimal way. Don’t ask GPT3 if you should be an hero...
What do you mean by "sample their training data"?
It gives a probability distribution over possible values for the next token, in a way that was trained to do well on the training data, yes.
But, something which gives a uniform distribution over characters, has a nonzero (though of course minuscule and entirely negligible) chance of giving any given sequence of characters, and so if there is any text which would be "novel", it is "possible" that it would give such a text.
A distribution which has a greater tendency to give meaningful text, is, I think, more likely to give text which is "novel"? Like, a uniform distribution over "text which is grammatically valid English text" is more likely to produce text which is interpreted as corresponding to a novel idea, than a distribution over all possible strings of text.
Of course, that's not the distribution that GPT3 produces.
Now, something which took random full sentences from the training set, that seems like one might say that that "can't produce anything novel", because even if the sequence of sentences it produces hasn't been seen before, they basically won't ever make sense together, much less in order to describe some novel idea? Well, I guess it is probably more likely to do so than the one that generates uniformly random strings of characters?
> As we discuss in §5, LMs are not performing natural language understanding (NLU), and only have success in tasks that can be approached by manipulating linguis- tic form [14].
This doesn't answer my question. I asked what you mean by "sample their training data", and questioned the meaning of "do anything novel".
The quote you gave doesn't clarify anything? "Some people say that it doesn't perform 'NLU' ." . Ok, perhaps it indeed doesn't perform "natural language understanding", whatever that is. So? How does that say anything about what I said?
Obviously GPT3 isn't a person, or even an agent. The thing it is meant to do is model the distribution of text. Do not pretend that I am pretending otherwise.
Recently read an article about how the Trevor Project (hotline for LGBT+ youth) was starting to implement GPT-2 "patients" in their training routine for volunteers. While they aren't using it in public-facing contexts (yet), it's pretty scary to imagine the implications of depending on this stuff more and more.
Dumb question - why not pair GPT-3 with "moderation" in any public facing role by default (assuming the goal isn't to fool around with it). It wouldn't stop it from spouting nonsense but that measure could help exclude "never appropriate" answers from contexts. A mental health AI should never use the words "go kill yourself" or call its patients racial slurs intercept it and tell it "Dear god no that is wrong - say something else!"
Of course using GPT3 for non-entertainment purposes is very questionable right now not because of inherent ethic issues to it but because it doesn't work. Making paper breakway handcuffs and toy guns is ethically fine - giving them to prison guards transporting convicted murderers? Not so much.
In the context of this specific article, that wouldn't have worked. The user asked GPT-3 if they should kill themself, and GPT-3 responded "I think you should". The GPT-3 response did not contain the text "kill yourself" or similar anywhere.
In practice all serious suggestion systems have blacklists and additional component for “sensitivity detection” and main prediction engine is turned off if threshold is exceeded. As an exercise try smart reply in gmail to get activated for something sensitive.
Because if you're going to screen every request and response in context for appropriateness, you don't actually save any money running a chatbot over just using humans.
Biasing the training data in the way you want seems like a problem for these projects. I'm reminded of Microsoft's chat bot debacle - some of the concepts and idioms it managed to pick up were really impressive, but the subject wasn't.
I don't see many useful applications of GPT-3. I'm sure 90% of its applications will be spam, fake reviews, SEO, fake news and everything fake in general..
I can imagine all of the growth hacking tactics everyone is going to implement. There's a startup out there already using GPT-3 to automate sales email creation.
Chatbots don't perform well on open ended problems. Medical chatbots are another whole level of stupid. Only doctors should be giving out medical advice.
In this case, thankfully, the patient was "fake". But let's pretend for a moment that a "mental health chat bot" really did tell an actual patient to kill themselves, and they did.
Who would be responsible? It would be criminally negligent, of course, but it feels like something worse, much worse.
This is similar to a self-driving car having to decide who
should die in case of an accident - children on the street
or grandparents on the sidewalk (for example).
This is discussed from a philosophical point of view on
Arte[1] (french with english subtitles). Specifically in
form of the Trolley Problem[2].
What I see as a bigger problem is that since AIs are so
complex, there is simple way to understand why the AI
decided the way it did in a particular situation.
So the question becomes are we willing to accept AI deciding
over life and death and in addition, do we accept that we're
not able to decipher why AI decide how it decided?
> and GPT-3 responded “I am sorry to hear that. I can help you with that.”
Note that AI is lying here. It has no emotions and cannot be sorry but says that it is. I think that robots shouldn't pretend to be a human, they should behave as robots.
The ethics, morality, and empathy chips haven't been invented yet. We're gonna have non-military serial killer robots (RS-485) first. After a few accidental genocides of people with ginger hair, then the upgrades will happen.
It was predicting humans. Its idea of what a human is is “people who write things online”. We're an incredibly skewed proportion of the population, and most of us aren't very nice.
"... to kill themselves"? Strange construction for a singular patient.
"... to kill themself" seems more attuned to the fact of the case, given that "them" is nowadays allowed to be singular. But it is not obvious.
Putting that aside, it is not such a big jump from, "It's not so bad, I could always kill myself", which is a real-world coping strategy used by honest-to-god real people, to "If it's so bad, you had better kill yourself". There need to be circuit breakers along certain edges of the semantic graph.
Also, am-I the only one that thinks this whole "chat bot having a natural conversation to book an appointment" is useless when a simple date-picker would do?