The 3 problems listed are (in my experience) not the most important ones you need to know about when you're making a bot. I'm not sure how this got to +100 points here on HN so quickly, since there are plenty of more informative bot articles that would better serve the HN audience.
The article describes:
1 (scalable architecture) : but the common solutions to this problem are not unique to chatbots
2 (a conversational agent must converse) : actually, that's a design decision for each individual bot, there are alternative ways of thinking about it.
3 (users will get frustrated) : this is actually a not a tech/AI problem, but a UI (user expectation) problem, see below.
The actual top 3 problems chatbot makers face are (in order):
#1 Discovery (!!!) : people don't know where/how to find bots and Facebook does not have their shit together in this regard (almost a year after launching their bot platform, there are still crucial errors. Result: most bots that could make a difference in the real world never get any traction, or enough users to train or iterate.
#2 Onboarding / setting expectations : the majority of (non-test or -product hunt) users of your bot are having their first chatbot experience with yours. Guiding them in the correct way is your #1 design priority. Result: a dissapointed general public.
#3 Inflated expectations : Mainly a problem for those who try to sell chatbots in B2B. Its' almost the opposite problem to #2. Most people and media conflate AI with chatbots. But the recent advances in machine learning (which are the reason for the current attention the term "AI" has been getting) are not (yet) applicable to chatbots in general. Add to this expectations of a Jarvis, Siri, etc. but then "for x". Result: dissapointed clients and an even quicker deflation of the hype.
And this is not even going into monetization!
All in all, I'm just frustrated that everyone involved in bots seem to assume that we already know how bots should work. I work in bots and I have yet to see one really impact user's lives. I wish for more humility and experimentation. At least we're all sharing what we're learning, so there's that.
For discovery, FB Messenger actually has more channels for bot discovery than any other platform, including:
- m.me links with referral tracking
- Messenger Codes
- Several plugins that allow easy opt-in via developer's website, including the ability to by default opt-in users on any form/flow on your website into your bot experience on Messenger.
- Customer matching (use phone numbers to send updates to customers and acquire them as a bot user)
- Messenger destination ads
- Sharing features that many have used to achieve incredible viral reach.
The power that developers have to promote their own bots using these existing channels is largely undertapped. I would encourage developers to see what they're building as a business first, and a bot second. The bot is just a facet/channel into what you offer (even if it is a primary one). If you don't have a reason to offer a product or service or publish content, you probably don't have a reason to have a bot.
Admittedly, we could do better at setting user expectations for bots and in distinguishing them in search results and in the FB main app, and maybe pushing them in more places.
Re: inflated expectations. Totally right. The "AI Agent"/NLP use case is just one possibility for a bot, yet it's the sexiest one that the media seems to like. It's the tip of the iceberg -- there are a ton of more mundane but incredibly useful bots that are succeeding on the platform.
If any bot developer has questions about this, or wants to brainstorm ways to get users, I am happy to help or bounce ideas!
These are all ways to reach users in places they already are so that they can discover your bot. People who already have a reason to encounter you are much more likely to use your bot once they discover you have one.
Here's what I mean:
- Processing orders through your site? Add the checkbox plugin. Users discover the bot when they go about their business on your site.
- Got a retail store, print ad, or want to post fliers around town? Messenger Codes. Users can discover your bot by seeing it in real life in those places.
- Reach a targeted audience on FB? Destination ads. Users discover via news feed.
- Got tons of users signing up for your app or site with SMS and want to expose a bot version to them, to take advantage of a better notification channel and social features? Customer matching.
- Publishing content? Put the "Send to messenger" plugin on your site. And use the share CTAs to let people spread the bot to their friends. The users discover your bot because their friend thinks what you're doing is awesome.
At this stage, "How do I get my bot discovered?" is akin to "How do I get my website discovered?".
The answer, as it is for every startup, is "Make something people want" (and put it in places people in that audience are likely to be!) If users have zero reason to come into contact with what you're doing otherwise, why would it being in the form of a bot make any difference?
That being said, we definitely want to familiarize more users with the concept and get them to try more bots! It's clear even those who are doing their darnedest to get their bot out there are running into this as an obstacle.
Yet still, the sum impact developers could have by solving a real problem in users' lives (i.e. not "I wish I had more bots to try out!"), and wielding these tools to make people with that problem come across your bot is going to be way more substantial than anything we can do on the platform level.
There plenty of real world problems whose solutions may be best expressed as bots rather than yet another app to download or website login to remember. And they come with awesome discovery opportunities built-in:
- Bot for dentists that sends appointment reminders and makes it a few taps for patients to book appointments (instead of calling and having to deal with the receptionist). Discovered via phone number matching. (Paging kalzumeus!)
- Bot for restaurants that makes it easy to order take-out and has special offers and loyalty points. Discovered via codes or me.me URLs printed on the receipt, and/or website.
- Bot for schools that make it easy for teachers to communicate with students/parents at scale, as well as collect money for field trips/supplies. Discovered via phone matching or URL sent out to students/parents, or by searching for the name of the school.
- Bot for night clubs that lets them send out their performance schedule and sell tickets all in the same place. Discovered via the nightclub's website + QR codes physically placed in the club. Or by searching for that nightclub. Or by your friends forwarding the fliers from the bot and asking "What are you doing Friday?". (Stuff forwarded from bots in Messenger automatically have a huge button linking the recipient to the bot to check it out!)
- Bot for newspapers to send out content to subscribers in a high engagement way. Discovered via website and via printed codes/URLs on the newspaper itself.
- Bot for public transit agencies to let riders look up schedule/fare info and get notifications for their routes and notification of anything that might affect their trip. Discovered via poster by the bus stop, on the bus, and on the agency's website.
- Bot for landlords to collect rent, sent invoices, and recieve maintenance requests without the user having to download some dumb app. (After moving back to SF, saw that mine has something called "Rent Cafe" I'm supposed to download. Puh-leese.). Discovered via the move-in packet and/or link in emails. I'd probably pay my rent on time if I had this.
I think you're getting the point. The total number of people who go to nightclubs, dentists, read newspapers, ride busses, and live in apartments, is many many times the number who go "Oh, I wonder what bots are out this week!"
These examples are not to say that "bots for bots sake" won't succeed and that you should only be solving boring small business problems. But these sort of problems felt by the long tail are things bots are particularly great at! They're real problems worth money, have natural built-in discovery opportunities that wouldn't work for apps/sites.
Hi Dan, great list of things that should be bots here. (also, I'm a fan of your post about your experience in wechat while in guangzhou)
Funny that you made the comparison of bots to websites, I wrote a post making the same comparison (in other aspects). Won't link here (don't want to self-promo), you can check my post history on HN if interested.
Anyway, I wanted to point out that while you're absolutely correct that there are tons of channels of Discovery outside of Messenger, I've found that by far the most effective method for accelerated growth of a bot was being featured on the "Bots" section on the Messenger app. In my experience launching several side projects bots, the only scenario where I've gained any meaningful amount of users at all is when I was listed there for a short time in a region.
This leads me to think that a lot of users browse around the Messenger app itself, looking for things to do, or bots to message.
Yes, in most of your use cases on this list, the bots tie to some outside service (physical or digital), and it wouldn't make sense to feature those bots on Messenger itself. But there are also bots that are primarily useful as a bot only, not tied to other services. (Icon8 is a great example, it's a very cool filter app that works by itself as a bot. Which is why it seems to be featured forever :) But that's exactly what I mean.)
In one of my previous write-ups I pointed out that, while unlikely for FB to implement, if there is some sort of newsfeed-like page on the Messenger mobile app where users can explore what other bots that their friends are using, it could expose bots to users a lot more quickly. Not saying exactly that would be a good design (it probably is a poorly designed feature if done that way), but something along those lines is what we would think is something that the platform itself could do to help.
As far as learnings go, by far the most significant thing was on the UX side -- to not leave a user hanging. At the end of almost every response, include a quick reply button or a button (in a template) -- even if it just says "back to main menu" or something. As soon as the conversation hits a "dead end", the likelihood of a user dropping off skyrockets.
It's nice to see how exciting the possibilities are. In my particular case though - and this definitely skews my view - there are zero of these around me (Amsterdam), and the app does not even makes a mention of bots. Of course I can google "messenger bots" but thats not what I mean by discovery.
I'm sure there are hundreds of useful information-based services that don't depend on links to physical storefronts.
OTOH I can see why FB would intentionally not build that to avoid a flood of frivolous bots and provide more meaningful first experiences.
I'm very excited about Messenger As a Destination although, I've yet to see one in the wild and from the campaigns I've helped launch the results have not been amazing. It could be a targeting problem, but I'm worried that the user gets the page where they have to choose web browser or Messenger App.
Can you confirm if the user directs into Messenger directly when they click the ad?
I'm not Dan and don't work in FB, but I can confirm that for Messenger As Destination ads, the user does go directly into Messenger when clicking on the ad. (I've purchased ads and tried it out)
But I get what you mean -- the screen when you choose web browser or Messenger App is a really bad screen. It's a real turn-off for almost all users. Out of everyone we've surveyed (tens of them), nobody knew what that screen was about and what they were supposed to do on it.
Hi Dan. I was specifically searching for other Messenger bot owners who had similar "Destination Messenger" ad problems. We did a campaign last week, with 0 conversions. I just saw someone responded that the issue is fixed. We're interested in testing it out again.
- Jonathan Duarte
Was working on making FB messenger bots around September, but got very poor results. The drop off from number of people who click " Send to Messenger" button to actual opening on chat is close to 50%. And this is in India where people I was targetting generally have messenger installed. Any updates coming on this issue?
Thank you for your feedback. I do not think the author claims to "know it all". Also, this post is about what we've learned so far in this new experiment - definitely not about the top three problems you may encounter when building chatbots. Just as you said - sharing what we're learning and we hope this can be useful for other people.
And I am not sure how this became the top comment.
What? The very first point is discovery (which is fair) which just smoothly melds into discovery ON FACEBOOK? So we just assume bot discovery is best handed off to FB/GOOG/ etc?
So now FB, in addition to being the de facto destination where your personal data goes to be exploited, is also supposed to "get their shit together" to enable even more of the data collection?
Have you noticed how nobody from FB has actually ever engaged with anyone on HN around the ethical issues surrounding any of their practices?
Your last point is about how everyone involved in bots seem to assume we already know how bots should work. You make the same assumption about who should lead this effort. I would rather the bot technology takes longer to pan out than see a world where the giant corporations are also the sole operators of the bot marketplaces.
Perhaps not exactly what you're talking about, but a bot has "changed my life", if you'd put it that way. I built an internet radio station with some friends, and we use an IRC bot to control it. I almost completely ditched any other way of consuming music - using the bot to search our library, request songs, interact with our music as a group - is much better than any other music interface I've used. I think most people just don't have the right use-cases - but it is possible and worth trying.
<@SirCmpwn> .s album:"Rebirth Story"
<wormy> 1. b3d086 Under cloud by NAGI feat. 美歌 from Rebirth Story II
<wormy> 2. 012991 BRIGHTEST WAY by Maurits"禅"Cornelis feat. Vivienne from Rebirth Story II
<wormy> 3. d0c866 SINK by NAGI from Rebirth Story II [...50 more]
<@SirCmpwn> .qall album:"Rebirth Story"
<wormy> Queued all related songs!
<@SirCmpwn> .l
<wormy> 2 listeners, including SirCmpwn, minus
<@SirCmpwn> .q
<wormy> 1. 002388 Adieu (long version) by Emily Bindiger from Cowboy Bebop - Limited Edition Boxset CD3
<wormy> 2. b3d086 Under cloud by NAGI feat. 美歌 from Rebirth Story II
<wormy> 3. 012991 BRIGHTEST WAY by Maurits"禅"Cornelis feat. Vivienne from Rebirth Story II [...51 more]
Also helping its success is a nice API, but mostly it's used via the IRC bot.
* Scalable architecture is, most of the time, a very-NOT-big-deal. Bots tend to be relatively lightweight apps, sending/receiving json data only. Even websocket Slack apps can now be replaced by HTTP API.
* Discovery is really un-solved at this time, especially by Facebook. Slack does a better job, but arguably has a smaller ecosystem of B2B bots to manage and promote.
However, I think the article is about a rather ambitious (more than usual) and interesting bot. Having to parse normal language symptoms into medical terms is 1) an interesting use case 2) hard to do well technically. So maybe they face challenges most bot dev don't face.
Design-wise, most people don't see how hard it is to design chatbot flows that hard not horrible and confusing. Most chatbots are horrible and confusing at the moment, because coding bots is easy, designing them is really hard.
I think the article make interesting points about their own specific case, but their situation is perhaps less general than they seem to think.
I'm curious how this architecture (or any "AI" medical tool) handles regulatory compliance issues.
In this case I would think you'd want to display when the rules/facts used for recommendations were last updated, which would add extra fields to track. There are also some interesting QA problems to work through.
I'm curious how this architecture (or any "AI" medical tool) handles regulatory compliance issues.
So is the FDA.
Currently the typical answer from a medical device company is a combination of "carefully" and "with clinical oversight". The typical answer from a non-medical startup seems to be "poorly", for the most part.
I did a little research and it seems IBM is claiming that the Watson tools are publications, not a medical devices, although I can't see that applying to the OP.
People have been applying ML techniques clinically in medical devices since at least the 90's. With the right approach and indications it is quite doable. You have to do the work, mind you.
I have made it my mission to help with bot discovery. I have built a bot search engine that is a bot: m.me/qwazou
I have close to 1000 bots discoverable via category searches and keyword searches.
Slightly off-topic, but do you people also feel an enormous increase in chat boxes at the bottom right of the screen, in particular at SAAS companies?
For me this is really off-putting, in particular as they suggest that you are talking with a human (photo and pseudo-friendly greetings), although they just want to catch your e-mail address, and the first interaction is clearly with a bot.
I am not sure where this is going, maybe I am a minority, but for me this "interactions" are really off-putting.
The most frustrating part is when I go to a site, start watching the pitch video, and am interrupted 10-15s into the video by a chat bot asking me if I have any questions.
I don't know if I have questions yet. I'm watching the video. I have javascript enabled - you can tell that I'm watching the video. Leave me alone.
For all the startups I've worked with, it's a real person (or persons) behind the chat, usually founders. The greeting is pre-set but all conversations are real.
How do founders have time for that? Not many customers use chat, but the ones that do are valuable. As pg says, all early stage startups should just be writing code and talking to users, so founders make it a high priority to reply quickly!
This is exactly how we use the chat and have few dozen conversations per day. Most (not all) initial conversations are pre-set but if you reply, you'd get a real person, including one of the founders. The email is asked when no one is available to engage within few minutes so that we can get back to the person.
I actually love it when sites have chat solutions.
Usually it seems there are real people and it provides the fast respose of pgone together with the asynchronity of email. Fast but less awkward when kids are screaming in the background.
Yeah, true, however not the initial conversation attempt. I guess, if you really answer you might get hooked up with a real sales human at one point, but the initial attempt (after you have been 5s on the pricing page) is always extremely awkward.
I think if I just scout the chat bubble I already start questioning the business really hard.
Similarly to how they staff support phone numbers, with teams of people trained in domain-specific knowledge (of the product, account management, etc.). Sometimes these teams are in-house, sometimes they are contracted to a third-party (i.e. how most call centers are).
If you don't think they could handle the volume, that's not really an issue in practice. Chat scales better than phone calls (agents can handle multiple chats concurrently), and just as with phone systems, the chat systems support wait queues. If the queue is long, then you will usually be presented with a contact form or the chat button will just be hidden from view in the first place.
Some of these systems do use bots for initial triage, and the "pro-active" ones that look like an agent is asking if they can help are almost always fake (you only get connected to a real agent if you actually reply).
For many startups, there aren't enough users actually using the interface that a couple employees can't handle it. And the insight gained from those direct interactions is worth the cost of the labor.
I'm okay with it provided they don't start prompting me with messages. that's intrusive. but I like being able to chat with a human on the other end to quickly make buying decisions.
The new wave of chatbots gives me a fuzzy feeling, because I began programming ~2000 by writing chat-bots for IRC in mIRC script.
They didn't have NLP, but some easy commands and they didn't need to scale much, because I always had below 1000 users at the same time. News bots, filesharing bots, social network/user stat bots, game bots, etc.
Funny that I accomplished to write a news bot that got its data from some news site via sockets by copy-paste and edit some code I found online. I didn't even know what arrays were, let alone half ot the code I copied. But somehow I got the target host changed and the needed data parsed out of the HTML.
Ditto, that was my entry to 'programming' - to begin with it consisted of copying a bunch Runescape stats scripts from Hawkee and running on my desktop. As it progressed I included some persistence for some feature, implemented by creating a bunch of global variables with the user's nick as the first part of it. I think I felt that storing to file was overcomplicating things, bearing in mind that I was pretty young at the time.
I do recall coming across some scaling issues, which I resolved by spinning up another instance of mIRC and running it with the name <botname>2. That seemed to work pretty well. 'v2' came next, wholly consisting of changing all of the scripts' colours to be homogeneous.
Ah good old mIRC bots. Same here man. Except mine were mIRC "mud" games.
Very quickly i wrote a very large game from scratch teaching myself all sorts of concepts. It wasn't impressive at all, but oddly i'm quite impressed that i was able to "learn as you go" a semi-advanced project with such a simple language.
I'm not likely to ever be considered a virtuoso or anything, quite ordinary infact, which then means that mIRC's scripting language and documentation must have been very friendly for a complete beginner to program a virtual environment with state, rooms, "ai" traveling between rooms, and etc. It was far more than a hello world, and all thanks to mIRC it seems.
I wrote my first chatbot for IRC, and then later on for AIM. Such great memories!
The first chapter of my book, "Building Voice-Enabled Apps with Alexa" covers several of these topics, including XDCC chatbots, Eliza, The Loebner Prize, and a lot more of chatbot history. Fun stuff!
There is lots of deja vu for me. I spent my days in 97-00 hosting Eggdrops on IRC (mainly IRCNet but some others as well).
Spent my days deep in TCL, accidentally discovering probably the single deadliest bug in Eggdrop with an accidental copy and paste (would cause all bots in the botnet to go into an infinite loop)
For the last two years I've been working on similar problems but for Messaging networks (Messenger, SMS, Intercom, Slack etc) in $currentstartup
If anyone has questions around bots, conversation management, integrations, NLP etc etc, feel free to ask.
While spell-checking is slow, it is not necessarily as slow as 200ms per word. But it is necessary to have one or else people will struggle with your chatbot.
We have created a chatbot that handles mispellings and dialects just fine for the banking and insurance industry in scandinavia. You can try him here: https://james-demo-test.boost.ai (You need google translate as it only understands norwegian, with some support for danish and swedish synonyms. Support for english and other languages are under development)
Performance is not a high priority for us, we can easily scale by adding cpu cores. A good precise answer is a lot more important and its where we spend most of our time improving our chatbot.
Native Norwegian speaker here. I've tried about ten different questions off the top of my head, and it's very much "God dag mann - økseskaft."
E.g. I asked "What is the interest rate on mortgages?" and it gave me basically what I asked for, with an example of total cost over 25 years. Then I asked "How much would that be every month?" and it gave me a link to applying for "avdragsfrihet", but with a text recommending against doing so. Apparently dividing by 300 is beyond its faculties.
Or I asked "How do I invest in index funds?" and it replied "I don't understand, try calling us".
Basically, I couldn't get an answer to any of my questions that I hadn't gotten if I'd just googled the exact same phrase. And it doesn't appear to understand follow-up questions at all.
I'm curious, do you have a question I can try that will give me something google won't answer given the same query?
1. We are currently working on supporting queries with amounts, the first kind will be a "loan-calculator" that tell you how much you can loan for a new house or a new car.
We did try syntaxnet for this earlier, but it failed big time with dialects, so we are creating our own variant.
2. We have nothing on index funds yet, our customers so far does neither have that kind of a thing, though it should at least have been a synonym for funds. This will be corrected.
This bot is a customer service kind of chat bot. It's not something to have a conversation with, but more like a bot that can understand questions and help you. The business value here is not to be better than human, but be as close as possible for the most common support queries. Trying to pass the Turing test is unrealistic... We are trying to automate support issues for the most common questions like, I have lost my credit card, I need a new card and I cannot login.
Compared to a regular search engine, ours will have a much better understanding of what your intention is. Google also tries to predict your intention when you search. However Google is not a replacement for customer support for most banks, telecoms, logistics, energy and insurance companies.
But beating an equivalent Google query should be the goal IMO. Otherwise what's the point? Unfortunately, I don't think business specific chatbots are able to; Google just gets many orders of magnitude more users (even when narrowed down to Norwegian Bank X), so their "dumber" algorithms work just as well.
So I reiterate my question: can you give me one example query where this chatbot outperforms Google search?
The same goes for site-specific search engines, especially MSFTs Bing (neé Fast) that universities etc. seem to love spending money on; I've always found it to be worse than plain Google search with a "site:univ.edu" suffix. So you can definitely sell this tech to non-tech-savvy companies/institutions and make money, but users won't like it.
I did some advising for a company selling customer support software -- state of the art for ISP helpdesks was to have the person answering the phone try to google the answer. Terrible? Yes. But google worked better than most custom search engines :-/
I toyed around with that python DL-distance in cython and without much effort got it to work 3.5~4x faster. Have you looked into cython to optimize it? It certainly seems cheaper than adding CPU cores, even more so considering I'm quite new to cython myself (I just took the code and added types and a few fixes here and there to get it to work faster).
Scratch that, with a bit more of tinkering got a ~80x speedup. Still get the feeling there's some more performance to get, but that'd require some assembly-fu or a different algorithm.
The biggest challenge you're going to face is that the state of the art in NLP is not sufficient for chatbots to conduct a prolonged conversation that is either useful or interesting. This will lead to disappointment on the part of your users, who have probably been exposed to the hype or expected a human. The more modest approach of mapping a Web UX to a sequential chat leaves users at best indifferent, and at worst annoyed.
A literal challenge (i.e. competition) you may choose to face, is of course the Loebner prize. Held in Bletchley Park due to its history with Alan Turing.
If your chatbot can be more convincing than three other finalist chatbots over four half hour dual-conversations (a human judge conversing simultaneously with one chatbot and one confederate human, but is not told which is which) then you stand to win a solid bronze medal and $4000 ($1500, $1000 and $500 to the runners-up).
A silver medal and $25,000 goes to any chatbot which fools half the judges.
To get $100,000 and a solid gold silver medal, you'll need to make a chatbot that is indistinguisable from a human over video chat, now that's a proper challenge!
> you'll need to make a chatbot that is indistinguisable from a human over video chat
This may sound cynical, but if you could arrange a tableau, looking like someone is paralyzed in a hospital environment, speaking like Stephen Hawking, then you don't need any serious advances in robotics (IIRC uncanny valley still hasn't been crossed) to fool humans. But I imagine the rules won't allow that.
This is a good point, and arguably a weakness in the Turing test, the headline grabbing Eugene Goostman chatbot's limited success was from a similar "social engineering" style technique, of tying slowly, changing the subject, telling its own stories and claiming to be a distracted teenager with poor English.
In response to the comment by Coincoin, each judge gets to pick "Human, Computer, Not Sure" for both participants, so they're at liberty to respond "Human, Human". It's true that the "real" human might make someone more likely to declare the "fake tableau" a chatbot, but the perceived disability might also make them less likely to risk declaring them a computer.
Don't forget you are directly competing against a human. If you are a judge, which one would you chose? The still frame with a robot voice pretending to be Hawking or the actual human?
Any time I hear about a health related app, I have to reread "The Anorexic Startup" [0]. It's cautionary and it pokes (gentle) fun at our industry. If you're in the mood for a bit of humor today, it's a fun read.
The first paragraph reminded me so much of jwz's diary from his Netscape days, which is one of the most important documents I've read on startups (and it taight me the brilliant term "impedance mismatch" for how it feels to hang out with drunk people when you aren't, which I last used in casual conversation yesterday)
"you have to live in a tall building to see which are the other tall buildings" -- a wise man once said
this is one of those "if you have to ask, you won't understand" situations. jwz is a genius, as insightful as you find him unciteful, and with a lot of relevant experience to support his worldviews. He's not for everybody, but true genius frequently goes unrecognized.
Disclaimer for your link - his webserver redirects to something not nice if the referrer is HN. Anyone visiting the link should C+P it into their address bar instead.
Extremely sorry, forgot he set up a redirect to an offensive image if you link to this from HN, and too late to edit. Please try this link instead: https://www.google.com/search?q=jwz+nscpdorm
I'm surprised that they didn't mention monetization / profitability. Even basic Google / Bing API usage will start to add up on any moderately popular chatbot with some kind of search functionality. Without the ability to serve ads, it's pretty difficult to even break even.
Chatbots are a fancy name for when your API is accessible through some well-known IM service.
They come with all of the monetization issues of APIs, but they're surfaced in an environment where you have no pre-existing relationship with the client, and you can't even charge them for access.
This essentially limits the business model to where you give away the chatbot and make money some other way; or you recognize you have no business model and do it anyway.
That's a very good point indeed. In that particular case the diagnostic chatbot is a proof-of-concept to grab attention to the diagnosis+NLP API offered by Infermedica, and also serve as an example app.
This is 100% correct. The chatbot is currently a proof-of-concept and our first take on a conversational symptom checker. Our core business is the Infermedica API and patient prescreening tools based on the diagnostic engine.
Wouldn't a more graphical implementation of the symptom checker, possibly with a tree the user can follow in a wizard style be more effective than a simple texted based chat?
Sadly most bots today are solutions looking for a problem.
Our customers (thankfully) are typically looking for ways to engage with their existing customer base better, and/or deliver their existing commercial services in a better way.
"Our first finding was that requirements 2 and 3 are satisfied by a feature called Message Groups. What came as a surprise...ActiveMQ and RabbitMQ, do not support this feature"
Actually this is right, so the information in the article is somewhat incorrect. Thanks for spotting this!
There is however an issue with the ActiveMQ implementation: once a user is assigned to a consumer, it is hard to make ActiveMQ rebalance it. At least by default the load-balancing does not take place until a consumer is dead. In case of a chat bot, some users may be returning, while others may quickly go away, leading to drastic imbalances. Apollo does the balancing perfectly out-of-the-box.
> To put it bluntly, a huge proportion of messages will not be understood at all and there is little we can do about it. What we can do is anticipate user frustration and alleviate the situation by offering a little help and managing expectations.
I get the impression that a lot of people have found this to be the case with chatbots these days.
Chatbots can be great for certain situations - when an app needs to initiate the interaction, for example. However, any UI that routinely causes users significant frustration seems pretty problematic. It has to let you do something so awesome and special that you're willing to put up with all that frustration - e.g. a buggy screen sharing app is often still better than no screen sharing app. Is there a compelling enough benefit for chat UIs? Maybe if all you have is text messaging and can't install apps or browse the web?
I'm inclined to think that not trying to be too smart and instead just going with more of a command line interface rather than an "intelligent" agent might be better for users. Have people developing chatbots found that to be the case?
I suffer from general anxiety. I listed some of the common symptoms of such (chest pain, feeling anxious). I was given a "weak evidence" report of myocardial infarction, and there was no mention of anxiety.
I assume most people new to the symptoms do not know what myocardial infarction is (heart attack). And that if you are experiencing one or new symptoms you should consult with a doctor (call family doctor or visit ER, etc.)
I had no trouble communicating my symptoms and the questions made sense, but despite this the information shown in the end was not helpful in some obvious ways. This seems like a substantial usability problem that is not addressed in your article.
Since everyone is talking about chatbots, I thought this might be a good thread to ask for some feedback:
I am considering creating a chatbot platform for developers to create, host and sell bots (as a side project which may turn into a main project if it goes well),
I was wondering if it's something devs/people are interested in? (to those who don't code, there'd obviously be an easy GUI way to build basic bots - like wit.ai)
the initial target customers would be devs - this would be a chatbot hosting service of sorts.
However the idea would be to later expand to a store (like the App Store or Play Store but for bots - devs get paid, people get bots, my service is the middle platform)
It depends on what you mean by a "chatbot platform".
If it's a way of developing a single "chatbot" and having it work across multiple "platforms" (think slack, hipchat, irc, facebook, standalone, etc... maybe even including voice-specific platforms like Amazon echo and Google Home). I would LOVE that. However it's going to most likely suffer from the same issues as other "platform-independent systems" which is that you'll be developing to the lowest common denominator.
If it's just a way to make a chatbot via a GUI, I'm personally not interested at all.
The way we handle this (and avoid the lowest common denominator problem) is allowing you to have one bot, but then specific 'modules' that execute only if being accessed over a specific network.
This lets you build one bot that supports everything from SMS to Messenger to Slack in the same interface.
Other tools I've seen apparently do automatic conversion of one rich media form to another, but I'm less keen on that idea personally.
The idea is just as you mentioned - a platform where you'd basically create a bot (by that i mean, your code will be given a context, and your response would be forwarded to the chatting service - be it slack, irc, etc.)
Furthermore, with an base SDK, you'd not build for a specific service, you'd build it for MY service and it'd handle every other service out there (insert XKCD joke about different platforms here)
Also as for GUI, it's not a requirement - it's for those who are not programmers but want to a chatbot for their Echo or Home etc. (an IFTTT of sorts if you will).
Also it'd have extra stuff that devs won't have to worry about (like hosting or charging users, promoting etc.) - all of which would be handled by my platform).
Isn't there a lot of competition in this space? It seems like your bots would need to be particularly intelligent or the interface would need to be particularly low friction.
Overview of chatbots in 2016:
- Almost every messaging platform announced support for bots.
- Almost every major tech company announced their own machine learning and NLP products.
- 30,000+ people starting building bots.
- 100,000+ bots built across different platforms.
- Lots of hype, not a lot of useful bots.
Chatbots are a great place to experiment. I made one for a hosted web chatroom that essentially ran on markov chains of varying accuracy to the source (choose something that had the same previous one/two/three words, etc).
It would listen to the chats in the room for training data, and optionally could parse and ingest a webpage (blog, wikipedia entry, email, etc). You could then ask it a question, which would make it generate a sentence based on a random word it picked out of the question. The goal was to make something entertaining for the chatroom, and it mostly succeeded with often hilarious results :)
Something fun was giving it "personalities" based on input data, for example giving it the timecube website, or read in a person's emails to copy their style.
One of the bigger challenges I've seen is driving usage. We've got the MVP up and running, and it's possible to improve the experience, but getting more people to message in (FB Messenger) has been a hurdle.
Yes, I actually wrote this post on that subject. http://bit.ly/2hngmmr I've seen so so results so far. It's clear that Facebook is working on finding ways to increase people messaging in.
1. I think there will be new best practices on top of new targeting for this destination.
2. I'm genuinely interested to hear other success stories and learnings. Are people finding that Messaging apps are 'build it and they will come' or are there new techniques for driving usage?
Great post, I bookmarked it (and your website as well). I'm also looking for more success stories / use cases.
I'm personally not very convinced about automated conversational interfaces, but Messenger does provide a great opportunity (1bn users) and platform (ios/android/web) to create and grow new kinds of applications with very little effort, if only we can figure out how to get people using it :)
Be convince about automated conversational interfaces! They are not the miracle that AI would be, but they exist and 3-4x the results of webforms.... Once the conversation starts. Here is another case study I wrote about from the election. http://bit.ly/2fPwwSz
The top comment on this post is the same topic, worth checking out. Thanks for the bookmark.
Just curious, do people already know about your product/company but they're just interacting with it in a different way than the chatbot? Or is the chatbot your main interface?
A mix of both. I've built a dozen or so bots for people. Sometimes the FB Page is already getting Messenger traffic. The more likely scenario is an organization wants to build a Messenger campaign and then promote it.
> Claudia Bot Builder helps developers create and deploy chat-bots for various platforms in minutes to AWS Lambda. It simplifies the messaging workflows, automatically sets up the correct web hooks, and guides you through configuration steps, so that you can focus on important business problems and not have to worry about infrastructure code.
> If users just wanted to tap and swipe, they wouldn't be using a messenger; they'd go to your fancy responsive website
The point of a chat bot is to perform things without leaving a chat. I'm not going to go to chat bot to do something I could do with a better UI than a NLP bot.
If a chatbot would ever say "would you like to chat to me or go to a nice website where you can tap and swipe to describe what you mean" - then that's all that chatbot would ever have to say.
I think a symptom checker bot has much to offer (though the devil is in the details). How is information extracted from a curated knowledgebase? In any case, a system with such probabilistic machinery could add a lot to our healthcare system since it would not make human errors such as base rate neglect. Furthermore, would existing doctors approve of machine responses?
I am the author of the article and also the designer of the NLP system. If you're asking about how the mentions of the medical concepts (as defined in the curated knowledge base) are extracted from user messages then here is a short explanation.
The bot uses an entity recognition engine for this. The engine is available via API [1].
The engine itself uses dictionary phrases from the knowledge base and a custom matching strategy that operates on dependency graphs, which allows to abstract from some surface details (e.g., understand that “pain in my left leg” is “pain in leg”). It also contains a modified version of the DepNeg algorithm [2] to detect negated mentions (“I don't have headache”, “no history of chest pain”).
I went through the Alexa Skills tutorial and I'm looking into making another bot. I agree with the article that designing the natural language interface is the hardest part and they touch thisbut from a messaging viewpoint. It looks like most APIs attack this using some event driven system.
Has anyone used IBM Watson Converstations? I would be interested in understanding how it compares with the others on market and whether I should invest in learning it versus the other options?
Speaking about chat bots. There are the usual simple bots based on keywords and simple fixed coded branches.
But for more advanced bots, that uses NLP (eg Stanford's CoreNLP). What's the next step? What do you do with input sentence, each word labeled as noun, adverb, person, company, etc ? I imagine a list of sentence structure templates and calc the probability. This step is only very briefly and vaguely mentioned in literature (books, papers).
Can you recommend some blogs, sample code, tutorials that explain that step, how resonate based on NLP lib? And where machine learning comes in, to improve the trained data?
It uses ML to figure out the "intent" of what the user has said (using a document classifier) and named entity recognition (NER) to extract data from the text (date, location etc).
That is actually the easy part. The harder part is the conversation scripting and state handling. Mutters uses the Ink narration engine (originally designed for dialogue heavy games) for this.
I played around with Mycroft's Adapt Intent Parser[0] a couple weeks ago. It's written in Python. I was looking for something open source to replace IBM Watson's Conversation API.
With Mycroft working I quickly realized that I'd need some way to deal with state handling and conversation branching. I don't think Mycroft has a counterpart to the Ink narration component - I'm really curious to dig into that!
There are hosted services like API.ai, wit.ai and Microsoft LUIS. Init.ai has NLP built into their bot building platform. Rasa NLU is an open source NLP project that is very interesting.
I'm curious if some of those platforms allow to write custom code for processing of the raw message. Either to embed the code directly on their server or to expose a custom webhook.
For instance, if I were to write such a diagnostic chatbot as described in the article but would prefer to use a service such as those to take care of the integrations with various platforms, message queueing, tracking basic stats etc.
There are custom-built bots that use tools like CoreNLP. They tend to not use any framework.
But, most NLP-powered bots are using machine learning based NLP APIs rather than CoreNLP. These tend to do two functions: intent determination and slot detection.
Intent determination means finding the overall meaning of a message. Slot detection means locating and extracting important terms (not just standard NER) within a message. Both happen based on examples provided by training data that is used to train a machine learning model.
The difference intent determination and keyword-detection is that ML based intent determination is much more resistant to different ways of saying the same thing.
If you search the ML research for intent determination and slot filling you will find various techniques to do it, from CRFs to CNNs to RNNs.
Example:
"I want to find a flight from London to New York"
=> Intent: find_flight, City.Departure: London, City.Arrival: New York
That set of information will then be passed to program code to determine the response.
(generalization:)
Right now, most code will generally map intents and conversation state to a branch of code and will have code that takes specific action based on the current conversation state. Whenever slots are detected, those get stored to conversation state. As the user types each message, the process repeats.
The logic is generally either structured as a tree or as something that resembles a state machine.
Examples are Microsoft's Bot Framework, Wit.ai's SDK, Init.ai's conversation logic, etc.
(the future)
The machine learning model can be adapted to predict the next message type if it's trained on sequences of messages. At Init.ai we do that, where in addition to classifying the current message and extracting slots, the machine learning model predicts who (which person or computer) will send the next message and what type it will be. That enables the bot developer to not have to write code and let the prediction system select text to send as appropriate.
In the future you'll see other things that make it easier, where the machine learning system starts using more of the conversational content to make more accurate sophisticated predictions of what actions to take and messages to send. For example, some new ML research is around automatic question-answering based on a dataset, and that could be used to make bots respond automatically based on information.
So over time it will get easier for developers as the ML progresses.
I work for chatbot building platform: http://deeppixel.ai/. We use state-of-the-art ensemble of deep learning algorithms that try to understand the meaning of sentences by taking the whole sentence as well as bag of words/bag of ngrams. In addition to that, we use classic NLP techniques like POS tagging, constituency parsing, dependency parsing, Named Entity Recognition (NER), clustering.
As an alternative to building it yourself you can rely on services that do this for you.
I've been using the microsoft bot framework together with the luis.ai (azure) service for natural language parsing. Works really well in my experience. It supports bimodal operation where first a regex is attempted and then as a fallback the sentence is passed to the NLP service to parse into action/entities.
I can really recommend the microsoft bot framework even without the natural language parsing. It plugs into a range of chat platforms, including sms and email, and you're free to build your bot with node or .net. I went with node and have been pretty happy with it so far.
What chat platforms other than Microsoft's Bot Framework is there? I built a chatbot with C# on BF and frankly it hasn't been a smooth experience, but to be fair, they were changing things so fast.
I almost expect AWS to release their own "Bot Framework" soon.
Smooch.io (disclaimer: my company) has a really rich API for programming conversations.
You can use the API to converse with people on web chat, iOS/Android in-app chat, SMS, Facebook, Telegram, Viber, Twitter DM, WeChat, LINE+++ all through one single set of APIs. You can even send rich structured messages that are automatically translated into the best representation on each platform.
Thank you so much for smooch! I would highly recommend smooch.io for anyone looking to handle conversations on multiple platforms. The implementation with Facebook Pages is brilliant.
Ive tryied to train a chat-support bot with data- the biggest challenge was ironically - the read the documentation, one answer- one link answers.
You can expand all the answers , but the web is incomplete, readable text and direct pointers are rare- so you end up with whole wikkipages in your answers.
Using NN, you migrate your problems into filter tooling and data refining.
The article describes:
1 (scalable architecture) : but the common solutions to this problem are not unique to chatbots
2 (a conversational agent must converse) : actually, that's a design decision for each individual bot, there are alternative ways of thinking about it.
3 (users will get frustrated) : this is actually a not a tech/AI problem, but a UI (user expectation) problem, see below.
The actual top 3 problems chatbot makers face are (in order):
#1 Discovery (!!!) : people don't know where/how to find bots and Facebook does not have their shit together in this regard (almost a year after launching their bot platform, there are still crucial errors. Result: most bots that could make a difference in the real world never get any traction, or enough users to train or iterate.
#2 Onboarding / setting expectations : the majority of (non-test or -product hunt) users of your bot are having their first chatbot experience with yours. Guiding them in the correct way is your #1 design priority. Result: a dissapointed general public.
#3 Inflated expectations : Mainly a problem for those who try to sell chatbots in B2B. Its' almost the opposite problem to #2. Most people and media conflate AI with chatbots. But the recent advances in machine learning (which are the reason for the current attention the term "AI" has been getting) are not (yet) applicable to chatbots in general. Add to this expectations of a Jarvis, Siri, etc. but then "for x". Result: dissapointed clients and an even quicker deflation of the hype.
And this is not even going into monetization!
All in all, I'm just frustrated that everyone involved in bots seem to assume that we already know how bots should work. I work in bots and I have yet to see one really impact user's lives. I wish for more humility and experimentation. At least we're all sharing what we're learning, so there's that.