I was leaving Lidl recently with my toddler and whilst demonstrating to him how to use the terminal I accidentally pressed the 'sad face'.
There was a collective gasp from all the checkout operators, so I assume there is a direct impact to their performance rating. We pushed the super-happy face a few times to compensate.
That’s my experience. Every time I go through Chinese immigration/customs they have these rating smiley buttons but even when my service has been slower than normal I never press the frown button because the person is right there in from of my face watching me. Too awkward.
The setup at immigration has it so the officer can't see what you pushed. Yes you can see them, but they don't know what you pressed -- at least at every Airport in China I've visited.
This 4 scale smily face is pretty common in China, the started over 10 years ago. Even at places like banks and government agencies. They were a key part in how China got it's day to day corruption under control. People would pay bribes for service but leave a poor rating. And while corruption still exists in China, it is no where as in your face or omnipresent as it once was.
No, not really. I think it’s similar to NPS. It’s visible to the store clerks to allow them to correct their mistakes on the spot, instead of having to involve corporate teams. It’s the same idea as the one behind short development cycles. It gets you faster to the goal. And the ultimate goal for the company is to provide good service and not to have another way to evaluate employees.
How could store clerks correct or even understand their mistakes just by seeing a frowny push? The data most likely do go to corporate (which will return to the PoS as retribution), as evidenced by distress seen on the staff's faces when the frowny button was pushed.
NPS is a disaster. Since it's directly tied to financial consequences for staff, (1) staff beg for 10s, (2) anyone who knows how NPS works and has a conscience always gives 10 in order to protect staff from maangemetn abuse.
I never understood why they even bother with 5- or 10-point systems when the only "good" signal is a 10. If that's all you want to collect, just ask me if I was happy or not. Don't give me a bullshit range and hope I know that only the most sparkling of reviews will save the employee I actually like.
Doing NPS is impossible, because nobody will use the intermediate ratings.
Besides, people are pretty much incoherent about what is the difference between a 3 and a 7. Some people's 3 are a much higher rank than other people's 7.
Putting 10 options in front of random people is mostly useless. There are very few places where it will lead to meaningful choice.
Netflix documented this pretty well. They also know that a 3 on a Monday means something different than a 3 on Friday night, even from the same person.
These and other things came out of the competition to improve the success rate of the recommendation system.
> How could store clerks correct or even understand their mistakes just by seeing a frowny push?
Because they are humans? Doesn't everybody learn from their mistakes? Some are worse at it than others, sure, but that's what upper layer of data analysis is for...
I've worked as a store clerk. You don't get positive reinforcement for doing a good job. You just get yelled at if you screw up. Or somebody else screws up. Or nobody screwed up but the manager expects you to be superhuman.
"that the most important information from any location comes from the number of frowny presses, which not even a dishonest employee would be able to undo."
So possibly pressing the happy face doesn't compensate. The article also mentions that there's a reset delay to prevent people from pressing buttons repeatedly.
In 2018, a reasonable person might suspect that this is what professional PR looks like, too, since a forward-thinking advertising firm wouldn't necessarily balk at hiring people to post nice things about it. :)
Interesting to see an article about this, I knocked together a React app the other month for my company based off questions from rands famous "Shields Down" https://medium.com/the-blueprint/shields-down-c291f015618f using this method.
There is one of these in the grocery store near my house. The store has very long lines between 5 & 6 P and I noticed that the terminal conveniently disappears during that time. It's always there on the weekends though.
We used to have this in society; it was the actual frowny or smiley face, like on your actual face. We gave them to each other all the time (well, maybe not in Finland). The problem now is that the people who get that information cannot do anything with it, because all of the power is too far up the hierarchy.
Are you saying that when a cashier asked, people didn't automatically smile and say "I'm fine, thanks" no matter how awful they felt the experience was?
Here we have a form of analytics that doesn't piss people off, because, as noted in the article, it makes it slightly more difficult than the standard javascript package to sell you more shit just because you wanted to let the store know they were not earning your business.
Human interaction is both under- and over-rated by tech companies.
On one hand, we spam the ever-living shit out of people with email ads, push notification ads, etc to try to generate engagement.
On the other hand, we sometimes go out of our way to remove the human element from other parts of the loop: consider how rare it is to see a prominent "never ever recommend me this [show|song|book|whatever] again" button. With the massive number of potential features to choose from for finding similar pieces off content, I'd like to have more control there. Like a way to tell Spotify "I want my Discover Weekly playlist to only pull from songs that are less than 5 years old" or such.
Spotify actually does have a "Never recommend this song or this artist again" feature. If you click on the "No" symbol next to the heart in your daily mix or Discover weekly.
They still need a better overall algorithm. I often find myself in an imagined battle with the daily mix list clicking no to every song it suggests in an attempt to train it away from the endless stream of top 50 pop trash. Once it just stopped playing completely, I decided to take that as a minor moral victory.
> Spotify actually does have a "Never recommend this song or this artist again" feature. If you click on the "No" symbol next to the heart in your daily mix or Discover weekly.
Ooh, thanks.
The iPad app seems to be missing this... No heart, either.
Now to google to figure out the heart vs the + button, too, hah.
I used to run an email based version of this aimed at tracking company/team mood. It was indeed frictionless and we collected a large amount of data (especially compared to annual employee surveys or every biweekly retrospectives), but it wasn't very actionable on it's own.
The best outcomes we ever achieved were triggering a conversation. Helpful, but I'm not sure I would call it "revelatory".
I've seen HappyOrNot terminals in airports (typically at airport restroom exits: "How clean was the restroom?"). This seems like a terrible idea from a health perspective! These terminals could be a serious vector for spreading illness, especially for people leaving a dirty restroom in a high-traffic international airport.
I can't remember the last airport I was at where there was an actual door you had to open. Although that is probably more for convenience w/bags than cleanliness.
At least around here, crowded places started removing door handles after the SAS epidemics on Asia. I am pretty confident it was because of health issues.
I am assuming the airport had stalls with doors, and those doors had latches. I admit I don't travel very widely, but I'm yet to see hands-free stalls.
It shouldn't be hard to order the sequence of steps such that washing your hands is your last activity before leaving the bathroom.
Good idea. The airport restrooms already have hands-free sinks and blow dryers. They could incorporate the voting mechanism into hand sanitizers so voting actually reduces the chance of spreading germs. :)
I was thinking of taking a Amazon IoT button (or particle photon since I found it again) and make it call a tech support line / inform IT if something is broken. Would other people be interested in a tutorial to how to do this?
The Amazon IoT button quickstart guide is a tutorial for making the button send an email to an address of your choice, so arguably this already exists.
The interface is very friendly -- it's so simple that I find it compelling to use. A button press doesn't take any time out of my day or any data I wouldn't want to share. (Hopefully there are no strings attached e.g. video monitoring.)
When they started appearing near my local pharmacy, it was placed next to a stand of toys with giant cute eyes, which drew my attention to the "matching" happy face. Nice job on strategic placement, I thought.
Reading the article, yes, you're leaving more information than you think (but maybe not more than you're willing to share).
Video monitoring is mentioned in the article together with timestamping. A store had a "customer satisfaction" problem in the morning, so together with video footage, they could determine there was slow-starting employee who was at fault.
> One client discovered that customer satisfaction in a particular store plummeted at ten o’clock every morning. Video from a closed-circuit security camera revealed that the drop was caused by an employee who began work at that hour and took a long time to get going. She was retrained, and the frowns went away.
Unless I’m missing something, timestamps seem to be HappyOrNot’s only defense against malicious users.
While that works against a toddler who relentlessly taps the unhappy button, it’s an ineffective defense against a manager intent on manipulating feedback.
"Well the stadium/store/restaurant/security-area has cameras and we saw you pressing the button [at all]. Care to explain that?"
(Also: why would they press the button? bad reviews being given)
"We see that, while your over all ratio of good to bad isn't horrible, you have quite a bit more bad than usual and more votes than usual without the sales we'd expect."
By the time you add the costs to administer these buttons which includes detecting fraud, reviewing security footage, and contacting the store, the value proposition of these buttons evaporates. It's no longer a cheap source of customer feedback.
"When I first met Theisen, I asked him what would prevent a store manager from standing next to a terminal and repeatedly pressing the smiley button, and he said that... the most important information from any location comes from the number of frowny presses, which not even a dishonest employee would be able to undo."
Less invasive potential starting point: compare number of impressions with number of sales, look for suspicious outliers. More sophisticated options could be derived from similar easy-to-get numbers, too, like amount of foot traffic, door openings, etc.
That angle is just not covered in the article, but wouldn't it be safe to assume that there is some sort of algorithm cleanup of the data? Collapsing clusters of pushes, or something.
The timing of this for me personally is very interesting. I encountered and used one for the first time at IKEA yesterday. Their terminal was aimed specifically at customers' parking experiences, which makes me think they already know what kind of responses they're going to get.
That's interesting that they'd measure the parking lot experience. I wonder how they can change the parking lot experience to experiment with happiness levels. Route traffic differently through the different aisles? Provide shuttle services? Add distractions/things to read/music on the walk into the store?
A portion of this story reminded me of the "Story of Manna" by Marshall Brain regarding the automated fast food restaurant.
These days, it would be very cheap and easy to wire up simple "HELP" buttons at all the trouble points in a restaurant, along with cameras and machine vision (other than cameras in the bathroom). And those sensors could create a synthesized view of a restaurant or department store, or whatever. The synth view can then inform where help is needed.
But these sensors, low energy cpu's and radios are already attainable at the cheap. And at industrial productions, could be as low as $.50 per button. I'm really surprised why nobody has done this yet - and I read the article and found that yes they have. It's called HappyOrNot, and it's for surveys and quality control of every metric in 1 variable.
I run into these all the time in Finland. I have never actuated their buttons. I often have feelings about the quality of the establishment or the experience but they can never be expressed adequately by a 1-5 scale. I suspect the data gathered has a significant bias.
The thing that really annoys me is when the question doesn't match the way the data's used. My ISP asks for a rating out of 10 for "how likely would you be to recommend <ISP> to family and friends" after every support call, but the phone operators explicitly tell you that they're rated on the score that you give them. Those are two different questions - I'd like to rate the phone operator highly, the back-end technical administrators as average (they once assigned the same IP address to myself and another customer - it's disappointing that this is even a manual process!) and the ISP as a whole as 'could be improved' due to their lack of upstream bandwidth at peak times.
It seems to mostly be about knowing "hey, there's a problem of some sort right around here," under the theory that once you know that, finding the solution is relatively easy.
It works that way in debugging software, at any rate.
Maybe this is evil of me, but I think it'd be super interesting to strictly vote "very unhappy" on everything. What kinds of changes do you think businesses would implement if people were still buying stuff despite claiming to feel very unhappy?
The interesting data is whether or not the business is getting better or worse over time, so any outliers in the data probably wouldn't change anything at all. If they're not outliers then a policy of voting "very unhappy" regardless is just adding to weight of other very unhappy voices.
The problem with this is that once every vote is "10 or bust" your signal saturates and becomes meaningless.
What they really need to ask, if you're right about what they want to measure, is "Was your experience today better / same / worse than last time?" And then they need to stop pushing for you to click "better" every time, "same" should be the usual response unless they've actually improved notably.
Mentioned in the article about the Swedish shop - more sale she in the afternoon than morning, so what was the am team doing wrong.
Turns out morning shoppers were happy, evening weren't (due to queues), and the way to increase happiness and revenue was more staff in the evening, rather than pushier sales in the morning.
I think this is very clever. There is a lesson in doing something so simple, as opposed to our engineering reflex to over-engineer everything.
My first exposure to an effective business metric was my first consulting gig which was at Domino's Pizza HQ. Tom Monaghan was convinced that shorter delivery times led to increased sales. He had been unable to convince the franchisees of this, so he started a "Service and Delivery" program. Each month, secret shoppers would order pizza two different times, record the delivery time (starting when they hung up the phone) and rate the quality of the service. I wrote the program to crunch the numbers, sorting by sales, and sorting by delivery time. The two lists had a very high correlation. He published these lists each month in a company newsletter. When I started the process, the delivery time was like 34 minutes companywide, and a year later, the time had been reduced to 28 minutes average. The sales rankings validated his intuition.
(The program had to be discontinued, after there was a lawsuit relating to a death caused in an auto accident involving a pizza delivery driver.)
So the lesson here for us engineering types is, as patio11 says, is to find a way to do business value in what you are about to build.
This seems simple, but it's actually very savvy. I don't have time to sit on an already long customer service call to answer a goddamn survey. I don't want to fill out yet another form on your gosh darn website. But let me mash one button on my way out? Hell yeah! Only too happy to vent or praise, if it literally takes a half second.
I agree. My first idle thought reading the article was "I could have done that.", My second thought was "I totally wouldn't, I would've made it way too complicated."
One issue with these is that they don't always ask you to rate what you expect. For instance, my local Walmart has one with a small sign above asking you to "rate your experience with self checkout" yet everyone who checks out with a cashier uses the button assuming you are resting any checkout experience.
I recently visited some place with these stupid emoticon buttons, probably an airport restroom. They're the same as the Uber star system: press the smiley-est button or nothing at all.
Hi can any statistician help? How many samples are we supposed to collect? Would this solution work in small boutiques where only you get 4 or 5 samples a day?
I'd like to see a comparison of results of regular HON (where buttons are arranged bad to best) vs. the one with randomly positioned buttons vs. color only buttons (no smileys).
Even easier than smiley-face buttons is a Javascript metrics bug you can embed in different places on your website; most of us could hack that up in about 20 minutes. But we don't so much wonder why Mixpanel and Google Analytics are so popular.
I wish AWS would employ this type of thing in their support department. Currently, support tickets are worked by multiple agents but you don't get the "survey" until the ticket closes.
What do you do if one agent is great and the next agent is terrible? Using something like a HappyOrNot button on every interaction would give you a lot more information about individual agents and their interactions.
But could you get an article about it in the new yorker, or get financing? It might be a largely purposeless, technically primitive product, but it provides something for management to buy, and numbers to put in reports.
The only place I've run into one of these so far was leaving an airport bathroom, which is pretty much one of the last places I'd want to share a button with strangers. There's a reason those places typically don't have doors.
I have nothing against the terminals themselves; they serve a good purpose. It's just that the smileys somehow look a bit disturbing to me. Maybe I'm in minority but jeezs why did they have to go with so goofy looking faces. The very happy face especially looks like it has a hefty underbite. But happy to see them succeeding, I think giving customers a way to vent out after bad customer experience matters a lot.
There was a collective gasp from all the checkout operators, so I assume there is a direct impact to their performance rating. We pushed the super-happy face a few times to compensate.