The key to chat sales solutions for businesses is building these algorithms —as we saw in this closed loop diagram— building the classifiers and issuing recommendations and feedback to the live chat agent. And then —and this is the key— we monitor the user behavior. We A/B test the recommendations and in some cases we tell the live chat agent to apologize. In some cases we say, you should offer something for free.
In chat sales solutions for businesses, the live chat agent is the mediator. We, the Data Scientists, don’t have direct interaction with the customer. If the live chat agent decides not to follow any of the recommendations our analytics are worthless. So we need to do live chat agent behavior modification —this is behavioral economics. We consider, how can I display this message in a way that gets across to this person? Is it how I position my response? Is it adding colors? Do some people respond better to, I think you should do this as opposed to this or this conversation is going really poorly, so take this approach.
We’re constantly A/B testing all of these things because for chat sales solutions for businesses to be effective, we need to understand the live chat agent as well as the customer. Ultimately we A/B test different approaches and we see how the visitor reacts. We predict what they would have done, then we see what they do. We do enough of that to generate a pattern to observe in the data. That’s when we’re able to learn causally what’s driving good and bad outcomes.
Information about the live chat outcome feeds back into the system. We score the recommendations that were mediated by humans with decision trees, and then push them back into the machine. Should the live chat agent go down this path or that path with the customer? Given a sampling of responses, the live chat agent chooses one and then autocorrects the bot response. The live chat agent tweaks the bot responses uses them with the customer.
When you take the human out of the loop, this is the result, taken from a customer/bot interaction from the airline Indigo:
Customer: “Thank you for sending my baggage to Hyderabad and flying me to Calcutta at the same time. Brilliant service.”
Bot: “Glad to hear it, keep flying.”
Customer: “Are you serious?”
It’s a bad interaction right? Eventually we’ll get to the point where the bot can pick up on those sarcastic comments. This is a nudge to Microsoft, but everyone knows about Tay the bot they released into the wild. Microsoft was very excited about this but people found ways to make Tay say things that were racist and homophobic. This is what we need to be careful of. The way you do that is having the humans and the machines work together, teaching humans how to interact better, having humans close the loop by teaching machines how to interact better, and then ultimately, we can build a truly bot-enabled world.
Transcribed from Dr. Michael Housman’s Lecture at UC Berkeley Business School in May 2017.
Learn more about the live chat agent training from the team at RapportBoost.