A/B Testing: Developing Personality For Bots To Increase Chat Sales
Since Facebook unveiled its bot campaign, e-commerce has seen the proliferation of bots. These artificially intelligent chat agents hang out in Facebook Messenger attempting to increase chat sales for big e-commerce brands, from Starbucks to Ebay to British Airways. They answer product questions, offer suggestions based on a customer’s purchasing profile, and even assist with sales transactions making mobile purchasing easier than ever.
After announcing the presence of 11,000 bots ready to chat last July, Facebook has scaled back its AI program significantly after reports of a 70% failure rate. In other words, human agents had to step in for approximately 7 out of 10 chat conversations. While the issue of how to create a bot personality advanced and nuanced enough to interact with today’s net savvy consumer is prescient, it may be better suited to the future of the bot landscape. Facebook may not see a ROI from its bot program, but there are a few valuable lessons to be learned through chat conversation analysis. For the foreseeable future, AI for e-commerce is best implemented as human + machine. When the personality of the bot matches the emotional intelligence and appeal of both its seat mate (read: live chat agent) and the customer, e-commerce companies can increase chat sales.
So how does one go about developing a captivating bot personality? If the goal of any bot is to reach a positive conversation outcome – whether recommending a restaurant or influencing a purchasing decision – then a bot’s personality should use language that makes its interlocutor sympathetic to its position. While this may sound like a tip from a Sales 101 textbook, the root of this phenomenon forms the basis of human behavior. Research shows that human personality is mutable. Whether it’s to win someone’s favor or to fulfill a need or want, humans naturally adapt the way they talk to suit their conversational partner. To increase chat sales, a bot needs to change the way it talks depending on if it’s speaking with a millennial or a baby boomer.
Furthermore, a conversation is not as simple as a speaker who talks and a listener who comprehends. Oftentimes, the listener is actively agreeing or disagreeing with the speaker’s point before she’s finished talking, and is formulating an appropriate response in real time. A study of speaker-listener interactions shows that conversation is best modeled as a multilayered event in which the act of speech is mediated by many micro-adjustments of acknowledgement, response, and further action occurring in real time.
If human conversation involves so much adaptive improvisation, how can we build a bot that’s smart enough to interact with – and adapt to – human conversation? By repeatedly testing the bot prototype’s speech for positive and negative outcomes, or in other words, through chat conversation analysis and A/B testing the results, we can build a bot that’s smart enough to increase chat sales.
At RapportBoost, we stand by the fact that A/B testing is what separates the bot winners from the losers, and here’s why: conversations need to be analyzed for causality. We conducted an analysis of hundreds of thousands of chats from one of our customers with an eye toward the factors that were driving order size. We found that a specific cluster of words had a statistically significant effect on order size: redeem, voucher, purchase, buy, coupons, already, enter. When these words or their iterations were present in a sales conversation, the order was significantly smaller.
A naive take on these results would suggest that the company should discontinue its voucher program because it was cutting into their profits, but our RapportBoost data analysts knew better. Rather than denoting a causal relationship, the live chat agent was offering coupons and shipping vouchers when a customer was already dissatisfied with the order, and because of this, was making a small or downsized purchase.
In order to understand if the voucher program was a cause of or an effect of the dissatisfied customer, we used A/B testing to generate the optimal conversational strategies that chat agents should employ based on circumstance. This in-depth chat conversation analysis revealed that customers who were offered a voucher or coupon were ten times more likely to become a repeat customer. Continuing the voucher program would not only protect the company’s revenue, it would foster long term customer loyalty.
So what’s the right approach for your company? That’s the RapportBoost secret sauce. It depends on the circumstance of the conversation, the characteristics of the data set, and the personality of the customer in any given scenario. In fact, an e-commerce company’s path to customer satisfaction may even change over time. The RapportBoost system analyzes a set of customer data, A/B tests different strategies, and once it’s developed a significant sample size, recommends a live chat strategy for optimal conversational outcome.