Skip to content

A/B Testing: Developing Personality For Bots To Increase Chat Sales

Share this article: Facebooktwitterredditlinkedinmail

Since Facebook unveiled its bot campaign, e-commerce has seen the proliferation of bots. These artificially intelligent chat agents hang out in Facebook Messenger attempting to increase chat sales for big e-commerce brands, from Starbucks to Ebay to British Airways. They answer product questions, offer suggestions based on a customer’s purchasing profile, and even assist with sales transactions making mobile purchasing easier than ever.

After announcing the presence of 11,000 bots ready to chat last July, Facebook has scaled back its AI program significantly after reports of a 70% failure rate. In other words, human agents had to step in for approximately 7 out of 10 chat conversations. While the issue of how to create a bot personality advanced and nuanced enough to interact with today’s net savvy consumer is prescient, it may be better suited to the future of the bot landscape. Facebook may not see a ROI from its bot program, but there are a few valuable lessons to be learned through chat conversation analysis. For the foreseeable future, AI for e-commerce is best implemented as human + machine. When the personality of the bot matches the emotional intelligence and appeal of both its seat mate (read: live chat agent) and the customer, e-commerce companies can increase chat sales.

So how does one go about developing a captivating bot personality? If the goal of any bot is to reach a positive conversation outcome – whether recommending a restaurant or influencing a purchasing decision – then a bot’s personality should use language that makes its interlocutor sympathetic to its position. While this may sound like a tip from a Sales 101 textbook, the root of this phenomenon forms the basis of human behavior. Research shows that human personality is mutable. Whether it’s to win someone’s favor or to fulfill a need or want, humans naturally adapt the way they talk to suit their conversational partner. To increase chat sales, a bot needs to change the way it talks depending on if it’s speaking with a millennial or a baby boomer.

Furthermore, a conversation is not as simple as a speaker who talks and a listener who comprehends. Oftentimes, the listener is actively agreeing or disagreeing with the speaker’s point before she’s finished talking, and is formulating an appropriate response in real time. A study of speaker-listener interactions shows that conversation is best modeled as a multilayered event in which the act of speech is mediated by many micro-adjustments of acknowledgement, response, and further action occurring in real time.

A/B Testing Bots Increase Chat SalesIf human conversation involves so much adaptive improvisation, how can we build a bot that’s smart enough to interact with – and adapt to – human conversation? By repeatedly testing the bot prototype’s speech for positive and negative outcomes, or in other words, through chat conversation analysis and A/B testing the results, we can build a bot that’s smart enough to increase chat sales.

At RapportBoost, we stand by the fact that A/B testing is what separates the bot winners from the losers, and here’s why: conversations need to be analyzed for causality. We conducted an analysis of hundreds of thousands of chats from one of our customers with an eye toward the factors that were driving order size. We found that a specific cluster of words had a statistically significant effect on order size: redeem, voucher, purchase, buy, coupons, already, enter. When these words or their iterations were present in a sales conversation, the order was significantly smaller.

A naive take on these results would suggest that the company should discontinue its voucher program because it was cutting into their profits, but our RapportBoost data analysts knew better. Rather than denoting a causal relationship, the live chat agent was offering coupons and shipping vouchers when a customer was already dissatisfied with the order, and because of this, was making a small or downsized purchase.

In order to understand if the voucher program was a cause of or an effect of the dissatisfied customer, we used A/B testing to generate the optimal conversational strategies that chat agents should employ based on circumstance. This in-depth chat conversation analysis revealed that customers who were offered a voucher or coupon were ten times more likely to become a repeat customer. Continuing the voucher program would not only protect the company’s revenue, it would foster long term customer loyalty.

So what’s the right approach for your company? That’s the RapportBoost secret sauce. It depends on the circumstance of the conversation, the characteristics of the data set, and the personality of the customer in any given scenario. In fact, an e-commerce company’s path to customer satisfaction may even change over time. The RapportBoost system analyzes a set of customer data, A/B tests different strategies, and once it’s developed a significant sample size, recommends a live chat strategy for optimal conversational outcome.

Follow us:Facebooktwitterlinkedinrssyoutube
Dr. Michael Housman

About Dr. Michael Housman

Michael has spent his entire career applying state-of-the-art statistical methodologies and econometric techniques to large data-sets in order to drive organizational decision-making and helping companies operate more effectively. Prior to founding RapportBoost.AI, he was the Chief Analytics Officer at Evolv (acquired by Cornerstone OnDemand for $42M in 2015) where he helped architect a machine learning platform capable of mining databases consisting of hundreds of millions of employee records. He was named a 2014 game changer by Workforce magazine for his work. Michael is currently an equity advisor for a half-dozen technology companies based out of the San Francisco bay area: hiQ Labs, Bakround, Interviewed, Performiture, Tenacity, Homebase, and States Title. He was on Tony’s advisory board at Boopsie from 2012 onward. Michael is a noted public speaker and has published his work in a variety of peer-reviewed journals and has had his research profiled by The New York Times, Wall Street Journal, The Economist, and The Atlantic. Dr. Housman received his A.M. and Ph.D. in Applied Economics and Managerial Science from The Wharton School of the University of Pennsylvania and his A.B. from Harvard University.

Leave a Comment

Scroll To Top