Coaching sales agents? Use AI and human coaches

November 20, 2020

Researchers from Temple University, Sichuan University, and Fudan University published a new paper in the Journal of Marketing that explores the growing use of AI to coach sales agents to determine if there are any caveats that inhibit the effective use of this technology.

The study forthcoming, in the the Journal of Marketing, is titled "Artificial Intelligence (AI) Coaches for Sales Agents: Caveats and Solutions" and is authored by Xueming Luo, Shaojun Qin, Zheng Fang, and Zhe Qu.

Many companies now turn to artificial intelligence (AI) to provide sales agents with coaching services that were originally offered by human managers. AI coaches are computer software solutions that leverage deep learning algorithms and cognitive speech analytics to analyze sales agents' conversations with customers and provide feedback to improve their job skills. Due to their high computation power, scalability, and cost efficiencies, AI coaches are more capable of generating data-driven training feedback than human managers. MetLife, an insurance giant adopted an AI coach named Cogito to offer training feedback to its call center frontline employees to improve customer service skills. Similarly, Zoom uses its AI coach, Chorus, to offer on-the-job training to its sales force.

Precisely because of the big data analytics power of AI coaches, one concern is that feedback generated by the technology may be too comprehensive for agents to assimilate and learn, especially for bottom-ranked agents. Further, despite their superior "hard" data computation skills, AI coaches lack the "soft" interpersonal skills to communicate the feedback to agents, which is a key advantage of human managers. The lack of soft skills may result in agents' aversion to receiving feedback from AI coaches, thus hampering their learning and performance improvement. Indeed, the design of AI coaches often focuses on information generation, but less on learning by agents who may differ in learning abilities. Therefore, it would be naïve to expect a simple, linear impact of AI coaches, relative to human managers, across heterogeneous sales agents.

With this background, the study addresses several research questions:

Which types of sales agents, bottom-, middle-, or top-ranked, benefit the most or the least from AI vis-à-vis human coaches? Is the incremental impact of AI coaches on agent performance heterogeneous in a non-linear manner?

What is the underlying mechanism? Does learning from the training feedback account for the impact of AI coaches?

Can an assemblage of AI and human coach qualities circumvent caveats and improve the sales performance of distinct types of agents?

These questions are answered by a series of randomized field experiments with two fintech companies. In the first experiment, a total of 429 agents were randomly assigned to undergo on-the-job sales training with an AI or human coach. Results show that the incremental impact of the AI coach over human coach is heterogeneous in an inverted-U shape. While middle-ranked agents improve the most, both bottom- and top-ranked agents show limited incremental gains. Results suggest that this pattern is driven by a learning-based underlying mechanism. Bottom-ranked agents encounter the most severe information-overload with the AI coach. By contrast, top-ranked agents display the strongest AI aversion problem, which obstructs their incremental learning and performance.

The slim improvement in bottom-ranked agents is an obstacle for AI coach adoption because they have the largest room and most acute needs to sharpen their job skills. The researchers re-designed the AI coach by restricting the amount of feedback provided to bottom-ranked agents. With a separate sample of 100 bottom-ranked agents, the second experiment affirmed a substantial improvement in agent performance with a restricted AI coach. A third experiment tackled the limitations of either AI or human coaches alone by examining an AI-human coach assemblage, wherein human managers communicate the feedback generated by the AI coach to the agents. A new sample of 451 bottom- and top-ranked agents were randomly assigned to the AI coach, human coach, and AI-human coach assemblage conditions. The results suggest that both bottom- and top-ranked agents in the AI-human coach assemblage condition enjoy higher performance than their counterparts in the AI coach alone or the human coach alone condition. In addition, bottom-ranked agents gain more performance improvement than top-ranked agents with the hybrid of AI and human coaching. Thus, this assemblage that harnesses the soft communication skills of human managers and hard data analytics power of AI coaches can effectively solve both problems faced by bottom- and top-ranked agents.

As Luo explains, "Managerially speaking, our research empowers companies to tackle the challenges they may encounter when investing in AI coaches to train distinct types of agents. We show that instead of simply applying an AI coach to the entire workforce, managers ought to prudently design it for targeted agents." Qin adds, "Companies should be aware that AI and human coaches are not dichotomous choices. Instead, an assemblage between AI and human coaches engenders higher workforce productivity, thus allowing companies to reap substantially more value from their AI investments."
Full article and author contact information available at:

About the Journal of Marketing

The Journal of Marketing develops and disseminates knowledge about real-world marketing questions useful to scholars, educators, managers, policy makers, consumers, and other societal stakeholders around the world. Published by the American Marketing Association since its founding in 1936, JM has played a significant role in shaping the content and boundaries of the marketing discipline. Christine Moorman (T. Austin Finch, Sr. Professor of Business Administration at the Fuqua School of Business, Duke University) serves as the current Editor in Chief.

About the American Marketing Association (AMA)

As the largest chapter-based marketing association in the world, the AMA is trusted by marketing and sales professionals to help them discover what is coming next in the industry. The AMA has a community of local chapters in more than 70 cities and 350 college campuses throughout North America. The AMA is home to award-winning content, PCM® professional certification, premiere academic journals, and industry-leading training events and conferences.

American Marketing Association

Related Learning Articles from Brightsurf:

Learning the language of sugars
We're told not to eat too much sugar, but in reality, all of our cells are covered in sugar molecules called glycans.

When learning on your own is not enough
We make decisions based on not only our own learning experience, but also learning from others.

Learning more about particle collisions with machine learning
A team of Argonne scientists has devised a machine learning algorithm that calculates, with low computational time, how the ATLAS detector in the Large Hadron Collider would respond to the ten times more data expected with a planned upgrade in 2027.

Getting kids moving, and learning
Children are set to move more, improve their skills, and come up with their own creative tennis games with the launch of HomeCourtTennis, a new initiative to assist teachers and coaches with keeping kids active while at home.

How expectations influence learning
During learning, the brain is a prediction engine that continually makes theories about our environment and accurately registers whether an assumption is true or not.

Technology in higher education: learning with it instead of from it
Technology has shifted the way that professors teach students in higher education.

Learning is optimized when we fail 15% of the time
If you're always scoring 100%, you're probably not learning anything new.

School spending cuts triggered by great recession linked to sizable learning losses for learning losses for students in hardest hit areas
Substantial school spending cuts triggered by the Great Recession were associated with sizable losses in academic achievement for students living in counties most affected by the economic downturn, according to a new study published today in AERA Open, a peer-reviewed journal of the American Educational Research Association.

Lessons in learning
A new Harvard study shows that, though students felt like they learned more from traditional lectures, they actually learned more when taking part in active learning classrooms.

Learning to look
A team led by JGI scientists has overhauled the perception of inovirus diversity.

Read More: Learning News and Learning Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to