19 May 2016

Emotional Intelligence-Driven Voice User Interface Design

In a recent series of radio ads that feature a bereft and primitive attempt at a telephone customer experience, Ruby Receptionist entreats us to “rediscover the lost art of human interaction.”  Have our automated voice systems and digital interfaces become too impersonal?  These ads succinctly call out the challenge faced by customer service user interfaces: “Making your customers feel special, while making you look good.”

tired indian businesswoman using landline phone

Another popular trend is to think of customer service interactions along the lines of effort.  Matthew Dixon claims that the best path to customer loyalty should not be the often sought after goal of a special experience, but rather the more mundane, but no less critical, effortless experience.

Whether we are trying to make our customers feel special, or to reduce their effort, the emotional toll on the user seems to be front and center.  At the end of the day, a customer who is saved from excess emotional effort and strain, it would seem, is the customer who will remain loyal and happy.  But surely the multitude of best practices we have learned in the last generation of voice and digital interface design, can still be exploited to achieve this goal?

Having begun to study Emotional Intelligence (EI) (see, for example, Replacing Bad Habits and Bringing EI to PTP) and how embracing EI competencies and techniques allows one to manage emotions and relate more effectively to co-workers and clients, I, along with a few of my PTP colleagues, started to notice some interesting connections between EI principles and Voice User Interface (VUI) best practices.  Perhaps by viewing VUI design through an EI lens, we can wrap best practices in the cocoon of emotional intelligence to simultaneously eliminate friction and make the customer feel good.

The essence of EI is the ability to identify and manage your own emotions and the emotions of others. User Interface (UI) design can benefit from a similar approach.  By recognizing, understanding and managing our own emotions and thereby influencing the emotions of others, we can improve our ability to relate to clients, win sales, and garner repeat business – so why not use these same ideas to improve our IVR and digital interface systems and improve our users’ engagement and satisfaction with our UIs?

Taking this line of reasoning one logical step further, it is possible, at least roughly, to map the established best practices in UI design for digital and voice channels to the standard set of EI principles, such as the one laid out by Daniel Goleman in Working with Emotional Intelligence (see Figure 1).


Self- Awareness

Emotional Awareness

Accurate Self-Assessment

Self Confidence


Understanding Others

Developing Others

Service Orientation

Leveraging Diversity

Political Awareness







Social Skills



Conflict Management


Change Catalyst

Building Bonds

Collaboration/ Co-operation

Team Capabilities


Achievement Drive




Figure 1: Taken from “Working with Emotional Intelligence” by Daniel Goleman

Can a User Interface Exhibit Emotional Intelligence?

While there may certainly be some VUI design best practices that have little to do with EI (e.g. making questions clear and unambiguous, avoiding phonetic similarities in response options, etc.), there are many that directly correlate to and complement EI precepts.  The list of best practices is readily available to anyone searching the internet (for example Jason Brome’s Commandments of VUI Design, Paul Seibert’s VUI Design best practices, and Vicki Broman’s 10 Best Practices for Making the IVR a Desirable Destination, to name a few).   When you read these, the connection to EI jumps out at you.  So much so, in fact, that one begins to realize that most techniques we use to engage the user, not overburden the user, treat the user in as human a way as possible – and even manage the user’s emotions – draw directly from the (until now, implicit) objective of imbuing the system with Emotional Intelligence. I’ll walk you through several ways emotional intelligence informs user interface design.


Examples abound. Take empathy, the grand-daddy of EI competencies.  How does a VUI express empathy for the caller?  The machine should be sensitive to the user’s answers.  It could:

  • Adjust subsequent options it offers the user based on choices and answers the user gives
  • Anticipate and address possible anger triggers, such as when a user response does not match the grammar, or when the identification number he provides is off by one digit
  • Offer to give the user extra time to answer when she might need to find a document or look something up (e.g. finding her checkbook to find a routing number)
  • Acknowledge difficult sections of the call flow and encourage the user to be patient and persevere

Self Awareness

Anthropomorphically, the UI can manifest the very human attribute of self awareness by keeping menus short, keeping prompts simple and to the point, and providing easy access to an agent.

Every list of best practices emphasizes the importance of an iterative approach – tuning an application incrementally until it is optimized.  This too can be seen as the hallmark of a self-aware application: one that is constantly reflecting on its own behavior and making adjustments based on the feedback it receives.

Likewise, a good application is designed to recognize how a user wants to use the UI.  It will adapt to the user, in the same way that a good customer service representative (“CSR” or “agent”) may realize, based on a caller’s reaction, that the CSR is not meeting this particular caller’s needs.  By realizing that he is thwarting the user’s goals, the self-aware agent (or UI) changes course and assists the user in a more perspicuous manner.

Most techniques we use to engage the user, not overburden the user, treat the user in as human a way as possible – and even manage the user’s emotions – draw directly from the (until now, implicit) objective of imbuing the system with Emotional Intelligence.


A big part of EI is building trust with your counterpart, thereby earning credibility and respect, and establishing a bond.  A good user interface should do the same thing with the user by consistently representing the company’s brand, being an authority, and acting respectfully.

Authenticity can also be achieved by setting realistic expectations with the user – and not giving the user the impression that the system can do more than its actual capabilities will allow.  For example, open-ended questions like, “What can I do for you today?” should be avoided unless the designer has a well-defined grammar and some great back-tracking logic to support it.

Similarly, when a utility company IVR system “knows” (by querying the back end) that the caller’s area (based on caller ID) is currently experiencing a power outage, the application will gain the user’s trust, via authenticity, by informing the user upfront about the outage.

Consequences of a UI with Low EI

Just as one might expect, the consequences of designing a user interface lacking in EI are predictably negative.  Among the undesirable impacts of an IVR with low EI are:

  • Longer calls to the IVR
  • Lower rates of successful self-service
  • Decreased containment – and more transfers to a CSR, thereby costing the company more dollars in the call center
  • Frustration, impatience, and anger, which can easily impact a customer’s attitude toward the IVR itself, not to mention the company
  • User experience suffers, due to an increase in abandons, retries, and effort – and a decrease in engagement, patience, completed transactions, and overall customer satisfaction (“CSAT”)
  • Downstream experience suffers: when the user fails to achieve her goals on the IVR, the call center will often be subject to increased handle time, escalation, and lower first call resolution

In fact, we can reasonably expect that a poor user experience with a phone user interface, due to low EI, can lead to increased distress, less patience and more frustration at whatever the next channel is that the user decides to try, be it the company website, chat, social media, contact center, in person, or even calling the same IVR again.

High UI EI Benefits

Conversely, many benefits may be reaped from an automated user interface which espouses and masters EI.  Users are:

  • Less likely to get angry or frustrated
  • More likely to show patience
  • More likely to trust the machine and self-serve
  • More likely to experience higher CSATs

Further, users will experience a more satisfying interface, even if they do not achieve their goal.  They will feel more positive about the company and its products.  And they will be more likely to transact via self-service in the future – thereby saving the company the cost of a human interaction. Wasn’t that the whole idea in the first place?

It is worth noting that equally as important (or perhaps even more important) as designing the UI to have high EI, is for CSRs to exercise good EI while speaking to customers.  After all, even the VUI with the highest EI cannot always meet the caller’s needs via self-service.  It is paramount that agents be trained to use the very same EI techniques to calm down irate callers, address concerns, empathize and build trust.

The Emotional State of the User

Figure 2 The best user experience lies at the intersection of best practices, high EI, and accounting for the user's emotional state.
Figure 2: The best user experience lies at the intersection of best practices, high EI, and accounting for the user’s emotional state.

While designing your user interface to be emotionally intelligent will go a long way to providing your users with a good UX, you would only have a partial solution without also considering the emotional state of the user.

As Goleman says, EI requires us not only to manage ourselves and our own emotions, but also to handle relationships.  The more we know, or can predict, about our counterparts, the better prepared we will be to ensure we relate to them in an effective way.

When designing a user interface, we are therefore faced with a challenging yet essential task: defining a persona, or set of personas, that captures what typical customers may be feeling before they engage with our UIs.  Some important questions we can ask to help achieve this are:

  • Is this UI typically the user’s first point of contact when they engage with our company?
  • Is this the channel of last resort?
  • Are they using our UI because they want to (for example: for fun or convenience) or out of necessity?
  • What else might the user be doing while they interact with our UI?
  • What is the typical user’s attitude toward or expectation of this UI?

Voice user interfaces have become increasingly ubiquitous in today’s modern culture.  There are a variety of VUIs that people use regularly.  We encounter hands-free GPS and phone apps in our cars.  We use Siri, Google Now, and Alexa to interact with our phones or even some household appliances.  Dictating text messages is catching on.  Meanwhile, the more traditional IVR that people phone for customer service, to check a balance, or to pay a bill, are a fact of everyday life.

When it comes to automated telephone call flows, for many people, the phone is the channel of last resort.  They have already tried to self-serve on the web, or even via chat, and now they find the only option left is to pick up a phone and talk to an agent.  Even so, most companies are unwilling to allow a customer to speak directly to a (relatively expensive) CSR, without first encouraging them to engage with an automated phone interface, where they might be able to get what they need without the dreaded transfer.  Let’s face it: not only is a transfer more expensive for the company, it is also more taxing on the user.  As callers, we would love nothing more than to complete our transaction without having to wait in a queue and explain our issues to an agent.  Undoubtedly, the psychological stakes are raised every time a user needs to engage with yet another channel of contact.

The Best User Experience

Figure 2 captures the recipe for attaining the best user experience.  We should endeavor to design our user interfaces using best practices, while at the same time taking into account the typical user’s emotional state, as well as embracing common EI principles to enhance the UI and sensitize it to the user’s condition. There is a lot more to say about this idea.  Keep an eye on our resources section for an upcoming White Paper on the same topic.  And if you happen to be attending SpeechTek 2016 this year in Washington, D.C., please stop by to see the my sunrise session (8am) on May 25.  I would love to hear from you!

Authored bY

Mike Mendelson

Throughout my 25 years of experience in the software industry, I have combined my dual passions for technology and language to build rich customer experiences. I am dedicated to improving self-awareness and emotional intelligence in myself and others, as I believe they are the underpinnings of great teams and leaders. Currently, I am leading PTP’s adoption of Holacracy, a new, self-organizing system for management.


Subscribe today to have our stories delivered directly to your inbox.

Invalid Email