IVR Analytics Best Practices Based on Customer Q&A

We routinely field questions from our customers about IVR analytics. I would like to provide you with insight by answering some of these questions for you. Understanding how often to pull data from your analytics solution and the use cases that determine this, will help you set up the right processes when implementing an IVR analytics solution.

However, before I do so, it’s necessary to look at some underlying concepts that impact how those questions might be answered. The below concepts are important to developing a schema around IVR analytics:

• Raw data collection frequency
• Raw data aggregation frequency
• Data aggregation fidelity

Raw Data Collection Frequency

This is how often you pull IVR data for processing. A common solution is to pull log files once per day (batch mode). The ideal IVR analytics solution allows direct feed (data consumed immediately upon call completion) or micro-batch mode to full batch mode access. The age of the raw data will impact how it can be reported on.

Raw Data Aggregation Frequency

Aggregation is the process where the raw data is processed and mathematically summarized according to the rules each business defines. For example, the data can be summed or it can have statistical calculations done on it. The frequency at which the data is aggregated affects when you can report on the data.  If you collect the raw data hourly, but only do the data aggregation one time per day, any report you view will be 24 hours stale by the time you are looking at the summarized data.

Data Aggregation Fidelity

This refers to the minimum time period for which data is aggregated. For example, if you summarize your total calls into calls per day then your report will only display total calls per day. You will not drill down and see total calls per hour and would have no way of reporting on calls per hour. The ideal IVR analytics solution should store the data rolled up to the hour to give the most practical amount of freedom when reporting.

Common IVR Analytics Questions

1. How often should we collect raw data?
2. On what time period should our data be aggregated?
3. What are the symptoms that indicate that we need to take action?

Answer to Question 1: Hourly Micro-batching

Our recommendation is to do hourly micro-batching. This allows you to view report data that is then one hour out of date. This approach gives companies the most flexibility.

Answer to Question 2: Comparison of Weekly vs Daily Rollups

Problems With Weekly Rollups

One day, during a weekly staff meeting, a Call Center Manager asked, “It’s strange, but we are seeing more calls on Wednesdays; can you show me what you are seeing in the IVR?” You are happy to show off your cool reports, so you quickly pull up the below “Weekly Call Counts” graph. She asks, “Can we see the daily trends?” Sadly, you cannot show her this because the data is rolled up weekly.

Why Daily Rollups Are Better?

On the other hand, if the data was rolled up daily you could show her the below graph: “Daily Call Counts.” This would allow a robust discussion on the trend(s) and allow data driven caller behavior to guide business decisions. In this case, we might decide to add more agents on Wednesdays to reduce caller wait times, hopefully improving the overall caller experience.

Answer to Question 3: When will we need data to take action?

The third question is too broad as it is stated. To help scope it, we will provide a specific example of one way in which an IVR Analytics solution could be used to solve a problem.

Once you have a rich data set to report on, you can then start making smart decisions to improve the IVR. You are once again in your weekly meeting, and the Call Center Manager is concerned that the IVR is broken because so many callers are getting transferred through to the general agents rather than the repair agents, which in turn is causing scheduling issues and an increase in labor.

Immediately everyone in the room looks at you. At this point, you can have the “deer in headlights look” or using the rich data set provided by your IVR Analytics solution, you could immediately start analysis. Your IVR is asking the caller “what is your part number?” and it is at this point that callers are branched to one of the two types of agents. So the question becomes, is the IVR functioning properly? You calmly pull up a Turn Detail Report.  This report shows you performance of the IVR at a particular dialog state.  You display a chart that looks like the below “Main Menu” graph.

It shows that a high percentage of callers are coming back with a “No Match” value from the IVR for the first two tries, fewer are coming back with a value of “Part Number,” which would then take the caller down the repair path. The “No Match” path results in the callers going to a general agent.  We now know that the IVR has a problem here and have an initial understanding as to why callers are being routed to the wrong agents. This is actionable.

The action we choose to take is to have the failed calls transcribed (a process where a human being listens to what the caller says and then types it up). These transcriptions are then analyzed.  It turns out that when the IVR asks, “what is your part number,” callers are responding with “don’t have it.” This drives us one level deeper as we examine the grammar. We discover that in the original design, we never accounted for the possibility of the customer not having the part number. The grammar is changed and the IVR is tested and redeployed. We collect data as always. A few weeks later, we run a new report and see a chart similar to the below “Main Menu” graph.

The graph shows us that we are now accounting for the callers that do not have their part numbers and are processing them accordingly. The ability to show before and after measurements is critical to being able to show value of the IVR to the business and its customers.

The ideal IVR analytics solution is designed around providing flexibility in the frequency of data collection as well as the fidelity of the reporting. Your analytics solution is an integral part of the IVR platform and provides you with actionable information to maximize your ROI as it relates to IVR performance.

Learn More about PTP’s Business & Analytics Solutions