Introduction
To get the best insights from Net Promoter surveys, you have to read your customers’ responses to the open-ended “What is the primary reason for the score you just gave us?” question. We call this “verbatim feedback” or just “verbatims”.
To turn your verbatims into actionable insights, you have to categorise the feedback so you can see what themes are being mentioned most by your customers. In that way you’ll learn what you are doing well that your customers love, and what you’re not doing well that exasperates them.
The most effective way of categorising verbatims is to do it manually in Excel.
Word cloud apps won’t cut it because they just look for word count frequency. You need a human to understand that…
“I wish you had solved my problem quicker”
… is the same thing as…
“You were too slow to fix my issue”
… and the opposite of
“You couldn’t have fixed my problem any quicker! Thank you!”
Machine learning/AI software is getting close to being able to do a pretty good job at identifying themes, but it’s not there yet. Until then, here’s you can do it in manually.
Step 1 - Tagging
To identify themes, load your scores and verbatim feedback into Excel. You can do this using the Survey Responses report, specifying the format=csv parameter.
The two main columns in your spreadsheet would look like this:
| Overall_Sat | Comments |
|---|---|
| 10 | You guys rock |
| 8 | The new self-service portal is really easy to use |
You will also want to keep the other columns, e.g. the Team (Support Group) and Segment (Customer Segment). Later, this will enable you to understand where themes might apply to specific teams or customer segments.
So your table would look something like this:
| Overall_Sat | Comments | Team | Segment |
|---|---|---|---|
| 10 | You guys rock |
SAP | Sales |
| 8 | The new self-service portal is really easy to use |
Service Desk | Head Office |
To keep things simple, we won’t show those extra columns in the rest of this article.
Let’s add some more feedback to our example.
| Overall_Sat | Comments |
|---|---|
| 10 | You guys rock |
| 8 | The new self-service portal is really easy to use |
| 8 | Sam was super friendly. I just wish my problem could have been solved quicker |
| 5 | After I logged my ticket it seemed to disappear into a black hole and heard nothing about it for months! |
| 3 | The whole experience was terrible |
| 7 | Seemed like I had to wait a long time for my issue to be fixed. It would have been better if someone had told me how long I was going to have to wait |
Before beginning to tag the feedback, remove feedback that is not specific enough to derive insights from. In this example, we'd ignore “You guys rock” and “The whole experience was terrible”. Maybe stick “You guys rock” on a post-it note on the fridge ;)
As you read through the feedback, add columns for each theme that emerges. When that theme is mentioned, put a “1” in the corresponding column, like this:
| Overall_Sat | Comments | Portal Usability |
|---|---|---|
| 8 | The new self-service portal is really easy to use |
1 |
Often, you’ll find one verbatim talks about multiple things. For example, “Sam was super friendly. I just wish my problem could have been solved quicker” refers to two separate things:
1. Sam was friendly.
2. The ticket took too long to solve.
In this case, add two columns to your spreadsheet, one for each theme and put a “1” in each.
| Overall_Sat | Comments | Friendly |
Portal Usability |
Slow Resolution |
|---|---|---|---|---|
| 8 | The new self-service portal is really easy to use |
1 | ||
| 8 | Sam was super friendly. I just wish my problem could have been solved quicker |
1 | 1 |
Because you’ll be tracking themes that are both good and bad, make sure your good and bad themes are clearly differentiated. We like to call positive themes, “Reinforcement” themes. And the stuff we need to improve, we call “Corrective” themes.
This will make sure you don't mix-up easy-to-use vs hard-to-use ‘Portal Usability’ themes!
By the time you’ve tagged the remainder of your feedback, you’ll have something like this:
| Overall_Sat | Comments | Reinforcement - Friendly |
Reinforcement - Portal Usability |
Corrective - Slow Resolution |
Corrective - Insufficient comms |
|---|---|---|---|---|---|
| 8 | The new self-service portal is really easy to use |
1 | |||
| 8 | Sam was super friendly. I just wish my problem could have been solved quicker |
1 | 1 | ||
| 5 | After I logged my ticket it seemed to disappear into a black hole and heard nothing about it for months! |
1 | 1 | ||
| 7 | Seemed like I had to wait a long time for my issue to be fixed. It would have been better if someone had told me how long I was going to have to wait |
1 | 1 |
Rather than creating themes (adding columns) 'on the fly', you might want to try reading through a random sample of the verbatims, say 10%, and creating an initial set of themes based on that. You can then revise those themes as you code the full set of responses.
What is the right number of themes? Aim for something like 10-20 for Constructive and the same for Corrective.
If you have too few themes you won't get any insights. To use an exaggerated example, a one-theme analysis of transactional surveys would be "Unhappy with support experience". What can you learn from that?!
With too many themes, no themes are going to stand out.. All your themes will have the same, relatively low counts. Another exaggerated example. "Portal difficult to navigate", "Incident form too big", "Form validation messages unclear" would be too granular. You'd be better off with "Portal Usability".
With transactional surveys, you will tend to find the same themes occurring over and over. There's probably no IT support survey in the world that won't have a "Too slow to resolve" theme in their verbatims! Over time, the counts will change but the themes are likely to remain relatively stable.
We therefore recommend each new analysis starts with the same set of themes that you used last time. Not only does this speed up each subsequent analysis, but you will also be able to see if the improvements you're making are having the desired effect. E.g. Between January and June, 20% of Detractors cited Portal Usability as an issue. July to December, after the Portal Upgrade Project, Portal Usability was only mentioned by 5%.
Step 2 - Analysis
Identifying the themes that are most important to your customers is simply a case of counting the tags. The tags with the highest counts show you where you’re strong and where you need to improve.
To really understand your strengths, count the Reinforce tags just for Promoters. And to understand where you need to improve, count the Corrective tags for Detractors and Passives.
You might end up with something like this:
| Reinforcement Themes - Promoters |
Count |
|---|---|
| Friendly |
125 |
| Portal Usability |
70 |
| Self-service Software Installs |
40 |
| Weekend Support |
30 |
| Genius Bar |
30 |
| Corrective Themes - Passives & Detractors |
Count |
|---|---|
| Slow Resolution | 210 |
| Insufficient Comms |
190 |
| Long Wait for New Hardware |
110 |
| Access Request Forms Too Long |
50 |
| Support Hours Too Short |
20 |
This gives you a pretty clear view of what you’re doing right (friendly staff, and your new portal) and what you need to do to create more Promoters (speed up resolution times, better communication on ticket statuses).
By using the additional columns in your spreadsheet (e.g. Team and Segment), you will be able to get even more detailed insights.
For example, if you knew that Sales had the lowest Net Promoter Score of all your Customer Segments, you could repeat the above analysis just using feedback from customers who work in Sales. The resulting analysis would then tell you which factors you have to improve to make Sales happier. Similarly, you could repeat the analysis for a single Team to learn what that particular team's strengths and weaknesses are.
Finally, although we’ve used feedback from a transactional survey in this example, the same principles apply if you were analysing feedback from Relationship Surveys, e.g. about the whole of IT (great for informing IT strategy) or about a single project (great for improving future project capability).
Step 3 - Communicate & Act
Once you’ve done your analysis you’ll know where you need to improve.
You can then prioritise your service improvement initiatives taking into account other relevant factors such as cost, resource availability and dependencies.
Importantly, don’t forget to communicate with your customers using whatever channels you have available to you.
We recommend you communicate three messages at three different times:
- Thanks for your feedback, here’s what we’ve learned from it (good and bad).
- Here’s what we’ve got planned as a result of your feedback (to build on the good and address the bad).
- Here’s what we’ve done as a result of your feedback (show you’ve delivered what you said you would).
Net Promoter surveys are customer friendly (super short) and deliver actionable insights when you mine the verbatims. Although it can take a bit of time, it’s simple. Customer feedback is pure gold if you want to improve service. Your customers are like an army of free consultants. Go use them!