Avoid relying on NPS metrics alone
Progress towards achieving your strategic customer objectives can lose momentum if your NPS and other metrics appear to be good but you’re still losing customers. Or if the link between your metrics and your commercial outcomes can’t be aligned easily.
What are the reasons for this, and what can be done about it?
The limited behavioural categories of NPS
It’s often forgotten that NPS only measures customer intentions, which don’t always turn into actual behaviour. Also, the rigid nature of the 0-10 scoring and the strict cut off points below 6 and above 8 may not always indicate true detractors and promoters.
According to NPS methodology, every customer scoring a 6 is a detractor, and every customer scoring a 9 is a promoter. In reality, both could be neutral (neutrals are customers who score 7 or 8). And in a sample of hundreds of customers, there may be many borderline neutrals like this.
A deep dive analysis into customer comments, rather than their likely behaviour, might paint a different picture, showing that customers cannot easily be classified according to NPS categories of detractor, neutral or advocate
Sentiment analysis versus NPS – a real world example
A mobile telecoms client of ours had an overall NPS of 27 – driven by a generally positive experience with customer service – but we knew this positive picture didn’t match up with customer behaviour, as they were still losing customers.
We measured the data over specific time periods and noticed little correlation between NPS and the firm’s commercial results.
So, in addition to NPS, we also collected customer comments, classified them according to topic, and scored them on a five-point sentiment scale of:
1=strongly negative, 2=negative, 3=neutral, 4=positive and 5=strongly positive.
When analysing these comments, we found many cases where the sentiment expressed in the comments didn’t reflect the NPS categories at all, which called into question the customer’s likelihood to recommend.
Reasons for the disconnect
Firstly, we found that many customers with scores that categorised them as brand detractors gave favourable comments – typically associated with a good customer service experience after an empathetic customer service agent had solved their problems.
When customer comments contain positive sentiment like this, it’s safe to assume they represent customers who’ve been ‘recovered’, and the ‘detractor’ label is no longer a reliable indicator of their disloyalty.
Secondly when we looked at NPS promoters, we often found comments that revealed underlying issues or problems, but the score was high because customer service had been good.
Improve engagement monitoring with a sentiment index
Given what we’ve observed, we’d argue that a customer sentiment index calculated from the positive and negative comments from surveys, social media and review websites is a highly useful way of monitoring CX performance.
The net sentiment index would be calculated in a similar way to NPS – the percentage of positive sentiment customers minus the percent of negative sentiment customers – resulting in a good measure for emotional engagement with the brand.
Loyalty is formed from emotional connection, so it follows that emotional engagement, as measured by sentiment, is a strong indicator of customer behaviour and should complement your other quantitative metrics such as NPS.
Use bottom-up analysis to plan and prioritise improvement
Returning to our mobile telecoms client, we used topic sentiment analysis to not only identify some weak customer journeys across the company’s website, app, billing and tariff-changing process, but to discover the issues and pinpoint the root cause.
We did this by analysing each customer comment to determine the topic, and then assigning a sentiment score according to the strength of feeling expressed in the comment.
The result was a hierarchical list of topics based on volume, and for each topic, the balance of sentiment. From this information, our client could prioritise the high volume, strongly negative topics, and dig deeper to find what the real problem was.
The customer journey issues weren’t evident from the NPS alone, and if we’d stopped short of sentiment analysis, we wouldn’t have learnt what was really driving the customer experience – it was the customer comments that contained the detail to diagnose root cause.
Useful sentiment tools
Ideally, the manual coding of customer comments, to define the topic and assign a sentiment score, will give you the most accurate results.
This may be difficult if you have a big sample sizes, or if the sheer volume of data from social media and review websites is hard to handle – in which case an automated system is more efficient, especially for topic categorisation.
Just be aware of the natural language processing algorithms used in automated systems, which can struggle to understand tone of voice and mean that sentiment results won’t always be 100% accurate.
We’ve found that automated text analytics is good for developing the topic hierarchy, leaving a more manageable manual exercise to assign sentiment to the high-ranking topics.
Either way, there’s a depth of insight contained in customer comments that can help you focus on achieving your strategic CX objectives - and resolve any contradictions you have with NPS.