Sentiment Index: a true measure of your customer’s voice

2020-06-25| Mike Cuell

As companies the world over realise the positive impact customer-centricity can have on commercial performance, more and more of them are turning to the increasingly popular Customer Experience (CX) measure of Net Promoter Score (NPS).

NPS measures how willing a customer is to recommend your products or services to others. And with executives seeing it as a logical gauge of overall customer satisfaction and loyalty, NPS has been widely adopted.

If you’ve not come across NPS before then here’s a quick overview. NPS is assessed through one single survey question: “On a scale of 0 to 10, how likely is it that you would recommend our product / service to a friend or colleague?”

Those scoring 9 or 10 are ‘brand promoters’, who are typically loyal, enthusiastic repeat customers. Those answering with a score of 0 to 6 are ‘brand detractors’, who are in danger of being lost as customers and potentially bad-mouthing the company.

Your NPS is calculated by subtracting the percentage of detractors from the percentage of promotors. Quick, simple, efficient.

With such an easy way of measuring your customer’s view of your products or services you’d think that achieving a higher NPS would always result in an increase to your bottom line, right?

Well, not always. 

Avoid relying on NPS metrics alone

Progress towards achieving your strategic customer objectives can lose momentum if your NPS and other metrics appear to be good but you’re still losing customers. Or if the link between your metrics and your commercial outcomes can’t be aligned easily.

What are the reasons for this, and what can be done about it?

The limited behavioural categories of NPS

It’s often forgotten that NPS only measures customer intentions, which don’t always turn into actual behaviour. Also, the rigid nature of the 0-10 scoring and the strict cut off points below 6 and above 8 may not always indicate true detractors and promoters.

According to NPS methodology, every customer scoring a 6 is a detractor, and every customer scoring a 9 is a promoter. In reality, both could be neutral (neutrals are customers who score 7 or 8). And in a sample of hundreds of customers, there may be many borderline neutrals like this.

A deep dive analysis into customer comments, rather than their likely behaviour, might paint a different picture, showing that customers cannot easily be classified according to NPS categories of detractor, neutral or advocate  

Sentiment analysis versus NPS – a real world example

A mobile telecoms client of ours had an overall NPS of 27 – driven by a generally positive experience with customer service – but we knew this positive picture didn’t match up with customer behaviour, as they were still losing customers.

We measured the data over specific time periods and noticed little correlation between NPS and the firm’s commercial results. 

So, in addition to NPS, we also collected customer comments, classified them according to topic, and scored them on a five-point sentiment scale of:

 1=strongly negative, 2=negative, 3=neutral, 4=positive and 5=strongly positive.

When analysing these comments, we found many cases where the sentiment expressed in the comments didn’t reflect the NPS categories at all, which called into question the customer’s likelihood to recommend.

Reasons for the disconnect 

Firstly, we found that many customers with scores that categorised them as brand detractors gave favourable comments – typically associated with a good customer service experience after an empathetic customer service agent had solved their problems.

When customer comments contain positive sentiment like this, it’s safe to assume they represent customers who’ve been ‘recovered’, and the ‘detractor’ label is no longer a reliable indicator of their disloyalty.

Secondly when we looked at NPS promoters, we often found comments that revealed underlying issues or problems, but the score was high because customer service had been good.

Improve engagement monitoring with a sentiment index

Given what we’ve observed, we’d argue that a customer sentiment index calculated from the positive and negative comments from surveys, social media and review websites is a highly useful way of monitoring CX performance.

The net sentiment index would be calculated in a similar way to NPS – the percentage of positive sentiment customers minus the percent of negative sentiment customers – resulting in a good measure for emotional engagement with the brand. 

Loyalty is formed from emotional connection, so it follows that emotional engagement, as measured by sentiment, is a strong indicator of customer behaviour and should complement your other quantitative metrics such as NPS.

Use bottom-up analysis to plan and prioritise improvement

Returning to our mobile telecoms client, we used topic sentiment analysis to not only identify some weak customer journeys across the company’s website, app, billing and tariff-changing process, but to discover the issues and pinpoint the root cause.

We did this by analysing each customer comment to determine the topic, and then assigning a sentiment score according to the strength of feeling expressed in the comment.

The result was a hierarchical list of topics based on volume, and for each topic, the balance of sentiment. From this information, our client could prioritise the high volume, strongly negative topics, and dig deeper to find what the real problem was.

The customer journey issues weren’t evident from the NPS alone, and if we’d stopped short of sentiment analysis, we wouldn’t have learnt what was really driving the customer experience – it was the customer comments that contained the detail to diagnose root cause.

Useful sentiment tools

Ideally, the manual coding of customer comments, to define the topic and assign a sentiment score, will give you the most accurate results.

This may be difficult if you have a big sample sizes, or if the sheer volume of data from social media and review websites is hard to handle – in which case an automated system is more efficient, especially for topic categorisation.

Just be aware of the natural language processing algorithms used in automated systems, which can struggle to understand tone of voice and mean that sentiment results won’t always be 100% accurate.

We’ve found that automated text analytics is good for developing the topic hierarchy, leaving a more manageable manual exercise to assign sentiment to the high-ranking topics.

Either way, there’s a depth of insight contained in customer comments that can help you focus on achieving your strategic CX objectives - and resolve any contradictions you have with NPS.

Feel free to contact Neil Sharp or Ben Donaghue if you have any questions or if you'd like to find out more about how we can help you transform your organisation through CX measurement.

Do you want to learn how you can improve Customer Experience in your organisation?

Enter your details to download our free eBook that walks you through the exact framework we use with our clients to help them become more customer centric