Is Your Net Promoter Score Biased?

Is your Net Promoter Score a reliable metric for judging what customers think of your business, or is it biased? This is a common question that can be particularly worrying for small businesses that maintain a close relationship with their customers and clients.

For example, can you trust Net Promoter Score data if the customer who left it happens to be a friend or acquaintance? Can you trust your Net Promoter Score as a whole if you keep in close contact with a large number of your customers and clients?

Bias in NPS can sneak in through several backdoors – like who’s chosen to take the survey, when or where it was answered, or how the questions are worded. This can make a big difference in the results, tilting them in a way that might not truly reflect how customers feel.

Biases are common in surveys and sampling, and while it’s impossible to remove them entirely, there are ways to improve the quality of your data. In this post, we’ll look at the effects personal relationships can have on the accuracy, value and “bias” of your Net Promoter Score.

Why does NPS matter to businesses?

Now, you might be thinking, “Okay, but why do companies care so much about this score?” Well, NPS is like a quick pulse check on customer satisfaction and loyalty. It’s a straightforward metric that can tell a business a lot about where it stands in the eyes of its customers. Here’s why it’s a big deal:

1. Decision-Making: NPS acts as a guide for improving products and services. If a business notices a dip in its score, it might dig deeper to find out why and make necessary changes.
2. Customer Retention: Keeping customers happy is cheaper than finding new ones. NPS helps companies identify the risk of losing customers before it happens.
3. Growth Indicator: A high NPS indicates that customers are likely to bring in more business through referrals, powering the company’s growth organically.

NPS is not just about a number. It’s a snapshot of customer emotions and intentions that can influence a company’s strategy in everything from customer service to product development. Now, let’s see how biases can twist this number and what that means for businesses.

Every business has a relationship with its customers

It can be easy to write off NPS data as biased or distorted if you maintain a close professional or friendly relationship with your customers. After all, it’s easy to visualize yourself leaving a review for a friend or family member’s business that’s overly supportive and positive.

With this in mind, it’s important to consider that every business has a relationship, of some type, with its customers:

• Any agency or service business that keeps in contact with its clients through account managers or other staff has a relationship with its clients that could potentially affect its ability to collect unbiased NPS data.
• Any product-based business that’s impressed its customers through great service or an excellent product has an indirect relationship with its customers through its products that could potentially affect its ability to collect unbiased NPS data.
• Any business built around a public figure has a relationship with its customers through publicity that could potentially affect its ability to collect unbiased NPS data.

Do you see a pattern forming? Relationships are a natural part of business, and certainly not an undesirable one. Every business with satisfied or dissatisfied customers has a “bias” that could affect its data, either positively or negatively.

Positive NPS “bias” isn’t necessarily a bad thing

Another important point to note is that positive “bias,” in the sense of customers and clients with whom you have a personal relationship increasing your Net Promoter Score, isn’t necessarily a bad thing.

The reason for this is simple: if you have a positive relationship with a client or customer, they’re far more likely to remain a client or customer than if they were a stranger. Businesses will have relationships with their customers by default, and a positive one is very much a good thing.

If you manage to get closer to a certain customer, it’s likely to result in some level of positivity for your NPS. The positivity isn’t a result of inaccuracy or sampling errors — it’s a result of the strong relationship you have with that particular customer (and potentially, with others).

As long as you have a large enough audience to generate reliable, accurate data, the biases in your Net Promoter Score aren’t a bad thing. Instead, they’re an honest reflection of your client and customer relationships and the level of satisfaction customers feel from your product.

Selection bias, on the other hand, can be a real problem

Here’s a surprisingly common situation:

Your small business surveys its clients and earns a very high Net Promoter Score. After digging into the data, you realize that the high score is a result of most customers ignoring your survey and only the customers with whom you have a friendship responding.

A situation like this gives you all of the “biased” data that can inflate your NPS without any of the data from customers who aren’t connected to you. The end result is a score that’s artificially high — one that can give you an overly optimistic view of your business’s retention-related health.

Net Promoter Score is designed for accuracy. The greater the response rate of your survey, the greater the accuracy. Most authoritative NPS experts recommend a minimum response rate of 40% for business-to-consumer businesses and 60% for business-to-business enterprises.

Anything below this is far more likely to be affected by selection bias and could paint an overly positive picture of your business’s ability to satisfy and retain its customers.

This means that if your business deals with a small number of customers or clients with whom you have a personal connection, you’ll need to aim for a higher-than-average response rate to make sure your data isn’t affected by bias.

Our guides to timing your NPS survey properly and choosing the right channel to survey your customers can help you improve your response rate, reducing the likelihood of bias affecting your data.

How to reduce bias in Net Promoter Score

If you’re concerned about biased results affecting the accuracy of your Net Promoter Score, you can take several steps to reduce the effects of overly friendly survey answers and feedback:

• Reach out directly to clients and customers with whom you’re close and ask them to be as honest and objective as possible with their scores and feedback. Explain to them that you won’t feel offended or upset by a low score or negative feedback, as long as it’s an honest summary of their feelings about your product or service.
• Sample a large number of customers and clients, so that “biased” results from friends and acquaintances are balanced out by the greater amount of results from non-friendly customers.
• Instead of focusing on static NPS data, such as your score at any point in time, focus on the progress your business makes by measuring improvements in NPS. These are much less likely to be affected by biased results.

Wrap-up: Understanding and mitigating biases in NPS

Recognizing and addressing biases in your NPS is more than just a technical fix. By understanding the various biases that can skew NPS results, you can take meaningful steps to refine approaches, ensuring that the insights gained are both accurate and actionable. Let’s wrap up by having a structured look at the different types of biases and more relevant actions to consider:

1. Sampling NPS bias

What is it? This occurs when the survey participants don’t represent the broader customer base.

Why does it matter? An unrepresentative sample can overly emphasize certain customer types (such as those who have recently made a purchase or those who are more digitally savvy), distorting the NPS.

Actionable Strategy: Broaden your survey’s reach and ensure it includes different customer segments to avoid giving undue weight to specific groups.

This involves using diverse sampling methods, such as online platforms, phone interviews, and direct mail to capture different customer segments. Employ random sampling to give all customers an equal chance to participate and use stratified sampling to ensure all customer segments (like age, location, or product usage) are adequately represented. Additionally, oversampling underrepresented groups can help balance their influence on the survey results.

2. Non-response NPS bias

What is it? This NPS bias arises when the people who choose not to respond are significantly different from those who do. Typically, the most satisfied (your Promoters) or the least satisfied (your Detractors) are more likely to fill out these surveys. Those in the middle (Passives) might not bother as much, thinking their experiences aren’t noteworthy either way. 

Why does it matter? If primarily satisfied or dissatisfied customers respond, it can falsely elevate or depress your NPS.

Actionable Strategy: Increase overall response rates by simplifying the survey and incentivizing participation (like a discount code or entry into a prize draw). Let customers know how much their feedback matters and how it directly influences the services or products you provide. 

Use emails, SMS, pop-ups on your website, or even direct calls for a more representative sample of responses. Follow up with non-responders via different channels (for example email or SMS) to encourage a more balanced response. Be mindful not to spam; a couple of well-timed reminders are sufficient. 

If you know certain segments of your customer base are less likely to respond, adjust your analysis to account for this. Statistical techniques like weighting responses based on demographics can help balance the scales.

3. Cultural NPS bias

What is it? Different cultural backgrounds can lead to different interpretations and responses to the NPS question.

Why does it matter? Cultural tendencies can lead to skewed data, such as more moderate responses in some cultures and more extreme feedback in others. For example, customers from Japan might be more reserved and less likely to use extreme positive or negative scores. At the same time, those from the USA are more willing to use the full range of the scoring system, reflecting more extreme satisfaction or dissatisfaction.

Actionable Strategy: Tailor the language and even the scale of NPS surveys to better fit the cultural context of each target group. Analyze NPS scores within cultural or regional segments to understand and compare feedback more accurately. Supplement NPS with qualitative methods, like interviews or open-ended questions that allow customers to express their thoughts in a more detailed and culturally nuanced way.

4. Questionnaire NPS bias

What is it? The phrasing and order of questions can influence how respondents answer.

Why does it matter? Leading questions or a poorly structured survey can lead to biased responses.

Actionable Strategy: Neutral language and a clear question sequence are good starting points to counteract NPS bias. Be strategic about the sequence of questions to avoid influencing responses based on previous questions. Use a consistent response scale and format across all questions to avoid confusion and ensure that responses are comparable and reliable. Pilot test the survey to detect and address biases before full deployment. Regularly test and update the survey to maintain its neutrality and relevance.

Simplify your NPS process with Retently

Recognizing and addressing biases in Net Promoter Score systems is not just about improving a metric but deepening your understanding of customer loyalty and satisfaction across your entire customer base. By acknowledging and correcting for biases, businesses can ensure that their NPS data is both accurate and reflective of the true customer experience.

Take time to review your current NPS methodology. Are there potential biases that could be skewing your data? Are certain segments of your customer base underrepresented in your surveys? 

Implement necessary changes based on your audit. Leverage advanced tools and software designed for NPS management. These can automate the process of gathering and analyzing NPS data while helping you identify and correct biases in real-time.

Retently makes it easier than ever before to survey your customer base and calculate your Net Promoter Score. Instead of managing your NPS process manually, you can automate your NPS emails to generate helpful, regular feedback from your clients and customers.

Learn more about how Net Promoter Score works and its benefits for your business and start surveying your customers using Retently now.