MarketTools Blog

Often when I’m working on global customer satisfaction surveys, I come across a common concern from business leaders outside of North America.  These leaders tell me they have seen their customer satisfaction scores compare unfavorably to scores from North America, even as they do all the right things to build up customer satisfaction and loyalty.  Year-over-year, they see their scores improve, but the North America scores go even higher.  They are concerned that management sees this gap and concludes that they are doing something wrong, and sometimes this concern even leads them to be resistant to the whole survey process.

And they’re right to be concerned.

As researchers, we know that there truly are cultural differences between survey respondents, and as much as we may caution our clients that Voice of the Customer / customer loyalty scores across regions aren’t directly comparable because of these differences, it’s human nature (and a very legitimate business question) to wonder how different regions measure up against each other.

So how can we disentangle true regional differences from artificial cultural ones?  Like most things in survey research, it all starts with solid questionnaire design!

Here are a few tips for improving cross-cultural customer satisfaction surveys: 

• 1.    Label only the endpoints of the ratings scales in your survey.  Fully labeled ratings scales present a translation challenge across multiple languages which can introduce cultural bias.  Are “Excellente” and “Bueno” the same distance apart on a scale in Spanish as “Excellent” and “Good” are in English?  The fewer points you label, the less likely you are to skew your scale due to translation issues.
   
• 2.    Use even scales.  Avoid offering a scale midpoint—some cultures are predisposed toward selecting a midpoint response if it is made available. 
• 
  3.    Avoid using “Agree” and “Disagree” as the anchors of your ratings scales.  Some cultures tend toward the “agreeable” answer more than others, which can skew cross-cultural comparisons for questions using agreement scales.
   
• 4.    Use extra care in selecting metrics.  Top- and bottom-box metrics as well as NPS focus on scores at the extreme high or low ends of the scale, neglecting scores in the middle.  This can amplify cultural differences since some cultures tend to use the middle of the scale more while others tend to respond more toward the endpoints.  

  5.    Consider including a control vignette.  Control vignettes ask the respondent to respond to a specific scenario that is presumed to be interpreted identically across cultures in order to establish a cross-cultural baseline.  

  I have seen very elaborate vignettes that pose a hypothetical scenario external to the respondent; for example:  Sally is a long-time customer of Acme, is very happy with their products, and recommends Acme to her friends.  How would you rate Sally’s overall satisfaction with Acme?  Other vignette techniques ask the respondent to name a company they personally consider “Best in Class,” and then to rate that company on the same key survey metrics they are rating the subject of the study on.

  These vignette techniques allow you to establish a theoretical maximum score on your key measures for each culture you are interested in.  You can then compare your own scores to these theoretical control scores—and the amount by which each group is underperforming the theoretical max can be compared across cultures.

Don’t let fears and misconceptions about making cross cultural comparisons stop you from getting the data you need to run your business!  These techniques will help to ensure you get the best possible cross-cultural data.

► Learn more Best Practices for Customer Survey Design