Recommending Net Promoter

by Erin Bradner

I have just one quick question for you:

"Would you recommend this blog post?"

Wait, I’m ahead of myself. Net Promoter fever is getting the best of me. Technically, I should let you read this blog post before I collect the one number I need to grow my readership.

Perhaps you've already been exposed to Net Promoter fever? The symptoms are:

• A compulsion to ask, "Would you recommend this product?"
• A maniacal classification of the respondents into three categories: promoters, neutrals and detractors
• Followed by an irresistible urge to subtract the percent of promoters from the percent of detractors to get your Net Promoter Score

Last month, my colleagues and I caught the fever. It all began when we used the Net Promoter method to analyze user satisfaction with a feature in our product.

We chose Net Promoter because we wanted more than an average satisfaction score. We wanted to understand how a specific feature, the L&T feature, factored into our customers’ total product experience. Through an approach called key driver analysis - frequently used with Net Promoter - we identified the experience areas that inspire customers to actively promote our product.

How and what did we measure? We launched a survey aimed at measuring customer satisfaction with discoverability, ease-of-use and relevance of the L&T feature.

We also included rating questions on overall product quality, product value and product ease-of-use. We then calculated mean satisfaction scores for each experience variable. Satisfaction is plotted along the x-axis in the chart below. Next we ran correlations between each satisfaction score and the question:

"Would you recommend this product?"

Those correlations are plotted against the y-axis in the chart below. We call the y-axis Importance because correlation to the question, "Would you recommend this product?" is what tells us how important each experience variable is to our customers. Moreover, it tells us what is important in a way that is meaningful to both our business and our customers.

The gist of it is that no one is going to recommend a product without really liking it. When we recommend something, especially in a professional setting, we put our reputations on the line. Recommending a product is admitting we're way beyond satisfied. It signifies that we are even willing to do a little marketing & promotion on behalf of this product. This altruistic, highly credible and free promotion from enthusiastic customers is what makes the recommend question meaningful to your business, and to businesses like Autodesk. Promoters are going to encourage others to purchase our product and are likely to re-purchase. Great products that ignite promoters are the products that drive the growth of our business.

I'm in the software design group, not in the executive suite, but I had to slip in the term driving growth to decode the method I referred to earlier as key driver analysis. This is because plotting satisfaction for the L&T feature against willingness to recommend showed us that the L&T feature is lower on the y-axis relative to the other measures. Since we have determined that satisfaction with L&T is not as important as willingness to recommend; it is therefore not as important to driving growth of product sales. At least not when L&T is compared to measures higher on the y-axis like product quality and product value.

            Figure 1 - Some graph data is simulated

 

Keep in mind; this analysis is not inferring there isn’t any work to be done on the L&T feature. On the contrary, the quadrants on the graph tell us exactly which aspects of the L&T feature to work on next. If something popped up in the upper left quadrant, labeled FIX, that would be the highest priority area. I've heard this quadrant affectionately called the FIX IT YESTERDAY! quadrant. Nothing showed up here, thankfully.

Still, we plan to invest design and engineering time on the L&T feature so our next priority will come from the LEVERAGE or MAINTAIN quadrants. We'll pick L&T RELEVANCE since that's what showed up there. Discoverability and ease-of-use of the L&T feature are in the HOLD quadrant so we'll prioritize those last.

Figure 2 - Some graph data is simulated

 

Next we got curious how much the user experience contributes to a willingness to recommend the product. We knew from previous research the strongest predictors to willingness to recommend are helpful, responsive sales support (in our case, resellers), and substantial functionality at a good price (value). We ran multiple-regression and found that the variables for user experience contribute 36% to recommendation scores (n=2170). Value and reseller support accounted for another 18%. Surprised by the significant contribution of product experience variables, we ran another multiple regression on a different data set (n=1061) and found the contribution of user experience variables to be 40%.

Now let's get back to the Net Promoter score. Or…let’s not. It turns out that calculating the net promoter score isn't as important as graphing and using the key driver charts. By graphing the charts over time, we see how our investments in key areas pay out. We can watch features move from the FIX IT YESTERDAY! quadrant safely into the LEVERAGE quadrant. Inspiring more customers to promote our product is what motivates us. It's not about the score; it's about how we use it. Try it yourself.

“I highly recommend it.”

Comments

[Steve Bernstein]

Hi Erin,
Excellent to see you writing about this excellent post -- I absolutely recommend it! We did write a bit more about the “importance of importance” (http://waypointgroup.wordpress.com/2010/05/12/whyderive-importance/ and with some good debate there) for more explanation on the process. While I certainly agree that the actual Net Promoter Score (NPS) became largely unnecessary here, it was through the Net Promoter process that we found that your loyal customers (those very likely to recommend) buy subscriptions and additional Autodesk products and services, while also driving down cost of sales and marketing by referring new business. And there’s still more opportunity in engaging the silent majority (per http://waypointgroup.wordpress.com/2010/04/09/slience-is-deafenin/).

So with proper analysis and prioritization we’re quantifying the business value (ROI) of the required product improvements to get everyone on board with making the right investments: Let's hear it more analysis driving real action!

Great stuff and thanks for posting this.