Helping product teams across IBM act on NPS

 
 

Role

Lead UX Designer

Team

7 people

Timeline

3 months

 
 

Project background

 

As product teams across IBM increasingly adopted net promoter score (NPS) as a new baseline metric to measure the impact of user experience design efforts, the excitement of product managers, eager to quantify design, was abated by the concern of designers, skeptical of the approach. 

On the one side, metrics driven Product managers were excited to benchmark the ROI of design effort, a one-size fits all approach. On the other, designers and researchers were concerned about one metric encapsulating design value.

We were tasked with figuring out a better solution.

 
 

Definition

Our problem statement was to enable offering and design teams to identify and prioritize areas of opportunity through the analysis of direct user feedback collected from NPS surveys without having to read each individual comment.

Our sub-challenges included, understanding free-form user feedback to identify patterns and challenges, identifying, sharing and ranking patterns and their impact on NPS, and tracking offering team progress over time, addressing top challenges and improving NPS score.

 
 
 
 

The solution

This application will allow its users to auto-generate common themes among customers by utilizing Watson’s cognitive capabilities. AI will counteract the need to manually categorize comments to gain insights on customer needs, expediting the offering improvement process.

 
 
 

Will it help me identify performance & usability issues?

The dashboard uses Watson's Natural Language Classifier API to group free form comments into categories that the user defines. It also notifies the user of any fluctuations to NPS, and calculates the percentage impact on NPS 

 
 
 
 

Will it help me understand high level trends?

Graphical representations focused on monthly trends of NPS flux were the most popular among Offering Managers. Correlation of NPS trends with other key product data such as paid and trial permissions was also a popular visualization. 

 

 
 
 

Will it help me quickly sort through comments?

The app allows users to sort free form feedback in flight by key word and tags. Furthermore, Watson capabilities will begin learning and suggesting tags.

 
 
 
 

Will it notify me of any fluctations?

Users are notified of critical fluctuations in NPS scores and trending tags and key words.

 
 
 
 

Can I quickly identify themes?

Interactive graphs added a deeper layer of insight extraction and provided new pathways for users to engage with the free form data.

 
 
 
 
 
 

Will I be able to identify key words?

The cognitive word cloud was a new concept created as a way for users to better understand key words in the free form data to help formulate trends.

 
 
 
 
 

How we got there

As the lead user experience designer on the project, I was engaged throughout the entire process of research, discovery, ideation, building, prototyping and testing the app.  

 
 
Group 7.png

Competitive Analysis 

The research and product management team conducted research on current in market solutions from other NPS applications to find possible differentiators.

 

Journey Mapping

The team mapped out the as-is journey to better understand current pain points and opportunity areas.  

 
IBM sketches.png

Sketching

Initial workflows, sketches and concepts laid the foundation for the NPS app. 

 

Architecture & Navigation

Understanding the current architecture and ecosystem in order to recommend how to best integrate our new solutions. 

 

Prototype & test

I prototyped a lot of different data visualizations using Plotly and D3 graphs to pressure test concepts and test with users.