Creating an effective customer feedback tagging taxonomy model for product teams

While product analytics has become a critical piece of product management in the past years, structured data is only a piece of the puzzle when it comes to understanding users — less than 20% according to Gartner. 

Unstructured data, which comes in form of support tickets, NPS survey responses, user interviews, and more, is not only the larger piece but is also growing 3x faster. More important than that, these sources normally bring more insights into customer satisfaction, desires, expectations, and the “why” of user behavior. 

But tapping into the value of these types of product data is not easy, and it requires establishing a process that organizes all that unstructured data from customer feedback into themes that can then be analyzed and understood. 

Enters taxonomy tagging for customer feedback analysis.

What is a customer feedback tagging taxonomy and why is it important for product teams?

Taxonomy is the practice of categorizing things according to a hierarchical classification that makes it easy to interpret and analyze them; tagging taxonomy is nothing more than using tags to classify elements according to a defined taxonomy. 

There are multiple applications for taxonomy tagging, one of them being the categorization of customer feedback. Categorizing customer feedback from multiple sources is important to quantify what users are saying and identify the most important themes that arise from their comments –  what matters the most to them.

What kind of answers can a tagging taxonomy applied to customer feedback analysis give?

It’s important to consider that the main goal of your taxonomy hierarchy is to help you to build – and continuously improve – products by listening to your customer complaints and expectations in a Customer Feedback Management process. Some of the answers that a well-established process can help you answer are: 

• What issues keep surfacing repeatedly?  What issues are trending?
  • “The app is not able to save my language preferences. Regardless of what I set, the movie is always played in English.”
• What features are missing according to lost or churned customers?
  • “I’ve been using this app while walking and just listening with my phone in pocket. I just learned each video uses 1 GB of data. Total waste of data and battery power. I’ll probably have to cancel.”
• What are the main opportunities I have to increase user retention?
  • “I’d really like to have the option of watching the movies faster sometimes, like have in other streaming platforms”
• What do users think of feature X? 
• And many others.

Common challenges in creating a tagging taxonomy

Everybody knows about the importance of analyzing customer feedback. A good tagging taxonomy can help PMs identify untapped issues, needs, and opportunities to be explored – the cornerstone of customer feedback analytics.

Categorizing customer feedback is more complex than it seems – we’re talking about user feedback in many different forms, multiple languages, and coming from an ever-growing number of sources. Organizing them in a useful way is key.

These are some of the common challenges companies face when trying to uplevel their customer feedback analysis processes by tagging pieces of feedback:

• Users talk about the same thing using different keywords – when talking about a login experience, people can mention login, log in, logging in, access, accessing, etc. Different keywords and variations need to be mapped and nested under a topic. 
• Users talk about different things using the same keywords – a person can say they are trying to “store” something – a feature -, that the app is not working at a “store” – another feature -, or that they can’t find a “store” – another feature. The context is important and needs to be taken into consideration.
• Users talk about different things or aspects of the same experience – while one person can be complaining about the “forgot password” experience, another can be saying that they wanted to have a SSO log in. These are different expectations around the same element, and they need to be grouped together.
• Users always say new things, and in different ways – as your product evolves, user experience algo evolves, and what and how they say changes. It’s impossible to predict everything that is coming. Feedback is a living thing — and so should be your way to manage it.

All of these challenges grow exponentially as your database of users grows, which leads to a key challenge: scaling with consistency. With more feedback to be read and categorized, you’ll likely need to have more people doing the job. Whether it’s support agents, product ops, or sales reps, these people’s jobs are not tagging feedback, and they have different perspectives and interpretations of feedback. Maintaining consistency and quality becomes nearly impossible. 

But don’t worry: we’re here to help. In this post, we will share some insights on building a tagging taxonomy, using it to classify feedback, and evolving it in a way that adds true value to your product management process. Let’s start.

The foundations of a customer feedback tagging taxonomy model

To start, it’s important to align on a few things and concepts. First, there’s no right or wrong way to start annotating user feedback towards an effective Customer Feedback Analysis process. Second, you need to organize Keywords (words or groups of words that are part of a feedback) according to Topics (relevant groups of themes that are relevant to your analysis process). 
A topic can have subtopics and normally has multiple keywords associated with it; the combination of multiple keywords arranged according to multiple topics builds what we like to call a Knowledge Tree. All of that can relate to your product features, outcomes, and hypotheses.

1. Defining your Customer Feedback Taxonomy Hierarchy

As we stated before, a tagging taxonomy exists to support your customer feedback analysis process. It may be simple or complex – and the hierarchy and granularity that you decide to use will have a direct impact on your results and may vary depending on the nature of your product or the stage of your business.

Some products might require just a dozen topics for a PM to answer a business questions, but that is not the case for most of the times: to get to the root cause of something, you will need to create levels (groups) of topics, building what we like to call a taxonomy hierarchy. When doing that, you have two options for organizing the data:

• Using a bottoms-up approach: in this case, you can start mapping the concepts, topics and descriptions which are important for your business and, once finished, group them as needed according to similarity, responsibility, user journey, desired outcome, or other criteria. Security issues, for instance, are traditionally related to Authentication (assuring the identity of incoming sessions/users), Authorization (controlling the type of access each user/session has over each piece of information), and Accounting (tracking every change/action made by authorized or unauthorized users). Each of these level 2 topics can have user feedback associated with them and later be grouped into a level 1 topic like Security. 

• Using a top-down approach: on an opposite end, you can start with the criteria of what are the most important levels to later break them down into smaller features, building blocks, or experience elements. Let’s say you have one topic that groups all feedback about the usability of your product. After tagging your feedback you can notice that USABILITY is a larger concept that encompasses design (as planned by your designers and developed by your Engineers), experience of use (as the user sees it), operability (how easy or difficult it is to perform tasks), learnability, etc. In that case, you will need to split one giant topic (level 1) into smaller pieces to get a better and more detailed picture of what is the issue or opportunity.

How to produce an initial set of Topics?

• Product-driven approach: Map the subjects that matters most to your product strategy into the most appropriate topic, according to its definition.
• Feedback-driven approach: Listen to your audience and identify the subjects they comment most on, mapping them into the most appropriate topic
• Combination of both: Starting from your product-driven approach (product taxonomy) and completing it based on findings you see on your feedback.
• Multi-classification is a powerful capability, in the sense that it allows you to represent the multi dimensions embedded in your feedback and in each part of it. 

How to reach an optimum group of Topics?

Sooner or later, you will question yourself:

• Why do I have all these topics? Can they be reduced without losing information? Are they clearly representing all the subjects addressed by the feedback?
• Is there any Topic whose definition intersects with another existing one and by consequence could be merged or even discarded?
• What are the ones deserving to be split because they group too many subjects?
• What are the ones deserving to be created to accommodate important subjects not created yet?

Answering these questions will help you to come up with a streamlined set of Topics you can start using for tagging (extraction and topic association), like in the example below – which illustrates a generic app tagging taxonomy.

2. Classifying feedback 

If you are doing Customer Feedback Analysis in a spreadsheet, you will need a staff very skilled in worksheets to search, find, tag etc. 

In our experience, the most difficult part is not reaching a first draft of classified feedback. The rubber hits the road when you need to move keywords from one topic to another, split topics, merge topics. Managing a taxonomy when there are hundreds of keywords involved is not easy and requires caution and the right tools to support you. 

To help you get started, we’ve created a Google Docs template for feedback analysis that you can access for free here. It contains a basic taxonomy and a structure to help you classify and analyze feedback.

But that is just a starting point: you’ll need to use your product and market expertise to create new topics that make sense to your reality. And that will also change, as customer feedback is live and constantly changing, whether by people saying the same thing in different ways of saying completely new things.

Tagging keywords according to topics is a constant part of the job, and a really important one. That’s what will keep your taxonomy model running and provide the data input for analysis. Some important things to consider when tagging keywords according to different topics: 

• Avoid using homographs (words with same writing but different meanings) – in cases like that, try to tag a longer phrase to add context
• Don’t tag every word – only the ones that are relevant to your product
• Don’t create different topics for positive and negative experiences with the same topic
• Create a glossary of topics with descriptions and meanings for each one

Assuming that each topic has an associated description to define what it should include, anyone can collaborate with the annotation task following the same criteria. Also, such a description should reduce misclassification and collaborate with uniformity. There are two main ways of classifying feedback:

Feedback-driven way 

In this way, you need to find relevant keywords that matter for your strategy/purpose in every review, select them and associate them to existing topics you have previously created. Repeat this process until you have tagged the most relevant keywords-subject existing in your feedback.

Although this process gives you a lot of clarity of the context for each specific feedback, it is less scalable and very easy to lose the big picture and create new topics in a way that will damage your taxonomy.

Topic-driven way

When starting from the topic, you focus on understanding the concept and meaning of a topic (or group of topics). From there, you start thinking of different verbatim that could be associated with it to classify these keywords.You can leverage query/filtering capabilities to find and select the terms you want to transform into keywords and associate with the topic you are working on. Once you’ve exhausted the keywords for that topic, you can move to the next one.

This process is more scalable and organized as it will allow you to tag multiple pieces of feedback at once, but might leave some “blind spots” in your process as you’re just trying to look for keywords that you’ve thought of.

3. Maintaining and evolving your tagging taxonomy library 

As stated before, a good tagging taxonomy is constantly evolving – just like your product. 

Your taxonomy needs to be reviewed as you launch new features, new products, or have different priorities, outcomes, and metrics. It also needs to be reviewed just given the fact that users are having different experiences and sharing new feedback. And, of course, all of that needs to make sense for the end goal of adding value to your product prioritization and decision-making. 

Ideally you will want to continuously evolve your taxonomy along with the feedback you’re receiving. As that is easier said than done, a good way to manage this process is defining milestones of when you want to review what you have in place. It can be based on a timeframe (every six months), on a product milestone (new geography, new feature, new persona), on a metric or context change (need to increase retention), or when a topic starts to become so broad that it becomes hard to get to its root cause or generate some insight without reading a lot of feedback. In those situations, you will want to define a process to review your existing tagging taxonomy and defining what else should be created. 

A good way to approach a continuous customer feedback tagging taxonomy is combining both the ways of classifying feedback previously mentioned: using the topic-driven way to categorize most of the feedback and the feedback-driven way every once in a while to audit/analyze some pieces of content and ensure you’re not leaving important things aside. This will allow you to have both coverage and scale and maintain a live structure.

4. Scaling your process 

No matter how organized and streamlined your process of feedback tagging is, we all can agree that this is not scalable, especially when your volume of feedback starts increasing – your team probably have more strategic things to do than reading and categorizing millions of user comments manually. Not to mention that this does not give you all the benefits of a mature Customer Feedback Management System mentioned by Reforge.

So how to scale your process?

AI-based feedback intelligence platforms are the answer. They not only automate the process of centralizing feedback but also help you find opinions, extract keywords, identify synonyms, classify sentiment and associate these features with one or more topics of your Taxonomy. Some of the benefits of these tools are:

• Automatic classification – new incoming feedback will be automatically classified based on existing examples. That means that only terms matching existing keywords already associated with a topic will be automatically classified.
• Pre-existing templates & topic customization – some tools already have taxonomy templates for different use cases and industries, which gives you a starting point. At the same time, they allow you to customize it to the specific aspects of your industry segment, product or service to make it really valuable.
• Powerful search, filter, and classification functions –  feedback analytics tools allow you to search and filter for keywords and topics and classify them in bulk.
• Smart suggestions – AI-based tools may help you to find keywords not classified yet, providing you with suggestions to add to each topic.

Here at Birdie we are diligently working to provide you with a platform supporting all these needs so you can smoothly elevate your Customer Feedback Analysis process and increase your span inside the organization:

• Templates to facilitate the analysis of your feedback and serve as a reference
• AI-based models to extract aspects and statements, apply sentiment and help you with the initial classification through suggestions
• A rich set of functions to customize and manage this knowledge tree according your business needs
• And more

If you’re interested in seeing in action, join our waitlist.