Powerful Continuous Discovery Program: 6 Key Insights from Product Managers at Docusign, Linkedin, and Zapier

In our second experts’ roundtable, we had the pleasure of having three product management experts with an extensive experience with product management and product discovery discussing the secrets of building a continuous discovery program.

Here are the main takeaways from this high-level discussion about continuously listening to customers to make product decisions correctly. 

1. A continuous discovery program requires clear outcomes, data, and collective collaboration

When thinking about the key elements that make a continuous discovery program successful, the first thing to be aware of is that while product discovery is about using data to validate assumptions of solutions to problems, continuous discovery is constantly doing that to measure if you’re on the right track or not.

According to Lisa Orr, whose background is in data science, continuous product discovery is nothing more than “gathering evidence and asking deep questions on if you’re on the right track or not.” For that to happen, it’s mandatory that you have a clear definition of success and that you empower your product teams with access to data that will help you measure the criteria.

After reinforcing the importance of data for discovery, Naren Raghavan added that is critical to acknowledge that discovery goes beyond product: you need to involve design, engineering, marketing, sales, and everybody in the organization at some level. According to him, “if you put those two pieces together that’s when you can really start thinking about a Discovery program and building on top of that”.

Shyvee added saying that, for her, the continuous discovery aspect is actually about continuing to build that conviction of “okay, we’re heading on down the right path and solving the right problem. It’s a conviction confidence-building exercise that you keep checking in.”

2. Continuous discovery is like exercising: it needs to become part of your routine

One of the core aspects of a continuous discovery program is ensuring that you really make it iterant. The premise is that your users’ needs are always evolving and so should be your product, and that requires a constant review of your goals, your results, and what users expect and how they feel about it. 

The most common mistake companies make is not turning continuous discovery into a routine inside the organization – making it really part of the process. What happens is that, when they get busy with something else, they stop the discovery process and just get back to it when they have a new idea to validate. 

The key here is, according to Shyvee Shi, making discovery a true habit – almost like exercising: “Being able to, on an ongoing basis, understand what’s going on is a new muscle that you need to exercise, and it takes a lot of activation energy. It’s definitely easier said than done.” That is especially true in a context like today’s, where everything is shifting so fast and where companies have limited resources and time.

3. Having a user feedback repository is a continuous discovery program must-have

In order to build continuous discovery habits, a product team must define some cornerstone habits that go beyond the “interviewing users every week” suggestion from Teresa Torres. One thing is documenting UX research and interviews to build a library of knowledge that can later be used by current and future teams. “Every time you onboard someone there’s still some level of learning curve, but I think documentation is a number one habit from a process standpoint.”, said Shi. 

But it goes beyond that: there is a difference between data and the insights you get from the data, and you should have access to both. All the panelists agreed that documenting and centralizing the insights from a conversation is important, but different people can get to different conclusions when looking at the same data in different ways or moments; they can use it for new analyses, to quickly validate new ideas, to do some additional exploration or even to get the evidence you need to justify a finding or decision. 

Naren recommends teams to have a centralized feedback river where you can take notes and share insights, but also get access to the raw data in an organized way: “having all the data in one place where people can come access makes the learning curve much easier, but the beautiful part of it is that different people can come to different conclusions using the same data”. 

The need for user feedback data and documentation is even higher in more mature cultures of discovery, as having evidence will be required by leadership at some point and will naturally be used by product teams when they want to pitch something they put in the roadmap.

4. Successful continuous discovery programs use data to reduce uncertainty without slowing down

According to Tim Herbig, Product Discovery is about having data to reduce uncertainty when making decisions. Although it’s not a linear process, it’s also not something that entitles you to spend as much time as you want on it, as it’s likely impossible to eliminate the uncertainty. 

When asked about when it’s the right time to move to the stage of prioritization, Lisa answered that it’s when you have the confidence that you’ve covered the risk that you’ve identified is a really important factor. She mentioned an example of COVID passports, saying that “it’s all about have you reached critical mass on covering the important risky assumptions and I you know, especially with a new product where time-to-market is so critical that you can’t spend too much time in Discovery before making your first bet.”

Naren added that there is a lot of judgment and product intuition involved as well to define how big a discovery should be, as you don’t always have the time to run a full process. For him, when a hypothesis is a reversible decision then you can move faster and make that a little bit more quickly. That’s where having available data to support you can be handy: “think about it as a confidence interval – you have to be able to make a confident decision, and if you have enough user data and feedback to make that decision and justify and communicate that to all of your stakeholders, you should be able to do it.”

For Shyvee, looking at patterns and trends in the data is critical. Most of the time the insights won’t be completely new and will likely reinforce something you discovered before, which is great and shows that you did a great job, and maybe 20% will be something new that you can explore. But it still needs to be continuous so you can identify when that happens: ”we do monthly calls reviewing support tickets that we can always spill on top like what’s changing from last period in this period to constantly understand what is going on.”

5. Don’t get stuck in the complexity trap

One of the significant risks with this process is trying to put a perfect process in place from day one. The best way to start is to go back to the first principles and show people the importance of data and how a data-driven decision can bring you the results you want. 

Naren’s suggestion is to pick a small project to start small and show some wins quickly: “you don’t want to say something is only valuable if you have every single piece of information to put it together and tell a story because then you’re gonna get into analysis paralysis”. He recommends teams to get something implemented and get some data to build from there to then show how that data enabled you to make a better decision so you can start finding the new data sources you need and incrementally build from there.

To add to that, Shyvee reinforced that continuous discovery is a way to validate and learn as much as fast as possible, which can be different in early stages versus more mature products. She circled back to the importance of building the discipline of aggregating all the different things you hear from the field to support your decisions, exemplifying that “being able to quantify things like 100 support cases being related to feature A versus 75 related to feature A so you don’t get biased over recency. Or if a big customer says something but if you look at historical data only five customers have mentioned that.”

Lisa complemented saying “I will always aim to action versus wait for a little bit more way to validate more”. For her, combining user behavior data with qualitative feedback is an important piece in the process of looking at your North Star metric and creating your own metrics tree in order to help you prove the value of what you’re doing. For that, she suggests what she calls a fun exercise, which is agreeing on which product metrics will impact a company metric, such as revenue: “when you can agree upon if we move this lever it will drive potentially this much revenue then you can get a lot of excitement behind you from leadership and from fellow product managers”. She also adds that having the evidence that “not just I’m making a customer happy, I’m making them happy and driving revenue for the entire company” is an ideal outcome from a great continuous discovery program. 

6. UX Researchers should support your product discovery program

Not every organization has the privilege of having a UX Research team to help them do product discovery, but they should be aware of the risk of overlapping functions when that happens. An even worse risk having UX as the only team talking to customers and feeding Product with their interpretation or insights from the conversation. 

One suggestion to avoid the issue of having two teams doing similar things that came from Lisa was splitting things up. In her case, they had the product trio focused on the near-term while UX research focuses on big questions: “we have this vision for where we’re going and she’s tackling the big risky questions about that. She’s even tapping into going to the market trying to talk to people who have experienced solutions that we’re aiming towards.” 

For Shyvee, user research comes in two primary forms: foundational research, exploring the user journey, jobs to be done, and the market, and usability studies, where having the skills to ask the right questions.

If you’d like to dive deeper, you can view the full recording. And if you’re ready to build a discovery program that is based on evidence from all sources of user feedback, visit our product page.