PM in Practice #1 You have flat metrics, but user feedback is positive. What do you do?
PM in Practice is a series where I share my approach to ambiguous product decisions.
Product decisions rarely come with clean signals. This is one of those cases.
You have flat metrics, but user feedback is positive. What do you do?
The situation: You planned, built and shipped a feature which was supposed to move user engagement upwards. When you analyse the feature’s impact, you notice that your north star engagement metric for this feature has been flat but the qualitative feedback received from users has been really positive.
The dilemma: If users are giving positive feedback, then why the north star is not moving up?
What would I do?
Positive feedback without metric movement generally signifies appreciation but not demand. I would not kill this feature yet. I will hold off on optimising it immediately for the north star.
Here are few questions I would like to answer to better understan the situation and what has actually happened:
- How is the adoption of this feature? (I would compare the adoption with other similar features and see whether it is meaningfully above the baseline.)
- Are there any specific type of users (based on usage, user persona, region, etc.) who are using the feature more?
- Is there any positive or negative movement in any other engagement metrics? Will this feature lead to a 2nd or 3rd order effect on the north star?
If feature adoption is the bottleneck, then it seems only a small chunk of users used the feature and gave the positive feedback. Keeping the trade off, between shipping better adoption vs value from the feature, in mind, I would decide if we should invest further on this feature or not.
If only a specific type of users are adopting the feature well, then that means either one of their needs is getting met or they are able to understand the feature better. There is an opportunity here to double down on these users around that need.
If some other engagement metrics are moving up but not the north star, then it might lead to a 2nd or 3rd order impact on the north star. I would revisit the objective I was trying to meet and assess if this was expected? If not, then there were some feature design flaws which did not directly move the north star up. (which is okay and it happens)
These answers add better context to the situation.
It’s important to understand where the value lies and how can we double down on it. In the end, every feature is a hypothesis about where the value might exist.
Disclaimer: This disclaimer informs readers that the views, thoughts, and opinions expressed in the text belong solely to the author, and not necessarily to the author’s employer, organization, committee or other group or individual.
