Instagram recently decided to hide like counts on posts. How would you measure the success of this new feature?
Interviewee: Is it only for user posts or is this feature also applicable to advertisements?
Interviewer: Applicable to both
Interviewee: What is the new experience like?
Interviewer: Likes will be only hidden from followers but the orginal creator of a post can still view the number of likes
Interviewee: I would now like to think about 1) user personas and their pain-points, 2) feature goals and 3) feature tradeoff and then have a discussion about metrics for measuring the success of this product
Interviewer: sure, sounds good
1a) Content creators: These are users who create posts through their pictures and share them with their communities and receive encouragement through likes. They get more followers based on numbers of likes and this has some correlation with them receiving business offers i.e., a fitness enthusiast receives healthy food to advertise. Likes create positive feedback loops. It can also be demotivating to a few content creators who don't have many followers and hence don't get as many likes for a similar post. It can also be initimidating for new users to put themselves out and share something given these uber popular content creators.
1b) Content consumers: These are users who interact with the feed on their home page and support feeds through their likes. They can get influenced by the existing number of likes
2) Feature goals:
a) To reduce biasing of users
b) To reduce frustration or intimidation or insecurity for a few content creators by making the platform equal
3) Feature tradeoffs:
a) Increase in number of followers for a few content creators (not based on likes)
b) Potentially business revenue for content creators
c) Increase in number of likes (social effect)
Success of this feature:
Think of metrics that directly correlate to the goal of this feature
1) # of posts before and after launch of this feature (should go up)
2) # of likes per post before and after launch of this feature (this may go down)
3) # of DAUs (should go up)
4) overall time spent on the app (should go up)
5) # of followers/user (the inflated following may go down)
Qual survey: Customer satisfaction survery similar to NPS to measure overall sentiment of the users.
In summary, the feature goal is to democratize the platform and reduce intimidation, frustration and biasing of users, and to measure the success, I would measure active users, number of posts, time per user, and average followers/user.