You are the PM for Facebook, Instagram. You want to add new "Mute showing food" feature. How do you measure success?

1. Clarification:

I'd like to better understand what are we talking about: I'm assuming the intention of the feature is to allow users to block images and video (let's call it media) related to food - in their newsfeed? Yes.

Since "mute" sometimes refers to turning-off notifications only, I'll assume we're NOT talking about muting notifications, but about actually not showing the posts? Yes.

I'm assuming this is the first category we're introducing for people to be able to block, probably based on the number of such requests. Yes.

2. Define the goal / alignment to mission:

Let me think how that aligns to the mission - "give people the power to build community and bring the world closer together". This feature aligns to the mission by preventing the opposite from happening - if users are annoyed by a certain type of content, or find it irrelevant, pushing such content to them would contradict the mission, so we want to prevent that from happening. 

The goal, therefore, would be to address negative metrics, namely - help people curate their newsfeed to prevent churn. The freed up real estate would still be filled with other types of content, so we need to make sure that it's more relevant and interesting to the user, so another goal would be to increase engagement.

It's important to note that this feature is a little different from "positive" features, which we usually would want to grow the usage of. In this instance, we only want to grow the usage of it for certain cases.

3. User journey:

There are multiple players here: the end user who sees the food content, the users who create food content and advertisers who publish food-related ads.

I know that ads already have the feature to mark them irrelevant or annoying, so I'd like to focus on the end consumers who see food content and their journey:

a. open newsfeed

b. see food-related media

c. click on the options button, choose "mute showing food"

or

d. click on the post, from within the post, click the options button, choose "mute showing food"

e. user is back in his/her newsfeed, the post is gone (is my understanding correct? Yes)

f. next time user opens their newsfeed, there's no food-related media posts

g. if user wants to revert their decision, clicks on general Settings button, navigates to options related to posts, where it's possible to see and edit the muted categories (for now - just Food).

4. Map the metrics:

I'll map the user journey to appropriate metrics so that we have a comprehensive list and then I will prioritize the more relevant ones:
 

5. Evaluation:

To evaluate the metrics, I'll use our defined goal and the complexity of the actual measurment to define a "north star" and primary and secondary metrics.

Since our main goal was to address churn, our North star metric will be 4.1 -  DAU for people who received food-related media and blocked it as opposed to similar people that received food-related media but didn't block it. Obviously, we don't want to just blindly encourage all users to use this feature, so it's important to maintain this balanced view of the two types of users.

The other main metrics would be related to engagement - 3.1 Time spent in newsfeed and 3.3 shares, comments and reactions - since, as we said, we want to also make sure the content is replaced by other content that's more relevant and interesting to the user.

Finally, the secondary metric to track would be 5.1 - % of un-muters, to keep an eye on potential bugs or popularity trends that would cause people to revert their previous decisions.

6. Summary:

In. order to address churn, we have introduced a mute-food feature, allowing people to block food media posts in the newsfeed and to track the success of it, we want to measure DAU of people who blocked vs DAU of look-alike people who didn't block it. Additionally, time spend and responses would let us know if the content is replaced by better content and finally % of un-muters would make we have addressed people's requests correctly.

Clarity

When you say 'mute showing food' features I assume you're referring to custom filters that lets you filter out 'topics' that you don't find interesting. 

Does this mean users can also set up filters for 'topics' they dont' find interesting outside of food? assuming that's the case

how does it work- does it work off hastag or image recognition, I'm assuming it the latter ( the scope of how that works is outside the scope of this answer)

Does this work on stories or feed or both? I'm assuming both 

Is it for a period of time, similar to how mute on a contact works on whatsapp? yes

Why would we do this?

Users may have personal preferences that are not reflected in the IG algorithms instantly, this signal enables the user to quickly specify a loss of interest in a particular category explicitly instead of gradual learning.

user stories

Rahul recently lost his dog Brutus and he otherwise loved watching dog videos but now they remind him of his dog so he wants to mute the topic for a while.

Rahul is trying to lost weight but all these delicious posts on IG keep tempting him so he wants to mute all food posts.

Hypothesis here would be that certain topics were preventing/creating friction to engagements from a certain set of users, if I allow users to weed out specific topics they should be happier and browser for longer.

The risk is that this reduces the number of posts the user can consume so we need to be tracking the impact

Metrics

Primary

(Posts consumer + stories consumed) per user 

This encapsulates impact of post reduction as well so covers risk.

other metrics to look at 

- Adoption rate: Since this is a new feature this is important to track 

- Opt out rate: Since this is a new feature this is important to track , users may try to use the feature and then opt - out for reasons that may range from quality of the product shipped to actually not wanting to use it since they didn't like how strongly it impacted their feed

- Time spent per user : This should ideally go up 

- Like rate/comment rate: This should remain unchanged, hypothesis here is that a group of users will mute topics like food that they enjoy but are looking to avoid leading to a drop in like rate BUT there will be a another cohort of users who will mute topics that they do not enjoy thus enabling them to discover /interacts with posts they actually like

Instagram goal is to allow people to share memories through pictures/stories

Clarify question:

1. Is the “mute” feature based on the category of the content or based on the hashtags? : The mute feature is based on the category defined at the explore section

2. Does this also applies to the stories? : Nope for now 

3. Does this feature affects only the viewer’s feed or even discovery/search feature?: Feed and discovery 

4. Is this feature specific to “showing food” or in general allows user to mute a hashtag? : let’s say “mute showing food” feature

Goal: Mute feature aligns with the uber goal of instagram which is bringing people together/ building community by showing relevant/interested information to the user 

Two types of users

1. Creators 

2. Viewers: This feature focuses on increasing the experience of the viewers by allowing them to select the “type” of content they want to see on their feed.

Pros

•  Better user experience 

Con

• Could affect the feed content, what if the people the user is following posts “food” picutres” majorly then the viewer wouldn’t have enough content to engage.

• Could discourage content creators of “food” channel if there a lot of people muting the topic 

Metrics (in order of priority)

• # of people using the mute- food topic

• Time spent by the above users in feed / day

• Time spent by the above users in search/explore / day

• # of posts pertaining to feed / user (if there are a lot of users using the mute feature, it might decrease)

• # of likes against the posts regarding food topic