What metrics would you use to measure the success of the Save feature at Facebook?
87.2k views
-icon.svg){kind=icon}
87.2k views
87.2k views
Liked by PM Exercises
Access expert answers by becoming a member
You'll get access to over 3,000 product manager interview questions and answers
Recommended by over 100k members
I will answer this Facebook metrics question by taking these steps:
- I will clarify what the feature is.
- Then I will explain what our business goal is with this feature.
- Next I brain storm the metrics.
- I prioritise the metics
- At last I summarise.
Clarify: Save Feature allows users to to Save Links, Pages, Posts, Locations, Movies, etc to view later. FB also reminds users about what items they have saved.
This feature affects users and marketer. Marketers do not want to be forgotten, so if they post something that attracts the attention of the user, they want the user to be able to find it again later, if they don't have immediate time to spend on it. For example, if there is a nice shoe advertised on FB and user likes it, but cannot check it now, or there is a discussion about a TV series that the user potentially finds interesting to watch later, the user can save it to check it out later.
User Goal: The benefit for user is that they do not need to copy paste or make screenshot of thing that they want to checkout later. They can have all these items in a categorised way (e.g. Movies, Pages) and can check them later.
So I as FB expect that Save Feature
- Marketer Goal: increases revenue for marketers by increasing clicks and impressions
- BIZ Goal: increases user engagement
- BIZ Goal: Increase FB revenue by increasing CTR, and CPC and CPM. Because user might make a click that he would not have done otherwise if he could not save the post. So the goal is to increase CTR and consequently the revenue.
Metrics to measure the success:
- Discoverability: %of users the have at least once Saved an item. --> User knows that he can Save items
- Discoverability: %of users from (1) that have at least once opened Saved Page --> User knows where to find Saved items and knows how to work with it
- Discoverability: %of users from (2) that at least once engage (metric 11, 12, 13) with at least one item in Saved Page
- The avg amount of time it took user from Saving an item to opening it again
- %of users that engage with an item in Saved page too late, so that the item is expired already (difficult to implement, how would the algorithm know if the page/link/... does not have the value it had at the time it was Saved?)
- Examples: it has been an offer for a discount/coupon that was only 1 week, or it has been a event would have been streamed live in an FB page
- #of times a user clicks to Save an item vs. the #of items the user visits (% in 1 month)
- #of times the user opens the Save page vs. the #of times the user visits FB (% in 1 month)
- #of categories in Save page the user goes through to see what items are saved in each category
- the amount of time user spends just looking at items in Save page
- distribution of #items Saved in each category, e.g. Links, Movies, TV, Locations. ---> This can help us to clean our list of categories and remove the noise (i.e. categories that are barely saved) or add new categories based on what is Saved in what category, but our algorithm tells us the category does not match.
- engagement: %of Saved items that the user opens from Saved page
- engagement with the items that are opened from Saved page, i.e.
- #likes
- #shares
- #clicking links
- #playing videos
- #comments
- engagement: amount of time spent on a page, after opening it from Saved page.
- %of Saved items that the user deletes without engaging with or opening them
- %of Saved items that the user deletes after engaging with or opening them
- retention: reaction to reminders from FB about items in Saved page,
- #of times user clicks on reminder to see Saved items.
- #of times he "engages" with items after getting the reminder.
- retention: change in the # of items that user Saves every month .
- Are we cannibalizing? ---> Is there a decrease or increase in likes, shares, comments, #of posts after pre-launching this feature for test group?
- User Segmentation: What is the user demographics for those who engage with this feature and those who do not
- Age
- Gender
- Mobile/desktop
- Location
- User Segmentation: What type of users use this feature most often? Power users? Passives? Medium users?
- If we have a problem in Discoverability, can we encourage power users to write about this feature in their posts?
- Shall we begin the testing phase of the feature with power users to fine tune it and do A/B testing?
- What features items have that are most Saved?
- Item type (page, link, movie, location)
- What is in the item?
- amount of text
- video
- image
- They have a lot of comments, shares, likes
- They are shared/liked/commented by a friend
- How close is the friend?
- what has been the type of the friend's reaction to this post? Like? Share? Comment?
- How many friends have reacted to it before I save it? --> Hypothese: It makes a difference for user if 2 people he knows have liked sth vs. 100 strangers.
- They have a high similarity to what user already interacts with: Hypothesis: We can use this information to suggest to users to Save an item, but needs to be implemented carefully to not to cannabalize active engagement of users with the hope of an engagement at a later time point.
- They have a high similarity to what user sees in his News feed everyday, but does not react with --> Hypothese: User has been potentially interested in these items, but did not have the time to interact with them? We can also suggest users to Save items that he rarely interact with and see what users reactions is? Does he Save them? Does he go back to see them?
- If we adjust News Feed of the user based on the items he has Saved, i.e. recommend new items in NF or give higher priority to items in NF that are highly similar to the Saved items, do we see an increase of engagement (likes, shares, comments, #of posts, clicking links, playing videos, time spent)?
- Monetization: %Increase in CTR (#of clicks measured for click ads, the increase should be very low, but still since CTR is always very low, we can only increase it in very small steps) --> increase in revenue for Businesses
- Monetization: %Increase in Impressions for Impression Ads
- Monetization: %of revenue for FB increase just based on clicks and impressions made through the funnel that includes Saved items
Prioritise:
Based on BIZ goals and user goals, I choose the following metrics:
1,2,3, 11, 12, 13, 21 (1-4), 22, 23, 24, 25
Summarise:
Save Feature is helping users to Save items to interact with them later. It expects to increase the revenue for marketers by helping them not to be forgotten by users. It also helps FB to increase engagement and finally revenue.
We brain stormed metrics to measure discoverability, engagement, retention and monetization effects for feature.
Define & Clarify
I’d first want to clarify and define what the FB Save Feature is. The Save Feature allows users to save favorite posts from their news feed for later consumption. Users can save a post by clicking Save from the drop-down menu in the upper righthand corner of a post. Once a post has been saved, you can view it later by accessing the “Saved” page in the lefthand sidebar. The saved posts are organized by categories.
At it’s core, the Save Feature solves for two main user problems:
- Users don’t have time to read favorite posts in one session.
- Users can’t find favorite posts during a repeat user session.
Set Goal
The success of this features depends on what my goals are as a PM. There are two types of goals I’d would evaluate:
Value Added to User
- Help users read their favorite posts in multiple sessions.
- Help users easily find their favorite posts during a repeat user session.
Value Added to Company
- Rev: Increase monetization opportunities
- Strategy: Diversify into the bookmark market.
I’d prioritize focusing on the user experience goals since it’s not wise to monetize features that don’t add value to users.
List User Actions
Before diving into metrics I'd first want to identify the user actions for the Save Feature because I can’t define the metrics without knowing what I should be measuring:
- User scrolls through FB newsfeed and clicks on ‘Save’ for the posts he wants to view later.
- User returns for repeat FB session at a later time.
- User accesses the Saved page from the sidebar
- User scrolls through the Saved content under the All tab
- User clicks on the category tabs to find the saved content
- User clicks through to read the Saved content
- User likes the Saved post
- User comments on the Saved post
- User Shares the post
- Poster navigates back to the Saved page to view more bookmarked content
Identify Metrics
Now I’d translate the user actions to quantifiable metrics:
- Avg # Saved posts per user
- Avg # page views of the Saved page per user
- % of users who viewed a Saved post after saving the post.
- Avg # likes of Saved posts per user
- Avg # comments of Saved posts per user
- Avg # Shares of Saved posts per user
Evaluate
To identify the most important metrics for success, I’d map the metrics back to my goals as a PM of this feature.
1. Primary Goal: Help users read their favorite posts in multiple sessions
- MetricA: Avg # Saved posts per user session
- MetricB: Avg # views per Saved posts per user within a 90 day period.
2. Secondary Goal Help users easily find their favorite posts during a repeat user session.
- MetricC: Avg # page views of the Saved page per user
The most important metric I’d focus on is MetricB — Avg # of views of Saved posts per user within a 90 day period. MetricA and MetricC are means to get to MetricB. I.e. Users would save posts and list the Saved page in order to read their favorite content at a later time.
Platinum PM
Step 1: Ask Clarifying Questions
Are we considering any particular objective in mind while defining the success of the save feature or is it something that i can assume. For example is the company objective to increase user adoption or user engagement or revenue.
Are we talking about the entire product or only a part of the product or is that something that you'd like me to define
Step 2: Talk and explain your understanding of the product.
The mission of facebook is to connect people across the world and bring them closer to each other.
Save feature is used to save content in the form of videos, articles, posts etc which can be used for later reading and consumption.
If a person saves any piece of content then it gets saved in the saved folder, person can create mulitple folders and name the folder depending on the type of content they are saving. Used for organising content also this feature is helpful when someone doesn't have time to go through a piece of content and want to do it in the future.
Save feature has been out for a while and gotten a very good adoption rate, however not everyone uses this feature. Therefore we can say that the feature is in its growth phase.
Broad level target audience for facebook save button are Consumers (Brands, Individuals who consume a lot of content and if they like any sort of content and would like to go back to in the future they save and bookmark it using the save feature)
If you save a particular piece of content other's wont be able to see but it helps facebook in making better decision regarding the kind of content you like and thus shows more personalised and related content to the user giving them a personalised and improved user experience.
Step 3: Identify and define about the goal of the product
Facebook save is part of the facebook ecosystem thus since its a very huge feature it can have multiple goals ranging from Driving user engagement as more saves would mean that people are liking and enagging more with the content, which will help facebook show better personalised content which will driver user experience. Driving engagement amongst users will lead to revenue growth for facebook
Thus these are the 3 goals that we have in mind for this product, however for this particular question sake I'll be going with Increasing/User engagement as the main goal of this feature as this is aligned to the mission and vision of facebook of bringing more people closer across the world.
Step 4: Go through the user journey (user actions) that will drive user engagement of facebook
1) User goes on facebook newsfeed
2) User sees a lot of content, but doesn't have much time to perform any action
3)User saves the content that he/she likes for future reference
4) Once user has time user comes back to the saved folder and engages with the content that he/she had saved earlier.
5) User can engage with this content in different ways, like, comment, tag someone or share that content with people in their network.
Step 5: List down the metrics that are aligned towards the end goal. (Since our goal is to drive user engagement some of the metrics that we will choose is)
1) % of total posts saved -This will tell us about the adoption rate of the save feature as how many people are using the save feature actively.
2) Average number of likes and comments per saved post on a daily, weekly and monthly basis- This is a very good metrics as it talks about the user engagement on a broader level
3) Average Number of users that use the save feature on a daily, weekly and monthly basis- This metric means is a good metric as well since if the adoption rate will increase chances of engagement with the feature and product will increase as well
4) Average number of posts saved per 1000 users- Same as 1 this will tell us about the adoption rate of the save feature
5) % of users who have saved and engaged (like, commented or shared) with atleast 3 posts in a week's time- This is a very good metric as well as it talks about and showcases user engagement in a limited timeframe
Step 6: Now we will evaluate and prioritize the metrics based on certain success criterias such as
Relevance to the end goal and company mission, Impact to End User and effort required to collect and measure these metrics.
Based on the above description of the metrics that were listed down in step 5, the 2 metrics that i think would be the best fit to measure success of Facebook save to drive user engagement are as follows
1) Average number of likes and comments per saved post on a daily, weekly and monthly basis
and 2) % of users who have saved and engaged with atleast 3 post in a span of a week.
Step 7: Summarize your Answer
This will just be the summary highlight the goal of the feature, metrics we listed down and the metrics we adopted.
Thank you 1) % of total posts saved -This will tell us about the adoption rate of the save feature as how many people are using the save feature actively. DIV 1 Alright so before proceeding with the answer to the questions i have a couple of clarifying questions that i need to ask.
Step 1: Ask Clarifying Questions
Are we considering any particular objective in mind while defining the success of the save feature or is it something that i can assume. For example is the company objective to increase user adoption or user engagement or revenue.
Are we talking about the entire product or only a part of the product or is that something that you'd like me to define
Step 2: Talk and explain your understanding of the product.
The mission of facebook is to connect people across the world and bring them closer to each other.
Save feature is used to save content in the form of videos, articles, posts etc which can be used for later reading and consumption.
If a person saves any piece of content then it gets saved in the saved folder, person can create mulitple folders and name the folder depending on the type of content they are saving. Used for organising content also this feature is helpful when someone doesn't have time to go through a piece of content and want to do it in the future.
Save feature has been out for a while and gotten a very good adoption rate, however not everyone uses this feature. Therefore we can say that the feature is in its growth phase.
Broad level target audience for facebook save button are Consumers (Brands, Individuals who consume a lot of content and if they like any sort of content and would like to go back to in the future they save and bookmark it using the save feature)
If you save a particular piece of content other's wont be able to see but it helps facebook in making better decision regarding the kind of content you like and thus shows more personalised and related content to the user giving them a personalised and improved user experience.
Step 3: Identify and define about the goal of the product
Facebook save is part of the facebook ecosystem thus since its a very huge feature it can have multiple goals ranging from Driving user engagement as more saves would mean that people are liking and enagging more with the content, which will help facebook show better personalised content which will driver user experience. Driving engagement amongst users will lead to revenue growth for facebook
Thus these are the 3 goals that we have in mind for this product, however for this particular question sake I'll be going with Increasing/User engagement as the main goal of this feature as this is aligned to the mission and vision of facebook of bringing more people closer across the world.
Step 4: Go through the user journey (user actions) that will drive user engagement of facebook
1) User goes on facebook newsfeed
2) User sees a lot of content, but doesn't have much time to perform any action
3)User saves the content that he/she likes for future reference
4) Once user has time user comes back to the saved folder and engages with the content that he/she had saved earlier.
5) User can engage with this content in different ways, like, comment, tag someone or share that content with people in their network.
Step 5: List down the metrics that are aligned towards the end goal. (Since our goal is to drive user engagement some of the metrics that we will choose is)
1) % of total posts saved -This will tell us about the adoption rate of the save feature as how many people are using the save feature actively.
2) Average number of likes and comments per saved post on a daily, weekly and monthly basis- This is a very good metrics as it talks about the user engagement on a broader level
3) Average Number of users that use the save feature on a daily, weekly and monthly basis- This metric means is a good metric as well since if the adoption rate will increase chances of engagement with the feature and product will increase as well
4) Average number of posts saved per 1000 users- Same as 1 this will tell us about the adoption rate of the save feature
5) % of users who have saved and engaged (like, commented or shared) with atleast 3 posts in a week's time- This is a very good metric as well as it talks about and showcases user engagement in a limited timeframe
Step 6: Now we will evaluate and prioritize the metrics based on certain success criterias such as
Relevance to the end goal and company mission, Impact to End User and effort required to collect and measure these metrics.
Based on the above description of the metrics that were listed down in step 5, the 2 metrics that i think would be the best fit to measure success of Facebook save to drive user engagement are as follows
1) Average number of likes and comments per saved post on a daily, weekly and monthly basis
and 2) % of users who have saved and engaged with atleast 3 post in a span of a week.
Step 7: Summarize your Answer
This will just be the summary highlight the goal of the feature, metrics we listed down and the metrics we adopted.
Thank you 1) % of total posts saved -This will tell us about the adoption rate of the save feature as how many people are using the save feature actively. DIV 145
Step 1: Ask Clarifying Questions
Are we considering any particular objective in mind while defining the success of the save feature or is it something that i can assume. For example is the company objective to increase user adoption or user engagement or revenue.
Are we talking about the entire product or only a part of the product or is that something that you'd like me to define
Step 2: Talk and explain your understanding of the product.
The mission of facebook is to connect people across the world and bring them closer to each other.
Save feature is used to save content in the form of videos, articles, posts etc which can be used for later reading and consumption.
If a person saves any piece of content then it gets saved in the saved folder, person can create mulitple folders and name the folder depending on the type of content they are saving. Used for organising content also this feature is helpful when someone doesn't have time to go through a piece of content and want to do it in the future.
Save feature has been out for a while and gotten a very good adoption rate, however not everyone uses this feature. Therefore we can say that the feature is in its growth phase.
Broad level target audience for facebook save button are Consumers (Brands, Individuals who consume a lot of content and if they like any sort of content and would like to go back to in the future they save and bookmark it using the save feature)
If you save a particular piece of content other's wont be able to see but it helps facebook in making better decision regarding the kind of content you like and thus shows more personalised and related content to the user giving them a personalised and improved user experience.
Step 3: Identify and define about the goal of the product
Facebook save is part of the facebook ecosystem thus since its a very huge feature it can have multiple goals ranging from Driving user engagement as more saves would mean that people are liking and enagging more with the content, which will help facebook show better personalised content which will driver user experience. Driving engagement amongst users will lead to revenue growth for facebook
Thus these are the 3 goals that we have in mind for this product, however for this particular question sake I'll be going with Increasing/User engagement as the main goal of this feature as this is aligned to the mission and vision of facebook of bringing more people closer across the world.
Step 4: Go through the user journey (user actions) that will drive user engagement of facebook
1) User goes on facebook newsfeed
2) User sees a lot of content, but doesn't have much time to perform any action
3)User saves the content that he/she likes for future reference
4) Once user has time user comes back to the saved folder and engages with the content that he/she had saved earlier.
5) User can engage with this content in different ways, like, comment, tag someone or share that content with people in their network.
Step 5: List down the metrics that are aligned towards the end goal. (Since our goal is to drive user engagement some of the metrics that we will choose is)
1) % of total posts saved -This will tell us about the adoption rate of the save feature as how many people are using the save feature actively.
2) Average number of likes and comments per saved post on a daily, weekly and monthly basis- This is a very good metrics as it talks about the user engagement on a broader level
3) Average Number of users that use the save feature on a daily, weekly and monthly basis- This metric means is a good metric as well since if the adoption rate will increase chances of engagement with the feature and product will increase as well
4) Average number of posts saved per 1000 users- Same as 1 this will tell us about the adoption rate of the save feature
5) % of users who have saved and engaged (like, commented or shared) with atleast 3 posts in a week's time- This is a very good metric as well as it talks about and showcases user engagement in a limited timeframe
Step 6: Now we will evaluate and prioritize the metrics based on certain success criterias such as
Relevance to the end goal and company mission, Impact to End User and effort required to collect and measure these metrics.
Based on the above description of the metrics that were listed down in step 5, the 2 metrics that i think would be the best fit to measure success of Facebook save to drive user engagement are as follows
1) Average number of likes and comments per saved post on a daily, weekly and monthly basis
and 2) % of users who have saved and engaged with atleast 3 post in a span of a week.
Step 7: Summarize your Answer
This will just be the summary highlight the goal of the feature, metrics we listed down and the metrics we adopted.
Thank you
The first question I would ask is: what does “success” mean? Is it aimed at increasing user satisfaction, or usage/engagement/increased ad revenue? My assumption is that the answer is “yes” – at the core of a successful feature is the personal and social value for the users which ideally also generates profitability for Facebook and its advertisers. (A win/win/win).
A lot of telemetry can be measured, but in order to gain the most valuable insights, let’s start with the value for the users. Here are some reasons why they might want to make use of the save function:
-1- Ability to return to a certain post at a later time to consume or share/react/comment
-2- Ability to browse more content faster, while marking (=saving) a subset of interesting posts for more focused consumption/interaction
-3- Ability to collect posts related to certain themes/topics
-4- Ability to collect posts from certain friends or groups (what happens when a friend deletes a saved post from their wall??)
The assumption is that the save function is successful if it enables the above listed abilities. In the absence of actual user feedback, the combination of the following measurements are good proxies to assume that the save function is successful:
a. #saves per user per Facebook visit
b. #of returns to saved content per user
c. #interactions with saved content per user (or per saved piece of content)
d. #saves per amount of content scrolled through per user and FB visit
High numbers in a. plus any of b/c/d are a good indicator that a user is likely satisfied with the feature.
However, we also need to look at how many Facebook users (with regular activity on FB) have never used the save function. A high percentage would indicate a possible discoverability (awareness) issue.
If such an issue had been identified and addressed, then a decrease in the population which has never used the function is another measure of success.
If, furthermore, good analysis of what individual users tend to save helps surface more similar content in their newsfeed, we might be able to measure increased engagement with the newsfeed on content which had been better personalized based on previous save actions. Any such engagement can also be considered an (at least indirect) success metric for the save feature. Relevant measurements would be:
– #reactions/shares/comments/active consumption (e.g. play video) for content suggested based on a user’s saved list (even if it does not lead to added saves, which would be counted in a. above)
Other success metrics could be derived from non-instrumented data sources:
– actual user-feedback (e.g. one-question pulse queries on individual FB features)
– counterfactual testing (what if for a population of FB users the save function would be removed; how many complaints would we get?)
– Mining of any mention of the FB save feature in FB posts, comments, or FB-related user forums
The cost of deriving the last 3 suggested metrics is not likely justifying the result. Especially the first two suggestions have the potential to irritate the impacted users, while the ROI for the third is likely low.
I would therefore focus primarily on the readily available, yet relevant, logged usage telemetry described under a.-d. above, secondarily on the “indirect” success metrics, and furthermore measure if we can confirm any correlation between engagement with the save feature and increased # of visible and/or clicked ads. The latter measurement would be a metric related to the successful generation of financial profitability for the advertisers.
Bronze PM
The Save Feature is an option to bookmark posts for viewing at a later time. Is that correct?
I haven’t used this feature myself on Facebook, but I have used the feature on Instagram and it allows me to save any picture to a specific collection. My saved collections are private and can be modified. The primary purpose of this is to allow users to view something they like later or create a collection of like items which they may reference for a variety of reasons. Is that correct?
When was this feature launched?
(about a year ago)
Ok, I am assuming we are no longer focusing on awareness and adoption. I can assume we are in the phase where we want to increase engagement and repeat use. We probably have users who frequently save and reference posts, other’s who only use the feature occasionally and some who have never used the feature. If that is the case I’ll make my goal around increasing engagement.
Engagement Goal: I want to increase the number of people who are actively using the feature.
Let me think of all of the actions a person can take related to the save feature:
Actions:
- Save a post/image//item for sale, etc
- Create a collection for saved items
- Collaborate/Share
- View collection
- Delete items/collections
- Click on an item saved previously ( to either enlarge an image or visit a group/listing or see more details of a post)
Metrics:
- Number of saves per session (This could get more granular)
Saves per items viewed in marketplace
Saves per photos browsed
Saves per posts viewed
- Number of collections created
- Number of times a user shared a collection
Could look at specific interactions in shared collections
- Number of times a user viewed saved items/collections per session (or per month)
- Number of items removed per view of saved items
- Number of times a user clicks on a saved item
Action | Engagement | Pros | Cons |
Saves items | low | Shows interest | If the user never returns this measure may lead to false impressions of engagement |
Collections Created | Medium | Shows the user cares enough to organize collections | If the user never returns this measure may lead to false impressions of engagement |
Collaboration | High | Sharing a collection shows engagement, but we need to know if there is continued interaction Shows users are building relationships | Limited to only those who wish to collaborate. Doesn’t measure individual engagement. |
View saves | High | Shows value in saving | Doesn’t show deeper engagement, what does the user do with their collections? Why are they viewing them? |
Remove items | High (but misleading) | Show level of engagement | Someone may be highlighly engagement and never remove an item. |
Clicks saved items | High | Shows interest in the items saved and intent on learning more or revisiting the item. | Some users may save pictures like an idea board and never need to click though. Just viewing the list of items may be all a user needs. |
The most important action is actually saving something, but if users never return to what is saved the save feature isn’t much different than a like button. So, I don’t think the number of saved items would be my north star metric for engagement. It would be essential for adoptions, but I want to look at something that signifies the user finds value in the feature. If a user returns to their saved items that indicates they saved the items for a reason. The collaboration metrics show a different type of use and would really help measure how well the feature fits with our mission to bring people together. If we were focused on driving interactions between people I might focus on the number of shared collections vs non-shared collections. Removing an item does indicate a level of deep engagement, but the absence of a removal doesn’t indicate someone is not deeply engagement. The user may simply really like all of the items they saved.
The number of times a user returns to their saved items and views them shows that the collection of saved items have value. The user is reviewing them for a reason. Clicking a saved item is a great indicator that the user saved an item to refer back to it later.
Conclude:
My north star metric to measure engagement would be the number of times a person views their saved items per collection created. I would also need to consider the total number of collections being created and the number of items being saved, but the return to a create collection would be the best metric to follow for engagement.
Platinum PM
Clarify
Could you explain more about the Save Feature?
Save post of other people
Save the draft of you post while writing
Archive the post (hide from public)
Product
Problem
Save for later reading
Don’t have enough time to read at that time
Good way to store good sources
Goals
What is the meaning of success? Do we have any specific goals to define the success?
Awareness / Engagement / Increase interaction / …
Engagement = Increase engage with post
Customer Journey
Flick through the post (from different source newsfeed, notification, exploration, …)
Engage to the post
Save it for later reading, watching
Come Back later to read the post again
Delete saved posts
Metrics
Flick through the post (from different source newsfeed, notification, exploration, …)
Discoverability: %of users the have at least once Saved an item
# and % of users coming from different sources
Frequency of saving post per users
Engage to the post
Avg engagement time before saving
% categories of the saving post (long video, short video, long text, …)
Click save
Ratio of saving / reading
# of saved post per user (per day, per week, per month)
Come Back later to read the post again (Most impactful to our goal)
Discoverability: % of users from that have at least once opened Saved Page
Frequency of reading again saved post
Avg engagement of saved post
Avg retention of saved post
Ratio of engagement time between 1st, 2nd, 3rd ,... , nth reading
Delete saved post
%of Saved items that the user deletes without engaging with or opening them
%of Saved items that the user deletes after engaging with or opening them
Evaluation
Increase engage with post
Impact | Complexity | Risk | |
%of users from that have at least once opened Saved Page | Medium | Low | Low |
Frequency of reading again saved post | High | Low | Low |
Avg engagement of saved post | High | Medium (need to break down more) | Medium |
Avg retention of saved post | High | Low | Low |
Ratio of engagement time between 1st, 2nd, 3rd ,... , nth reading | Extremely high | Low | Low |
Recommendation
Frequency of reading again saved post
Avg engagement of saved post
Avg retention of saved post
Ratio of engagement time between 1st, 2nd, 3rd ,... , nth reading of save posts => North star metric
Success of Facebook “Save” feature
Describe the feature The Save feature on Facebook allows users to save posts, videos, or other content they find interesting but don't have time to view immediately. It solves the problem of losing track of content that users want to revisit later. Saved items are stored in a dedicated section of the user's profile, making it easy to access them when convenient.
Facebook's Save feature supports its mission to unite the world by enhancing content discovery, engagement, personalization, platform usage time, and user satisfaction. It helps users explore new interests, interact more through likes and comments, and curate personal content collections, making the platform more valuable and engaging. This contributes to a stronger sense of community and closer connections among users.
Choose a goal The primary goal of the Save feature is to increase user engagement with Facebook by encouraging users to return to the platform to view saved content.
Walk through the customer journey
User discovers interesting content but doesn't have time to view it
User clicks the Save button to store the content for later
User navigates to their Saved items section when they have time
User engages with the saved content by viewing, liking, commenting, or sharing
Map and quantify user behaviors in the customer journey
Awareness:
Metric: Save feature awareness rate
Definition: Percentage of users who are aware of the Save feature
Measurement: User surveys or in-app user testing
Acquisition:
Metric: Save feature adoption rate
Definition: Percentage of users who have used the Save feature at least once
Measurement: (Number of users who have saved at least one item) / (Total number of active Facebook users) x 100
Activation:
Metric: Save feature activation rate
Definition: Percentage of users who save an item within the first week of feature adoption
Measurement: (Number of users who save an item within the first week) / (Number of users who have adopted the Save feature) x 100
Engagement:
Metric: Average saved items per user
Definition: Average number of items saved per user over a specific time period
Measurement: (Total number of saved items) / (Total number of users who have used the Save feature)
Metric: Saved item engagement rate
Definition: Percentage of saved items that users engage with (view, like, comment, share)
Measurement: (Number of saved items engaged with) / (Total number of saved items) x 100
Retention:
Metric: Save feature retention rate
Definition: Percentage of users who continue to use the Save feature over a specific time period (e.g., 1 month, 3 months)
Measurement: (Number of users who have used the Save feature in the current time period) / (Number of users who used the Save feature in the previous time period) x 100
Evaluate your metrics
Metric | Impact | Confidence | Effort | RICE Score |
Save feature awareness rate | 2 | 3 | 2 | 3 |
Save feature adoption rate | 3 | 4 | 4 | 3 |
Save feature activation rate | 3 | 4 | 4 | 3 |
Average saved items per user | 3 | 4 | 4 | 3 |
Saved item engagement rate | 4 | 4 | 4 | 4 |
Save feature retention rate | 4 | 4 | 4 | 4 |
RICE Score = (Impact × Confidence) ÷ Effort
Prioritize the metrics
Metric | RICE Score | Rank |
Saved item engagement rate | 4 | 1 |
Save feature retention rate | 4 | 1 |
Save feature awareness rate | 3 | 2 |
Save feature adoption rate | 3 | 2 |
Save feature activation rate | 3 | 2 |
Average saved items per user | 3 | 2 |
Summarize your answer The primary goal of the Facebook Save feature is to increase user engagement by encouraging users to return to the platform to view saved content. The key metrics to measure the success of this feature, in order of priority, are:
Saved item engagement rate and Save feature retention rate
Save feature awareness rate, Save feature adoption rate, Save feature activation rate, and Average saved items per user
Clarify: There are 3 types of save features on Facebook. when you say save feature, is it the "saved" feature that allows users to check their saved posts section or the "save post" feature which allows users to just save a post, or the "save link" feature that allows users to save sponsored ads or all of them?
Interviewer: All of them in general
Me: I would like to start by defining the business goal for the save feature is, what the feature does and which metrics to measure the business goal
Interviewer: OK
Me: the overall goal of the Save feature is to increase the engagement of Facebook users and increase revenue for Facebook through sponsored ads. The users could save a post/sponsored ad, come back to it at a later point in time to take an action. They could react to the saved post, share the saved post or share the saved link with others, or make a purchase if the saved link is a sponsored ad. Facebook also reminds users to take action on the saved item. Is this something you have in mind too?
Interviewer: sounds good
Me: I would like to now start defining metrics that would measure the success of engagement and revenue creation through the save feature.
Metrics to measure engagement:
1. I would like to measure the average number of posts that are getting saved per day/week/month. I could learn the stickiness of this feature by measuring this metric
2. The main purpose of the save feature is that users should come back to the saved post. I want to measure the proportion of users who view the post after saving the post in a week/month
3. Also, we consider it a success if users engage with the post after saving it. So I will measure the proportion of users who react/comment/share the post after saving the post in a day/week/month
4. if users are saving many posts, the users should be able to find the saved post in the saved posts section easily. So I will measure the average time taken to click on a post after a user enters the saved post section.
5. If the user takes a lot of time searching for his/her post in the saved posts section, and desists from clicking anything it is an indication that users are not able to find saved posts easily. So I will measure the number of users who desist from clicking a saved link after spending some time in the saved post section
Metrics to measure revenue:
1. To know if users are saving the sponsored ads in the first place, I would measure the proportion of sponsored ads saved out of the total saved items in a day/week/month
2. To understand if the users actually have an intention to make a purchase through saved ads, I would measure the number of unique clicks on saved sponsored ads out of the number of saved sponsored ads in a day/week/month
3. We will consider the "save feature" a success to generate revenue, if users make purchases by clicking a saved sponsored ad link. So I would measure the number of saved sponsored ads that were monetized out of the number of saved sponsored ads per week/month
4. I will monitor the revenue generated out of the total saved sponsored ads in a week/month to understand the revenue generated per saved ad
My northstar metrics for engagement would be average number of posts that are getting saved per day/week/month,
To summarize I focused on the business goal of increasing user engagement and revenue generation from the save feature and would consider this increase as success. I analyzed how the save feature would work for a normal user and advertiser. Then I defined some key metrics that I will measure under engagement and revenue and I will monitor these metrics to see if they are moving in an expected manner
Clarifying the scope
What does the save feature do?
Allows users to bookmark posts on feed/from users.
The posts maybe from a page or user or brand - it could be content or a product listing.
Users can organise this into collections or keep them as individual posts.
Why would the user want to save - its an artificat that they want to come back to, it could be an interesting funny post or a really nice dress that they were inspired by or something that they want to keep on their radar.
As a second order effect business or content creators may benefit from this via additional engagement or sale $ but I want to take a pure consumer facing view here since
1) optimising for sale dollars seems counter to facebook's primary metrics which are network growth oriented (I'll look for confirmation on this)
2) Creators organically get additional engagement when the users come back to it so ideally I'd expect the save primary metric to be in line with additional enagement for creator metric
Why was save feature built - user need and feature goal?
- Facebook today allows you to discovery content from a variety of sources in the feed + dedicated tabs for gaming and video(watch)
- When a content creates a lot of value for the user it usually is because its extremely interesting(this amazing video on mantis shrimp has a superpowerful punch) or entertaining(this amazing Porsche GT 3 RS launch video that gave you goosebumps) or useful (exercise video or cooking recipe or pair of sneakers you had your eye on)
- Usually users may want to come back to it to rediscover the delight or the utility
- Facebook wants to grow the network because the usefulness/value of the network grows with its size
- Save allows you to get user investment upfront from users for the future, users will need to come back to facebook to check the content they had saved
- As a second order effect because users are viewing saved content you may get additional engagement however it also runs the additional risk of canabalising engagement from the feed leading to a net negative
PM
The save option shows up when you click the 3 dots on the top right of a post. It serves a purpose to 3 main stakeholders: the content creator, user and Facebook admin.
The user:
1) If they have no time, they can view it later: %of times posts were saved before logging out of facebook, avg number of posts saved per session.
2) If they find something interesting they'd like to see again, they can: % of times people look at saved posts again, average number of times a user views a shared post
3) Centralized platform to view all the posts they like, sort and filter through them: Engagement with the saved posts interface, Search efficiency
The creator:
1) Adds an option for people to interact with their posts more.
Facebook admin:
1) Acts as a point of data collection to see what kind of posts users wants to see multiple times, i.e. is VERY interested in. % of posts similar to posts in the shared section the user engages with (likes, comments, saves)
2) Allows for more clicks in ads: % of times an ad is clicked again when it is saved.
Our next step will be prioritising these metrics, that can be done based on facebook's priority here (engagement, revenue, retention, etc)
PM
Just to clarify: The save feature allows users to save or ‘bookmark’ specific pieces of content like posts?
Clarifying question: Is it a link just on a newsfeed post? Or would you like me to specify on a specific link? Answer: all links.
For users who are exploring content they have the ability to save the links so that they can revisit the content at a later point in time. And for people producing the content it also helps them gain more traffic to their content.
A user that goes through their saved links and by clicking on it return to the links that they had before.
What is the goal of feature:
The goal of the feature is to ‘increase interaction with content’. Given the environment people are in on the platform, they sometimes don’t have the attention span to look at content straight away. However it’s important that users get to interact with the content for multiple reasons:
Advertiser revenue depends on vision of content
It’s valuable to users and can help them
It helps the content creators by providing more reinforcement for them.
The content consumer journey:
As a consumer of content:
I’m browsing my newsfeed.
I see a piece of content that i’m interested in but don’t have time to read.
I hit the ‘save for later’ button.
I continue browsing content
At a later point I find the ‘saved content’ section on my phone to revisit the content i saved.
For content producers:
They create their post or content
It gets saved as a bookmark for consumption later
They receive engagement over a longer period of time, and that content becomes more valuable as a result.
Key Metrics to track in the journey: Engagement
Content consumer:
North star:
#of links revisited
Secondary metrics:
# of users that see the link save button
# of links saved
#user exposed to button / # of links saved
Growth in links saved
Time from saved link to visit
Content producer:
North star:
# of impressions based on links
Secondary
Growth in post revisits from links
The time from post saved to post revisit
Prioritization of metrics:
Metric | Reach | Impact | Confident | Effort |
#of links revisited | High | High | High | Low |
# of links saved | Medium | High | High | Low |
# of users that see the link save button | Medium | high | high | Low |
Time from saved link to visit | Medium | High | High | Medium |
Growth in links saved | Medium | High | High | Medium |
Summary:
# of links revisited is the North Star metric for us, because it helps us understand how many people are going back to the content they are saving, which is our key goal with this feature.
We want more interaction with content, that will bring in more advertising dollars and increase engagement/stickiness with our users.
The # of links being saved is also very important to track, because it indicates the visibility of how many people are saving links, and related to that is the ratio of people who see the save link button to people who click it (this will help us understand which pieces of content are more likely to be saved).
The time from saved link to the revisiting of the link is also an important metric when we think about it from the lens of our advertisers. If users are taking too long to revisit the link, then the content on the link might not be as relevant as it was before.
Think there r about 5 categories of metrics to look at for most feature rollouts:
1) who is using it – persona / segment / type (novice, proficient, expert)
2) when – what do they do immediately before / after, etc.
3) usage – number of times in a session / week, duration between 1st-2nd, 2nd-3rd usages, etc.
4) impact – aarrr (short-term & long-term), funnel, etc.
5) cannibalization – did usage of some other feature decrease
Bronze PM
My approach would be understanding the feature, the goals of the feature how it is helping the user and the company.
I would understand the user journey, the hypothesis, and the assumptions and come up with quantifiable metrics.
The Feature: Save feature allows users to save the content inside their Facebook account in a private folder and allows them to revisit the content through any device any number of times.
User Journey:
User visits Facebook
The user scrolls and finds content they are into
The user saves the post and continues through his session
The user comes in a later session
The user goes to the saved folder
The user picks the content of his choice and engages
Assumptions:
Users need a repository of their favorite videos
Users love the content they saved
Users value the folders they created
Users would require a private space for their favorite content
This is creating a new use case for the product
from this, the metrics I can come up with the mix of user journey and assumptions are:
Adoption:
1. Total no of users using the save feature
Engagement:
2. Percentage of users revisiting the saved folder
3. Increase in time spent for adopted users - I will track this as a saved folder vs a feed. The first one is to see if this is a new use case and the second one is to see if the insights from the save feature are helping Facebook improve the user's feed.
4. Increase in % of engaged posts on the feed of adopted users.
Retention:
5. Retention rate of adopted users
6. Drop in deactivation rate for adopted users i.e assuming pop-up talks about the data loss after the user clicks on deactivate( this is a long-term metric but is a crucial one as a repository increases the value of Facebook to the user)
Other:
7. Number of posts with 'Save' as the sole interaction ( this helps to evaluate if the algorithm needs to weigh the save interaction more).
I would prioritize metrics 2,3,6 as this is related to the core value of the product.
I have assumed the feature doesn't have a long journey for saving like saving it into a folder of choice, which would add the metrics like time spent to save and the drop-off rate.
Describing product/feature:
1. The save feature at Facebook is used to save posts (of all types) via Facebook feed and divide them into multiple categories of their choice.
2. Users can view saved posts via the 'Saved' section
Assumptions
I will talk about consumer-level metrics
Mission of Meta / Facebook
To bring the world closer by connecting them
Goal
1. Business Goal: To increase engagement of the 'Save' feature so that more people save posts and engage with saved posts which further brings the world closer.
2. User Goal: To let users get back to the posts they're interested in and interact with them.
User Journey
Users scroll feed -> Go through a post -> Find it interesting -> Save the post in existing category -> Interact with the post -> View 'Saved' posts -> Interact with the post
Metrics
High-level metrics
1. Total posts saved
2. Total visits to the saved posts page
3. Total time spent on 'Saved' posts
User-level metrics
1. % & # users who saved at least one post - total, in a week
Saved posts metrics
1. Avg. # posts saved per user (who have saved at least one post) - totally, in a week
2. Avg. # posts saved / # posts scrolled per user (who have saved at least one post) - totally, in a week
3. Avg. # categories created per user (who have saved at least one post)
Saved Posts Visits metrics
1. # users visiting 'Saved' posts / # users who have saved at least one post
2. Avg. time spent on 'Saved Posts' per user
Metrics prioritisation
I would check high-level metrics for the overall success of the 'Save' feature.
Silver PM
Facebook's Save feature allows users to save items like posts, photos, videos, places, events, movies, TV, and music to view later. Saved items are organized by category and appear in a bookmark-like icon called Saved Items that only the user can see. Users can also receive reminders about their saved items in their News Feed.
This is an engagement feature and the user journey goes like this
User Journey 1: User logs in -> Watch the feed -> Sees a post that they like -> Saves the post
User Journey 2: User logs in -> Go to the saved post -> Watches the content again -> shares it with friends
Metrics are also typically broken down into L0, L1, L2 etc.
Acquisition
- Number of items saved - break this down by type of post, geography, age, etc.
- Save frequency - per session, per day, per week other than the normal breakdown of type, region, etc.
Activation/ Engagement
- Number of times the saved catalog is visited
- Frequency of the visit - per session, per day, per week other than the normal breakdown of type, region, etc.
- Number of times the saved item is shared
- Time spent on the saved items
- Number of times an ad is saved
Gold PM
- this feature allows us to save reels/post(image/video) stories for furture viewing ?
- Is there a time period that till time save can exist in your application
- up to you
- Typically save feature is like to add to favourties which is stored at the account level there by it stores data at server side and not locally using phone memmory ?
- Geo - Not required (geo wise)
- Objective - engagement
| B2B(Shop+Ads) | B2C(Social media) |
| Large congolomerates/Big organisation/Ecommerce companies (Sizeable in terms of employees and GMV with high spends) | High Frequency(Active 5 to 7 days on Socila media and is buying something via facebook shop) |
| Influencers/Finfluencers/Trainer | Mid Frquency (once in 15 to 30 days) |
| Drop shippers | Low Frequency (once in 3 to 6 months) - negligible |
| Sellers/Retailers |
Metrics
| Category | Metric Type | Comments |
| Engagement | Avg Save per user per day/week/per month | |
| Engagement | Total number of save per user per day/week/Month | |
| Engagement | Total saves | |
| Engagement | Like/Save, Comment/Save ratio - Daily/weekly/Monthly | |
| Engagement | Avg no of times post was viewed post saving - 2 , 3 (weekly/daily/Monthly) | North Star |
| Engagement | Viewing pattern (1-5), (5-10) (10-15) | |
| Engagement | unsave pattern (1-5), (5-10) (10-15) |
These are the metric i would want to study/use for measuring the sucess of feature.
Gold PM
Describe what is Save feature -> Allows users to to Save Links, Pages, Posts, etc to view later. Also reminds whats saved
Clarifying Questions:
Works similarly across all app versions & web -> Yes
Should I be mindful of any particular geography -> irrelevant
Any particular user category to focus on -> your call
Why would people use this feature -> people don’t have time then and there to view the content. User might still want to review even if seen earlier
Facebook’s mission -> helps build people community
Describe how does this feature work:
Sees a post, page, reel etc.
Click on save
Content gets saved
User receives notification about saved content
User accesses save tab either in the same session or comes back later in a different session
Clicks on save tab
Selects what to see
Once seen, the post, page, reel gets unsaved
Is the above workflow correct -> Yes, mostly but we can continue with this
User Persona:
Basis frequency of usage:
High
Medium
Low
Basis age category:
Teenagers
Working professionals
Retired people
Since the workflow remains standard across user category, the adoption, engagement might still differ. Since working professionals would form the highest user base, prioritizing this base from an impact perspective. Also this user base would be most time crunched and would be motivated to use the feature the most.
Metrics:
North Star Metric - MAU/DAU/WAU
Feature Metrics:
# contents saved/user
% distribution of types of contents being saved
% users who saved atleast one content and later visited the save section
% contents viewed after being saved
% inc in absolute session length on a weekly/monthly basis
Clarification
- Is save feature only for facebook app / instagram or all ?
- Assuming it’s only for facebook app as of now
Overall Approach
- Problem & Persona
- User Flow
- Goal/Objective
- Prioritize Persona/Use Case
- Metrics Constellation
- Retention
- Activation
- Engagement
- Monetization
- Ecosystem/Business
- Health
- Satisfaction
- Loyalty/Advocacy
- Prioritization and North Star Metric
Problem it solves for different personas
- Content creators - Ideas/Inspiration for future content creation (daily frequency)
- Hobby Interests - Save interesting content that they can reference later such as recipes, travel collections (weekly frequency use case)
- Brand - Post content on their page and would want to see how is it resonating with their followers (Frequency can depend on nature of brand)
- There can be other personas identified with different use-cases & natural frequency. Is there any important persona or use-case that I have missed that you need me to include ?
User Flow
- From feed, user clicks on more menu button and clicks on Save Post
- User can save post to an existing collection or create new collection and save it there
- User visits save page from navigation menu to see Saved Posts Page organized by collection
- User clicks on a collection to see saved posts list
- User clicks on a post from collection feed
- Brand posts content on their page
- Followers receive notification or see it one their feed
- User saves post
- Question - Is there a nudge for users to discover this feature ? If not, then defining the set-up moment becomes difficult in the activation journey.
High-level Goal/Objective of feature
- Indication of interest for future recommendation of content to increase engagement and hence increased monetization
Prioritization
- I am focusing on the hobby interest savers with a weekly frequency use case as it would be a larger portion of the audience
Constellation of Metrics
- Retention
- Content Creators - Daily Active Savers, Daily Active Reviewers (who come back to review saved posts)
- Hobby Interests - Weekly Active Savers, Weekly Active Reviewers (who come back to review saved posts)
- Activation
- Set-up - % users seeing nudge to save a post within x days (assuming this is already done)
- Aha - % users reviewing x posts within first x days (value realization is more from reviewing a saved post than actually saving a post)
- Habit
- % savers reviewing saved posts within x days of saving
- % users saving x Posts within first x days
- The values for x need to be identified based on correlation analysis with long term retention, and previous stage metrics for different values of number of posts and number of days
- Engagement
- Cumulative
- Total Posts Reviewed per week (High level for trends)
- Total Saved Posts Saved per week
- Cumulative / user
- Avg. Posts Reviewed Per User per period
- Avg. Saved Posts Saved Per User per period
- Segmentation
- Power - % Users reviewing > xx (say 30) Posts per period
- Core - % Users reviewing > xx (say 30) Posts per period
- Casual - % Users reviewing > xx (say 30) Posts per period
- The values for xx need to be identified based on correlation analysis for different values of number of posts
- Cumulative
- Monetization
- Average revenue per active saver
- Ecosystem Metrics
- Incremental avg. revenue between saver and non-saver
- Incremental avg. time spent between saver and non-savers
- Health Metrics
- Error rate of Save API
- Latency of Save API
- Satisfaction
- Customer effort score
- Time spent distribution from starting a save to actual saving
- Sean Ellis style survey for feature-fit
- Referral
- % of users sharing saved posts
North Star / Success Metric
- Depending on goal/objective for this feature we can define north star / success metric
- Improve Engagement
- Impact Metric
- Avg. incremental time spent between saver and non-saver users (assuming positive correlation)
- Outcome Metric
- Activation - I would want to identify which part of the funnel has the maximum drop-off in activation - Set-up, Aha or Habit and try to focus on improving that
- Engagement - If activation is saturated then I would focus on engagement metric based on strategy
- Increase intensity
- Avg. Post Reviewed Per User Per Period
- Avg. Post Saved Per User Per Period
- Increase frequency
- Avg. Post Reviewed Per Session Per User
- Avg. Post Saved Per Session Per User
- Increase intensity
- Monetization - Incremental average revenue per saver
- Impact Metric
- Improve Monetization
- Impact Metric
- Incremental average revenue per saver (assuming positive correlation on engagement & hence monetization potential)
- Outcome Metric
- Activation - I would want to identify which part of the funnel has the maximum drop-off in activation - Set-up, Aha or Habit and try to focus on improving that
- Engagement - If activation is saturated then I would focus on engagement metric based on strategy
- Overall - Avg. incremental time spent between saver and non-saver users
- Increase intensity
- Avg. Post Reviewed Per User Per Period
- Avg. Post Saved Per User Per Period
- Increase frequency
- Avg. Post Reviewed Per Session Per User
- Avg. Post Saved Per Session Per User
- Impact Metric
- Improve Engagement
Silver PM
First I will make a Hypothesis that why user will use Save feature.
Hypothesis: User will only save those things which they have to look back in future.
What can be things which users can look back in future and are on Facebook.
- Post
- Inbox messages
These are the things which user wants to see in future
- To save something closer to them, any memories posted by friends and family.
- To save any joke / meme.
- To save something which can increase there knowledge later on like some study material or updates.
- Some important message shared by someone on facebook which can be a link, image or text.
- Discoverablity : % of users using save feature.
- Usablity : Save items distribution (Feed vs messages)
- Revisit : % of users opening saved posts (% is out total users who have saved an item)
- No. of Post Saved in a week
- Time spent on Revisit
- Type of Post they are saving : Eg images, short videos or just text posts.
Looking forward for the Feedback!!!!
PM
So I'd define the main metric of the Save feature as:
"The number of used items in the Save list" devided by "The number of items in the Save list".
Clarify : Is this Save feature where you save your favorites? Does this include any pics/posts?
Constraints: Global/Regional. Any timelines by which this needs to be concluded. Resource constraints? Platform - mobile or desktop? Any preferred target segment?
Assumption: Global. No constraints. Yes this is the save on fb. For mobile & desktop
Structure:
Company mission
Company maturity
Product maturity & goal
Business metrics
User metrics
- Segments
- Primary, secondary metrics
- Counter metrics
Summarize
FB mission - give people power to share and make people more connected
Product goal: Goal of the save feature. Lets users pin their favorites so that they can see those posts later and also share with their friends. This is because out of the long feed, it is possible that you may have missed these.
Company maturity - fb is a mature company and revenue is the overall goal at this point including retention.
Product maturity
Now the product is fairly new which means acquisition and engagement are more the north star
North Star
Acquisition/Engagement
Business metrics: WAU & MAU (how many times this feature is used in a week or a month)
Counter Metrics: Is this feature reducing overall shares/comments/likes?
2 types of users:
- People who post content - these can be marketeers or users - Creators
- People who are saving content - Users
I will cover metrics for both segments.
Segment 1: Creators
Going through the customer journey, this is how content is created:
- Creators post content which shows up on users feed
- This gets saved by a user
- It has a lifetime and sits in their list for an amount of time
- At a later stage, this is viewed and the creator gets a like/comment or inturn this is shared with somebody else
NSM : Creator should post content that is relevant and is 'save able'. Hence, a good NSM here would be to 'no. of comments/likes/shares done on a post after x days of posting'
Secondary metrics :
- No. of saves on a post
- Time lapsed between saving and posting a comment
- Views on a saved post
Counter metrics
- Posts saved may never be viewed
Segment 2: Users
Going through the customer journey, this is how posts/videos/content is saved and accessed later
- A users sees content on his/her feed
- They have no time to view this and wants to view it later. Saves this
- When the user has time, goes into his saved items and views it
- Likes/comments/shares
NSM: Users should know that he can save and also find the content 'good/interesting enough' to be shared or liked at a later stage
Hence, a good NSM here would be 'How many posts has he retrieved from his 'saved' category'
Secondary Metrics:
- Are users aware of this feature? No. of times 'save' is used for a post
- Frequency of times, the 'save' feature is accessed
- No. of likes/comments/shared on that saved post
Counter metrics:
- Ratio of saved posts vs. actual posts. For eg. If people are saving more posts to view later then they are being distracted which feeds into the limited time that they might spend on the app
- Hence, time spent on fb browsing posts . See if this is reducing overall
To summarize, to define success of the save feature on fb, we will look at a north star of increasing acquisition & engagement by looking at the following NSM:
NSM would be WAU, MAU for the Save feature. Hence, no of users using this feature within a week and a month and see if this is increasing
Counter Metrics would be to see if this hasn't cannibalized overall likes/comments/shares per user.
Problem/Pain point - to view posts, videos , page or event or link posted by friends or businesses at later stage.
Goal: To engage more users and collaborate people to express their feelings.
User journey:
Users log in into fb
Scrolls his/feed
Tap ... and select 'save post'
User keeps scrolling the feed and keep himself engaged for next 10-15 mins
User taps on menu bar
User then taps on 'Saved'
Users now see his saved post /video/link/page post /event
List of metrics
1. No of users using 'save' / total no of fb users ( daily or weekly)
2. No of users who sign up and use save Feature within a day
3. Avg no of users who use ' save' feature more than 5+ ( daily)
4. Users viewing the saved post/total no users who saved the post /videos ( weekly or monthly )
Most important metrics :
Since Fb mission to connect more people and increase engagement and retention.
Therefore engagement and retention are most vital metrics
I will focus on 1,3,4 metrics to evaluate the performance of 'save' feature.
I’d like to first address what this product is, who are the users, and how it brings value to them.
The Save feature helps users bookmark items (posts, videos, photos, events...) they are interested in and bookmark them for later. This helps with FB’s mission by allowing users to bookmark those moments shared by the rest the community that resonate with them.
This is a multi-sided ecosystem made of:
- the ones who post the content that is saved. They can be other users or Pages like business pages
- users who bookmark the content that is relevant to them but cannot access it now and want to return to it later.
For Fb, the value comes in (1) create a better user experience by addressing FOMO (posts thar are of value to user who just 'dissapear' in the feed) and (2) business pages might potentially get more viz for their products which in turn might lead to a better experiences for businesses on the platform. This might lead to an increased revenue for Fb.
I would like to start brainstorming a North Star metric, then break it down into smaller, measurable metrics for each category of user. Finally, I’d like to think about some counter metrics or downstream metrics that we should keep in mind while developing the product. I will choose the North star metric by assuming that the goal of this feature is to increase engagement with pages in general, and business pages in particular.
A good North star metric is at the interesection in value between users (who want to be able to find that post later) and pages (who want to ensure their content does not get lost in a multitude of posts in feed and they can target users better).
North Star metric: % of MAU who has saved a post from a business page at least once
Metrics:
% of saved posts that come from pages -> has engagement (e.g. total interactions per user) with those business pages increased after saving a post?
Total num of saved post per month by source (e.g. total posts coming from a business page saved per month vs other users)
Median number of posts saved per user per month
Demographics: which cohorts are saving posts/links/videos the most (e.g. Young adults, females etc)
What type of content is most often saved (e.g. video,text)
Source of saved posts (e.g. groups, feed, marketplace)
Down metrics /Counter-metrics
% of saved posts that are not opened again after saving
Understanding the problem
First off let's establish a common understanding of the product by briefly describing it and how users use it. The Save feature allows Facebook users to save an individual post, page, or photo for viewing at a later point. Users have a "Saved" section of FB where they can easily access all of the different pieces of content they've saved. You could save anything from a funny meme video to your grand daughter's prom photos.
In order to set the stage for this problem I think it's important to examine the product in the context of it's parent company and their mission. Facebook's mission statement is to bring the world closer together and give people the power to build community. The Save feature makes it easiest for people to connect virtually with others by letting them consume the content at a time that is convenient for them.
Metrics
There are a ton of different metrics we could use to monitor the Save feature and one of the ways I like to organize them is by the different steps in the user lifecycle. This just helps add structure and relates it back to the bigger picture.
- Awareness - # of users who have moused over or clicked on the save button
- Activation - # of users who have viewed content from their "Saved" section
- Engagement - Average number of times a user views a saved item per week, number of users who viewed a saved item at least twice in a week, average number of saved items per user
- Retention - % of users who have viewed a saved item at least twice in a week who also viewd a savied at least twice in the week prior
- Monetization - Let's ignore, not really a monetization feature. Ideally increases time spent on platform which means more display ads, or more user data on what the user likes so better ad targeting
- Referral - N/A, would be something like number of times someone suggests someone else save something. Let's ignore.
PM
How i would approach this problem:
Clarifying Questions:
What Save feature is:
Save feature empower people save any type of posts for later viewing.
Answer:
Framework i will follow:
- Facebook Mission
- How Save feature helps fb mission
- Save feature mission
- Objective of Save feature
- User Actions
- Metrics
- Success Metric
- Tradeoff metrics
- Counter Metrics
- Summary
Execution:
Facebook Mission: Help people build community and bring people together
How Save feature help fb mission: It helps people save the posts and reflect back on them when needed which may prompt long term interactions on the posts leading to people discussions and eventually people coming together to learn, grow and do so many other things while discussing on the post.
Save Feature Mission: Help access Any post Anytime, Anywhere.
Save Feature Objective: Assuming its in early days of roll out since its not very well known, i would focus on engagement of the product and not on growth/adoption of the feature which we will do once we feel there is product market fit (i.e good retention).
User Actions:
- Open App
- Save a post
- Access a post
- Interact (comment, reactions etc)
Metrics:
User Adoption: WAU viewing saved list
User Engagement:
- Total Saves
- Total View of Save list
- Total saved posts access/viewed
- Total Interactions
User Retention: WAU viewing/accessing saved posts
Prioritization of Engagement metrics: Do Pros and Cons of each metric.
Pros Cons
- Total Saves. - Shows inputs to feature |. Not engagement
- Total View of Save list - Show intent to access|. Not engagement
- Total saved posts access/viewed - Shows user engagement with feature | Does not tell about retention
- Total Interactions- Shows user deeper engagement| Not every person interact with posts they view
** Leaving out Monetisation metrics since currently, we plan to focus on engagement.
Tradeoff metrics: Retention Metrics - Adoption Metrics - Levers to accessing the posts.
Counter metrics:
- Saved posts accessed by few users only will give the wrong indication hence posts viewed/user should be a counter metric.
- Total posts viewed by a bot activity
- Total posts viewed but flagged as incorrect posts save
Summary:
For save feature total saved posts accessed per week should be a success metric we should track while keeping counter metrics in consideration.
Describe the product feature – FB is a social media platform which is used to share the information, emotions, communication between the connected people.
Clarification
What all things will be in scope of save feature? Assuming all type of content video, links, text, pictures.
What is defined as success? Assuming we would like to measure the customer engagement activity with respect to the ‘SAVE’ feature.
Have we recently introduced this feature? Assuming the answer is yes
What is save feature?
It is used to save the content like videos, links, text so that users can see it at a later stage. This drives the customer engagement on the social media platform.
Feature Value from content viewer standpoint: It is been used by the individuals who does not have time to watch the content right away but as it is relevant so they want to watch it later. The feature is also useful for the android users who would like to use the content offline when internet connectivity is not available. User would also like to use this feature as there is wide variety of content available like posts, videos, links, picture so there are high chances that the content might be missed out if the user leaves it for the time being without using Save feature.
Content viewers can be of two types like casual users who want to view the data just for entertainment or knowledge and the business users who want to see the content to buy a product or take a decision based on the available knowledge.
Feature Value from content creator stand point- As a content creator I might realise that some changes are needed in the required content like video, posts, picture message etc after posting to the public. Hence, I want to bookmark or save them so that I keep a track of it to review and edit the same.
Goal of the application with respect to Save feature is to increase customer engagement so that a greater number of users should use the feature to view the content with ease at later point of time.
Metrics use to measure the customer engagement
1. The increase in the number of people who are watching the content before the Save feature was available and the percentage increase on it.
2. How many times the same viewer is watching the same saved content.
3. Does the save feature has increased the number of hits on the big size content like videos as users can save them to be viewed at later stage
4. Number of hits on the save feature and cancel
5. Does the number of comments on content has been increased as the viewers might comment on the content after watching the same.
6. How many users are using the save feature just after registering themselves?
7. How frequently the users are using the save feature?
8. Is there rise in the number of shares for the content has been saved and watched by the user?
9. The percentage of users are searching the content based on the save content that they watch previously
Evaluation
Reach- The save feature will increase the user base as the content can been seen even offline. This is definitely going to increase the customer base for facebook
Impact- The impact will be high as the people would like spend time to watch the content during there leisure time that might have been missed out in the flow while scrolling the page. There are high chances that the content viewers who is using the save feature will start watching the same type of content type again.
Silver PM
What is the feature: To allow users to save content they like/ create collections that they can come back and view later.
The goal of the feature:
#1 Drive up users' engagement with the platform by allowing them to come back and view their curated content.
#2 Creating a data loop for FB where it receives more data about its users and can use that to show more personalized content and targeted ads. This helps create more revenue for Facebook.
Here I will choose #1 as my goal.
User Journey: I scroll through my feed. I find a post I like and want to come back to it later, I save it in my collections. Later I go to my collections and view my saved posts.
Metrics:
Acquisition/Activation: Have users started using the product?
- How many users are sharing posts? Is this number increasing over time?
- Avg saved posts per user
- Ratio of the number of posts saved to posts viewed. Is this ratio increasing over time?
Is the feature being used as intended?
- Are users saving posts and not coming back to them?
- Are users saving posts accidentally and deleting them immediately?
Engagement: Is this feature driving up user engagement with the platform?
- Avg number of times a saved post is revisited
- Avg time being spent by users on FB. How much of that is attributed to viewing saved posts?
- What is driving up the feature usage? Photos, videos, etc. / Demographic information
Retention: Is this feature causing users to come back to the platform regularly?
- Retention of users who saved posts v/s those who don't save posts. Has it changed?
Referral: Are users getting other users to use the save feature? By curating collections together?
- Percentage of users saving posts, who initially started off by adding to someone else's collection
- Saved posts/collections being shared with other users
Evaluation of metrics (Measured on a scale of 1-3. 1 being Lowest, 3 being Highest)
| Metric | Impact | Complexity in measuring |
| Number of users sharing posts | 3 | 1 |
| Avg saved posts per user | 2 | 1 |
| The ratio of the number of posts saved to posts viewed | 2 | 3 |
| Average time spent per user on saved posts | 2 | 3 |
| Avg number of times a saved post is revisited | 3 | 2 |
| Retention of users who saved posts v/s those who don't save posts. | 3 | 2 |
| Percentage of users saving posts, who initially started off by adding to someone else's collection | 2 | 3 |
| Saved posts/collections being shared with other users | 2 | 1 |
Based on the above evaluation matrix, I would like to prioritize the following metrics:
1. Number of users sharing posts
2. Avg number of times a saved post is revisited
3. Retention of users who saved posts v/s those who don't save posts.
Gold PM
Describe the product:
'Save' feature enables users to save a post or an ad that they want to see in more detail but do not have time at that moment to do so.
Clarifying question:
Since 'Save' feature has been there for few years now, I will assume that it is in its growth phase. is that correct? (Yes)
Goal: This feature helps user save something of interest in a specific category so user can check it later when he/she has time. This helps with user engagement as well as FB Revenue. I will prioritize increasing Engagement as Goal as I believe Revenue will automatically follow higher engagement.
User Journey:
User scrolls through feed, sees items of ineterste- post, sponsored ad etc. Reads comments but does not have enough time at that moment. User wants to come back and see more details, so user saves the post in some category.
Later user remembers to go back on his/her own OR FB reminds user to check the saved items.
User goes back, checks the details. May React, Comment, Click on the link, May make a purchase etc.
Metrics for defining success of 'Save' feature:
- # of post Saved per month and trend over time tells how often it is used
- # of posts saved per user per month tells how wide speard is the adoption and its trend
- Average # of categories created per user
- % of time user clicks on the reminders and goes to see Saved items tells how reminder feature is working to help Save feature'
- CTR on the links
- Conversion Rate
- Revenue generated
- Average # of reactions and comments on Saved posts as compared to Non Saved posts
Prioritize:
- # of posts Saved per month and trend over time tells how often it is used
0 - Question: Since the “Save” feature was introduced several years ago, is it safe to assume that it’s on the mature stage? - Yes.
1 - Product Description:
The “Save” is the FB feature that allows users to save things they see on Facebook to view later, like the links or videos their friend's post, events, Pages, or photos.
2 - Facebook Mission
Facebook's mission is “To empower people to build community and bring the world closer together”.
3 - Product Goals
For Facebook:
The “Save” feature allows users to watch more of other people’s content, which they would otherwise miss. This, in turn, should increase engagement between users (spark new conversations, increase the number of comments/shared posts/ etc) and therefore contribute to the overall FB mission. Also, I assume that the usage of this feature should increase the amount of content users consume -> amount of time spent on site -> amount of ADs they see -> revenue for the company.
For users:
On the other hand, the opportunity to “Save” content makes browsing Facebook much more comfortable and stressless for users. Firstly, they will be able to watch specific content at the most convenient time, accessing the information that they would otherwise miss. Secondly, they can save the content that they really like to watch again later and engage with other users regarding it.
Since the feature is in a mature stage and directed towards increasing users' engagement with the content and between each other, I would focus on Engagement & Retention metrics.
4 - Users
Who creates the content (post/video, etc.)
Who views the content
For the sake of this exercise, I would prioritize the second group of users (who view the content), because they are the main beneficiaries of this feature.
5 - User Journey - Actions & Metrics
Step | Action | Metric |
1 | Scroll the Newsfeed | Avg #posts scrolled per user per day Avg time spent per user per day scrolling the Newsfeed |
2 | Stop at interesting post | Avg #stops per #of posts scrolled per user |
3 | Review it | Avg #time spent reviewing the post base on a type (text, video, posts with a picture, etc) |
4 | Decide if they want to Watch it now / Save for later / Skip completely. | Total #of users who "Saved" at least 1 post ever Total # of users who "Save" at least 3 posts per 4 weeks Total # of postes "Saved" per day Avg # of "Saved" posts per user per day/week/month % of "Saved" posts out of all new posts per day |
a) | Option 1. Watch it now. After they watch/read the content they can “Save” it. If they "Save" it, then: | % of users who decided to watch it now AND "Save" it after that % of posts "Saved" after users watched the content |
Never watch it | % of users who have never watched it after | |
Watch it and remove it from the saved posts | % of users who watched it and then removed it after | |
Watch it and keep it in the saved posts | % of users who watched it and kept it % of users who watched it multiple times (>3 times, for example) | |
b) | Option 2. Watch later. Click “Save” to watch later. | % of users who decided to don't watch it now and "Save" it for later % of posts "Saved" by users who didn't watch the content at this stage (didn't spend more than 15 sec on it, for example) |
Never watch it | % of users who have never watched it after | |
Watch it and remove it from the saved posts | % of users who watched it and then removed it after | |
Watch it and keep it in the saved posts | % of users who watched it and kept it % of users who watched it multiple times (>3 times, for example) | |
c) | Option 3. After quick review (<15 sec) they realize that they are not interested in the post at all. | % users who ignore the post after the quick review |
5 | (Optional) Interact with the post - Like/Share/Comment | % of users who interact with the post after reviewing it in the "Save" tab Avg # interactions per Option 1 VS Option 2 "Saved" posts (users "Saved" the post because they didn't have time to watch it VS "Saved" because users liked it and want to watch again" |
6 | Continue to scroll or close the app | Avg. # posts scrolled through by users who "Saved" a post VS who didn't "Save" a post |
7 | Users return to the app to watch "Saved" content | Avg. # of logins per day when the user "Saved" content VS days when the same users didn't save content. Avg time spent on FB per user on day when they "Saved" content VS days when they don't save content % of posts that were watched/reviewed after being saved within 10 days. |
6 - Prioritization
Since our focus should be on Engagement & Retention, I would choose 3 following metrics to measure the success:
Avg time spent on FB per user on a day when they "Saved" content VS days when they don't save content.
Total # of users who "Save" at least 3 posts per 4 weeks.
% of posts that were watched/reviewed within 10 days after being saved.
Bronze PM
| Question | If you were the PM for the Save feature at Facebook, what metrics would you use to define the success of this feature? |
| Clarifying questions | Want to be sure the functionality of this fetaure: A user can click at any item in the news feed and click on the save CTA to view it later |
| Customer use cases | At least few different use cases |
| 1) I do not have the time to watch/read the content piece right now and I would want to revisit this later. | |
| 2) I really liked the content and I want to bookmark it so that I can visit this again and again. (Maybe performance by an artist that I am a fan of or an educational video that may help me prep for my exams) | |
| 3) I have already consumed the content and I need to respond to this later, as I do not have the time to do so right now. (Fairly common for political debates) | |
| Business case | Increase engagement at platform level, as by saving the user has indicated their interest in the content |
| Possible user journeys | Save -> Revisit -> Consume -> Unsave (Optional) |
| Save -> Revisit -> Consume -> Revisit -> Consume (repeat few times) -> Unsave (Optional) | |
| Save -> Revisit -> Respond -> Unsave (Optional) | |
| Save -> Forget to revisit | |
| Save -> Revisit the list of saved items -> Get lost in the long list of saved videos (Happens to me other bookmarking services I use) | |
| First three journeys represent the feature as planned. The other two journeys are less than ideal. | |
| Metrics | |
| At save-level | Number of users using the save feature |
| Number of times "Save" feature is being used | |
| Both metrics should be measured separately for save without full consumption (revisit use case) and saved after full consumption (bookmark use case) | |
| At revisit level | Number of times saved content is revisited and consumed |
| Number of users revisiting the saved content | |
| Engagement level for saved content (Likes, comments, shares etc.) The hypothesis here is that the engagement level should be higher as the user has already indicated their interest in the content | |
| The ratio of revisits to save (We would need to drive this metric up through product interventions like reminders) | |
| Number of unsaves after consumption (This may be a useful metric too, as we may not want the saved-list to be too long) | |
| At post comment level | The hypothesis here is that when a user takes the time to save and respond later, they may draft a more well-thought response as compared to a knee-jerk reaction in real-time |
| Length of response or comment accompanying the share | |
| Engagement level for others to this response (likes, comments etc.) | |
| At individual content piece level | Which posts and which kind of posts are getting saved the most? |
| Which ones are getting revisited the most? | |
| Since save indicates a very high level of interest, this can later be used as one of the feed ranking signals. | |
| At feed/session level | We may also want to check if having a save-feature is letting users postpone the content consumption, which otherwise they may have done in the same session. Keep in mind that the user may not revisit the saved content. |
| Average session time (Need to check if these are going down) | |
| Total minutes spent by the user on the platform |
Bronze PM
- Clarify the feature - Saves helps the user to save a particular content (video, post etc) to watch later
- Goal: define the overall health
- Metrics:
- Adoption:
- Avg # of saves/user - This will tell us whether an avg user is even aware of this feature or UI needs to be more obvious like the placement of the button or increasing the size or giving a small pop with a super simple message, something like "Save now to read this later"
- Cohort by new (0-1 year old on FB), young (1-3), adult (3+ years) to understand which segment needs more awareness. It could be new segment as new to platform or old as the old segment could be 40+ years old
- Avg # of saves/user - This will tell us whether an avg user is even aware of this feature or UI needs to be more obvious like the placement of the button or increasing the size or giving a small pop with a super simple message, something like "Save now to read this later"
- Engagement:
- % of the content (posts/videos etc) opened back after saving/user
- Cohort by light (0-1 saves/wk), moderate (1-5 saves/wk), heavy (5+ saves/wk) users
- Funnel metrics -- maybe user saves too many that when he goes he is not able to find. So, maybe we need to categorize or give user some sort of search functionality or sort by some specific order like most recent on top
- We can design a test to understand the drop off
- Test and control groups would be the users who have saved content > X, say 10 posts, and who go to the page with saved videos
- Metric: % of people that open at least 1 video
- Goal: Increase the % of users from x to y, a z% improvement
- Duration: Until we get conclusive results
- Size: Work with engg team to decide this
- We can design a test to understand the drop off
- % of total time spent on the content saved
- % of likes, comments, shares, sent in DMs/user from the Saved items out of total/user
- % of the content (posts/videos etc) opened back after saving/user
- Retention:
- # of saves/user every month
- % of users that saved 1 or more post, say last week, saved this week as well.
- Revenue:
- % of revenue coming from the Impressions/clicks/conversions from the ads saved
- Adoption:
- Prioritize Avg # of saves/user and % of the content (posts/videos etc) opened back after saving/user as it is driving adoption and engagement. I believe if they increase it will increase retention and revenue down the road as people will be spending more time.
PM
At first, I would start with asking clarifying questions...
If I Know about the product feature... I will map the user journey of the product and ask the interviewer if he is fine with it.
If I don't know about the product feature...I will give and get a 2-liner on the product feature and then map the user Journey.
Before mapping user journey... I would also ask a question as to what is business goal in this context. If the interviewer is in general view of "Success" then:
My clear view of defining success of a feature lies in defining High priority Metrics across various sections... like I want to have a list of most important metrics in Acquisition, Actiavtion, Enagagement and Monetization.
So to begin with, Let me map a sample user journey for Facebook "Save":
1. User is going through his posts and finds some post or a news feed very inetresting but doesn't have the time to explore and will "Save" it for later.
2. User will re-login after some time whenever he is free and goes to the particular category of "Saves" he is interested in.
3. User clicks in Saved Content.
4. User expresses his feelings on the saved item (like,share,comment) [Optional].
I will ask the interviewer to add any steps which I had missed out... or do I need to add any more steps.
Now coming to Metrics:
Acquistion focuses on increasing user base for the "Save" Feature.
People will use save feature more and more if right content and huge amount of similar content is available to them.
So the most important metrics for Acquistion/Activation would be:
# No of users who used "Save", # No of Users who added new categories and interests to his list.
So the most important metrics for Engagement would be:
# No. of Saved posts revisited per user in 1d/1w/1mo.
# No of new Saved posts per user in 1d/1w/1mo
So the most important metrics for Monetization would be:
# Normal facebook metrics like CPM,CPR,CPC.
Platinum PM
Interviewer: you got the sense of the feaTure yes. Essentially, anything Save. It doesn't matter if you talk about SAve section or ability to save because they are tied together.
I: Yes, that makes sense. So I think it will be helpful to thinkabout why Save is important for FB. In anything in life, to know if you are successful at someething there has to be a goal that you know or else you don't know if you got success. Once we have that, I think we can then start thinking about how we could measure success and maybe I will be able to pin point the most important success measure. Does this sound like an acceptable appproach?
Interviewer: Yes
I: Saving someting enables me to be able to easily find something I had liked or didnt have a chance to explore when I found it. It also enables me to always have something to send to a friend or let's say to a date and helps me have a conversation topic and seem engaging. Why FB cares is because FB is in the business of connecting the world and FB makes money from advertising mostly. By making it easy for me to find something I had liked, it makes me want to get back to FB. When I do that, FB likely makes money from CTR and Impression on various feeds from marketers. So ultimately it's a monetization play for FB but FB can do that by increasing user engagement. Save I think is a User Engagement play.
Interviewer: Ok sound approach. Let' go with that.
I: There are two types of users who would want to save things a) Consumers like you and I b)Businesses for things like having a marketing content that they oculd save and come back to to publish again during another campaign or something. Let's see how user like me , a consumer uses save feature.
I: While navigating FB I like something. I save it. Later I am talking to a firend and someting comes up and want to share a context or want to share something funny. So I quickly log in to FB. When I do that I invariably end up checking out more things and in the process end up seeing an ad or something. But FB is so sticky that I end up spending another 15 min on it too. So I think the way to think about success for Save is how sticky have I become. I think that's one way. So let's see:
1. # Saves a month (would like to see these tick up) - tick up tells me that I am viewing more and more content or else I woudn't be saving more. More content watched means more revenue for FB and it also means increased user engagement.
2. % of Save shared vs. Saves made / month
3. % of time I spend on each FB session where I navigate to Saved section on FB vs % of time I spend on FB during sessions in which I don't navigate to Save (seeing a tick up on this tells me that everytime I go to Save I am spending more and more time)
I think base level metric that one must meausure is #1 and #3 kind of goes along with that but that could be the 2nd one to measure. The only thing with #3 is that it's not a clean cut metric becasue whe n I got to SAved section even if I am spending more time I could just be viewing Saved section. That may not necessarily add more revenue for FB because maybe I have only added content that don't generate revenue. But #1 means I am spending more time watching bunch of stuff or else I woudln't be adding to # of saved items.
Silver PM
Hello,
This is my first try at answering a question on the website. Please feel free to give me your feedbacks and advice to improve.
Alright, just to clarify my understanding of the feature. The save feature is the button you click on when you see an item you wish to go back to and check again. Afterwards, if the user wishes to check the content again, he can go to the Saved section. Is that right ?
The feature has been live for some time, so the metrics we'll have to put in place will focus more on the engagement and retention level, rather than acquisition and adoption (which we will also monitor). Interms of methodology, I propose to validate the product goal, define different metrics where we will appoint one as the North Star, and then prioritize the different metrics.
Before going into the metrics, first, we'll have to agree on the product goal. Indeed, we as users see and consume a lot of content on social media whether it is on FB, Instagram or others : Some we might scroll over quickly, others might be of more value for us and we'll spend more time interacting with it. So the goal of the Save Feature is to make interesting and valuable content easier to access again for the user by making it available in the Saved tab, and thus give an incentive for the user to return and engage once more.
So in terms of metrics : We will define some to see if the feature is seen and used by the users. If so, do they engaged with it frequently ? And does it encourage the user to be more engaged on our platforms.
Stage | Metrics |
Acquisition / Adoption | % users that used the Save Feature at least once (this allows us to evaluate the penetration of the feature and if it is well known/used) # Saved items / user |
Engagement | # Visits to the Saved Tab / week (If the feature is used, does the user check his saved content again or not) # Content revisited (Does the user check the saved content again and at which frequency ?) |
Retention | # daily visits / user Time spent per visit ==> Does using the save feature encourage users to come back more and spend more time on the platform compared to the users that do not use it ? |
For me, the north metric of this feature would be the "# content revisited" that show us if the users are using the feature and if they are indeed interacting with the content they saved and if they come back to check it again.
Other metrics I would prioritize are also :
- % users that saved an item ==> This shows us if the feature is well known or if we need to inform the user of its benefits ? If we see that an adequate share of users already use the feature, we will replace it with the "# saved items per user"
- # daily visits / user and time spent ==> This allows us to see if we are indeed enriching the user experience and encouraging him to come back by making the content that interests him easily available
These metrics will help us evaluate if this feature is a success or not.
While these metrics show us how the users are interacting with the feature, some points to keep in mind :
- These metrics do not show us if the feature is well exposed and top of mind for the user ?
- Some content might incentivize the user to check it again frequently (like videos or articles), while some might just be reminders and will be less revisited (like a post from a friend listing an interesting tidbit)
- On retention metrics, these need to be put into context : It needs to be compared to users of similar habits (ie. If compared with power users, we might be less engaged for example)
The Save feature helps users save items for quickly accessing later.
When measuring, it's important to measure:
- Do users find the feature, understand it and use it?
- How are users using the feature? Is it as intended? E.g. are users archiving items they read through and liked for later; or are users taking a raincheck on items that look interesting but they don't have time to look into right now? What platforms are used to save and access?
- Do users access the saved items after saving them, and how frequently?
- How much time are users spending on saving and searching though the saved items, vs reading saved items?
- Can we learn anything about user journeys that start off from the saved items list?
- Could all this information help improve other features and verticals in the company?
- Engagement and data collection,
- Time on platform and exposure to ads,
- Social interaction,
- Indirect contribution to other products.
Silver PM
The save feature on facebook allows the user to save content that they like or find useful for them to want to refer back to. This helps the user organize the content and saves time for the the user when referring back to it. It also helps facebook to understand what type of posts the user saves both in terms of media type and content type. I will focus on saving posts vs other type of content.
The main goals here I would say is activation since it is a new feature and engagement since it shows the usefulness of the feature. These in results will indirectly generate revenue for facebook.
In terms of activation, I would measure % aware users that save, and # new users that save and % growth in the new users, % users who open saved content at least once
In terms of engagement, I would measure
% users who save in a month and % growth
Avg # saves per user per month and % growth
# of saves per user vs time spent on facebook
Avg # saves per post
# saves per content type (video vs audio vs image vs text), (food vs politics vs general) and % of opened save page per content type
# saves per user per number of x views per user
Avg # opened saved content per month and % growth
# of reopen saves per user vs time spent on facebook
Avg time spent on reopened content
To prioritize with relevance to business, impact to user, confidence and level of effort, I would say one activation metric would be relevant since the feature is new, but the focus and the goal should be engagement in a higher level. I decided to not choose awareness as facebook always launches new features and places them in ways for users to find them.
I choose the following activation metric as it would be most relevant to business in terms of its spillover to engagement (and thus revenue)
# new users that save and % growth in the new users
% users who open saved content at least once
In terms of the engagement metrics, here would be the focus:
# saves per content type (video vs audio vs image vs text), (food vs politics vs general) and % of opened save page per content type
% users who save in a month and % growth
Avg # saves per user per month and % growth
Avg # opened saved content per month and % growth
Summarize:
Facebook save feature allows users to save content they like to refer back to it later. It helps them save time and improve customer experience by finding the content they like. It helps facebook increase engagement, and thus revenue.
We brainstormed metrics focusing on activation since it is a new feature and engagement since this is the goal.
Success can mean many things. So, we have to find metrics which signal success for the Save Feature and then define which are our key metrics in those.
1. Growth Metrics :
Here, i will focus on growing the number of users who use tis feature and the number of content that is getting saved.
- How many new Users are using this feature?
- What is the percentage increase/decrease in New User count over 1 day,1 week, 1 month etc.?
- How many New Unique Posts are getting saved? What type of posts are they- Videos/Links etc.
2. Engagement and Activation
- what is the average number of times a user uses Save Feature every week?
- How is the 1st metric increasing with time?
- What is the number of posts which are getting Unsaved?
- What is the number of users who are unsaving more and saving less posts
- How many users Save 1 post, 2 posts, ....n posts? Where is the maxima?
- What is the average duration between 2 successive visits to the Save feature?
- Is this average duration increasing/decreaing with time?
3. Retention
- What is the number of Users who come back to watch the saved post?
- What is the average number of time a saved post is engaged with again?
- What number and percentage of users never come back- churn rate
4. Effect on other metrics
- Is something else moving down due to Save Feature?
- How does engagement compare in people who use Save versus those who don't?
- How does engagement compare in people who use Save over a time period?
I think we should select one metric each in growth, engagement , retention and other metrics to focus on.
In Growth, it should be the number of new users using this feature.
In Engagement it should be, the number of times any user uses this feature.
In Retention, it should be the number of users who use this feature again.
In Other metrics, it should be change in engagement of users using Save versus those who are not.
primary:
– # unique users that clicked ‘Save’ on post
– # unique users that clicked the “Saved” section
– # unique users that re-opened at least one saved post
– # items “saved”
– # times “Saved” section got clicked
– # times saved item re-opened
secondary:
– Overall DAU, WAU, MAU
– Stickiness (DAU/MAU)
– Average Mins per user spent on facebook before and after (platform, watching videos..)
– Monthly revenue from ads
Top Meta (Facebook) interview questions
- What is your favorite product? Why?89 answers | 263k views
- How would you design a bicycle renting app for tourists?62 answers | 82.5k views
- Build a product to buy and sell antiques.54 answers | 66.8k views
- See Meta (Facebook) PM Interview Questions
Top Metrics interview questions
- How would you measure the success of Facebook Likes?40 answers | 61.3k views
- How would you measure the success of Gmail?23 answers | 17k views
- What metrics would you look at as a product manager for Instagram ads?19 answers | 23.8k views
- See Metrics PM Interview Questions
Top Metrics interview questions
- What goals would you set for a Facebook Dating feature?14 answers | 9.4k views
- How do you measure success of the Facebook Fundraising feature?13 answers | 5.9k views
- How would you measure the success of Google Photos?13 answers | 11.4k views
- See Metrics PM Interview Questions
