How would you measure the success of Facebook Likes?

Understanding the problem

Generally speaking a "Facebook like" is pretty ubiquitous in the sense that most people know what they are but for the sake of thoroughness let's briefly recap. Users are able to give posts, stories, and pages a Like by tapping the heart icon that displays underneath these different types of content. The content creator will receive a notification alerting them to your like and the like counter will increase letting other viewers know that you've liked this. 

In order to set the stage for this problem I think it's important to examine this feature in the context of it's company and their mission. Facebook's mission is to bring people closer together and help build community. The ability to Like a piece of content directly furthers this mission by giving people a quick, easy, and positive way of interacting with the others through the content on the platform. This provides positive feedback and encourages the creator to make more content, which is essential because at it's core Facebook is a UGC platform.

Metrics

When it comes to likes on the Facebook platform there are both content viewers and creators. Viewers give likes and creators through their content receive them. We will discuss some potential metrics for each of these user groups. 

Amongst the customer lifecycle, there are 3 different steps in particular that are especially important to quantify and monitor through metrics:

1. Adoption (How many people are using the product)
2. Engagement (How much are they using it? Are they getting value out of it?)
3. Retention (Are they coming getting enough value to come back?)

how to measure success of facebook likes?

i would like to start by asking some clarifying questions

1. what do we mean by success - user engagement, fb product adoption/usage, user retention, new user acquistion?
2. i would assume it is engagement and usage, which would also be impacting user retention in long term - is this assumption correct?
3. are we referring here to any specific technology - web or app - mobile or desktop?
4. are we considering here any particular geography?

 

i would like to divide the users of FB into 3 segments for the sake of understanding their specific needs and using that to understand user behaviour

 

1. creator of post

2. viewer of post (who likes)

3. viewer of post (who views but doesn't like)

 

i would now like to start with focussing on the "why" and "how" part of users using facebook likes for each of the 3 segments:

 

for the creator of the post

1. acts as an indicator how good or bad the post is
2. motivates the creator to put more posts in future - getting more likes is the expectation of every creator before posting a new post
3. expands creators reach - more likes you get -> more users notice the post on their walls -> more popular you become
4. builds a perception of user's virtual image - interests - which might not be the real personality - but how s/he wants the world to believe who s/he is

 

for the viewer of the post (who likes)

1. impacts user's virtual image (perception) - though not to the same extent compared to that of the creator's 
2. acts as an input for the platform (FB) to provide similar content - a win for both business (FB) and user
3. contributes as a reference or guide for the user incase sh/e wants to create a post in future
4. easy for users to signal interest - user effort is minimal - learning curve is short
    

for the viewer of the post (who views but doesn't like) - tough to measure success?

1. is not comfortable disclosing preferences (likes) on social platform
2. genuinely doesn't like the post or content

 

how to measure success?

for the creator

1. is there a positive trend in the number of likes being received on the posts? (logged chronologically)
2. is there a positive trend in number of users viewing the post?
3. ratio of (number of likes received)/(number of users who viewed the post)
4. are the number of posts created per week/month increasing?
5. how many % of new friends/followers are from the users who are seeing the post for the first time? (might not be entirely valid for FB but would work for linkedin/insta) - is it a positive trend? 
6. how many % of existing friends/followers are liking the post?

 

for the viewer of post (who likes the post)

1. how many of the top % of recommended posts are being liked? - eg: FB can recommend 50 posts but top of the feed should be the ones which should be most relevant and should generate most likes -> would have a direct correlation to recommendation feed performance
2. metric1: (number of posts liked)/(posts viewed)
3. metric2: (top of the feed posts liked)/(posts recommended in top feed)

 

for the viewer of post (who only views but not likes the post)

1. average time spent on post over time - do we see a consistent behaviour?
2. for this segment, "likes" may not be useful - reccommendation -> to launch like version2 -> which allows users to - "give anonymous likes" 

 

 

Explain the Product or feature

·        Facebook is an application that is present on both Windows and the Web to connect with Friends and Family. People share interesting Videos, Photos, Location check-ins, Feeds, and stuff that they like on other people’s profiles.

·        The function of the like functionality is to show the appreciation of the content that is shared by Friends, Family, or Someone in the connection.

·        This is also a fast way to show appreciation/like as there are times when we do not have time to write a comment.

·        People sometimes also give like to bookmark content so that they can visit it later.

·        Also, to keep a track of who all are liking the Post coz once you like you have this in your feed and you can get notification on who all are commenting on the feed.

·        Facebook as a company also gets insights about what are users interested in, who is connected to who, and what people are liking.

 

Choose a Goal

Facebook’s main goal is to connect people and communities. And it's north Star metric I think should be time spent on the Facebook application sharing content and engaging with what friends and family are sharing on the Platform

‘Like’ feature have been there for a While so awareness, Acquisition, and activation will not be my primary focus here. I would like to Focus on User Engagement and User retention on the like Functionality.

User Journey

·        The user logs in on the Facebook application and sees feeds from people he Knows.

·        He gets Feeds about what the Known People are sharing like Pictures, videos, location, other People's content that they like, check-ins, etc.

·        User scrolls through the Feeds and like the content that is of his interest. In some of the feeds, he might like and comment. He can like, share and comment.

Metrics

Awareness

·        # of people hovering over the Like button

·        % of people Logging Vs Liking= # of people who hovered over Like/number of people who logged in* 100[Daily, Weekly, Monthly, Yearly]

Acquisition

# of distinct who clicked Like at least once who never clicked like before

% of people who clicked Like vs. logged in users= #users clicked Liked/# of user who loggedin*100[D,w,m,y]

Engagement

·        Number of Users Clicking on Like more than 5 times Per session [ MAU, WAU,DAU]

·        Number of Users who are Clicking Like and Commented

·        Average weekly Like Click vs user logged in

·        Average like per user.

·        Number of Likes Per user per week, month, year

·        Users who are clicking on Like, Commenting and Sharing as well.

·        # user clicking like and Sharing content.

·        Session Duration Vs number of Likes.

·        Segmentation

o   Content-Type: Videos, feeds, Pictures, Shares etc

o   Age Group: 12-18,18-32,32-50,50-60,60-above

o   Male/female:

o   Geographic location

Retention

% of users Clicking at least one like per day for a Week, Month, year

% of Users clicking at least 5 likes per week

 

Revenue

 

#of likes Vs. Number of ads scrolled

Clarifying Question

Me: Here we are discussing about FB Like on posts of other users or Likes in chats like FB messenger? 

Interviewer : FB likes on posts

Feature Description 

Facebook Like feature from:

User's perspective

1. action to show appreciation for the posted content 
2. improve the feed quality by utilizing personalization by FB based on interests 
3. signal intent to the person who has posted trying to build connections

1. better targeting of ads based on personal interests
2. getting more data about the user's interests which can be utilized in future 

User Journey 

• Scroll through FB feed
• Find something insteresting 
• Click on Like button

Metrics

1. avg. likes/ session - shows the discoverability of the like feature 
2. % of users who used FB and liked at least 1 post in last 1/7/30 days - shows adoption of the feature 
3. change in no. of posts vs avg. number of likes recieved on posts - shows if likes are motivating creators to share more content 
4. likes vs content type(image/ video/ text, etc.) - shows which type of content is getting highest engagement
5. likes/session vs session time - better personlized feed would lead to increase in engagement 

1) Clarify

By Facebook Likes, we are talking about that feature in posts (videos, picuteres, videos, ads) where people can quickly react with "likes, hearts, applauses, sad, etc", right?

Perfect! Meta's goal is to connect people, bring them together from all over the world. By using Facebook Likes, people get more engaged with each other, thus spending more time on the platform, which will ultimately contribute to the goal and revenue from ads. So "measuring success" I would assume that we want to see if this feature is indeed contributing to increase engagement, and impacting on revenue. Does it sound good?

Moreover, I would assume that we are trying to measure success in a broader way. In other words, I am not trying to measure success for a specific type of user, geographic location, platform, or timeframe. Is that ok?

Therefore, the first question becomes "How would you measure engagement/impact of Facebook Likes in a general way?"

2) Brainstorming

To keep in mind, when talking about success, we need to compare things to actually know if we are are doing good or bad, otherwise that number is just a number without any meaning.

- Does people know about this feature? 

        -- How many users use this feature at least one time per session, week, and month.

        -- How many users use this feature (not the like, but hearts, applauses, others) at least one time per session, week, and month.

- What is the usage distribution?

        -- How many users use the feature per type of post (friends, followers, pages, ads)

        -- How many users use the feature per content of post (videos, photos, text, etc)

- What is the session time?

        -- Session time on average from users who use the feature to those who do not use it.

        -- Session time on average per "type of user" (those who use more Likes, those who user other functions, per frequency of clicks)

- What about the posts that receive Likes?

        -- Reach/Views of posts who receive Likes versus those who do not (difference per quantity of likes received).

- What is the impact on revenue?

        -- Revenue generated from per user usage of likes. For instance, I want to know if a user who uses the feature 10 times in a week generates more or less revenue than an user who does just once in a week.

3) Prioritization

After this brainstorming, I would like to list the metrics by order of relevance. 

1. Meta Goal: Since Meta focuses on connecting people, meaning engagement, the first metric would be comparing session time of Daily Active Users who use the feature versus those who don't, or session time of Daily Active Users by frequency of feature's usage. 
2. Results Goal: Looking from a business goal perspective, I would analyze the revenue generate from those groups in the previous topic. Therefore, we could actually check if that engagement is translated into business results - revenue.
3. Feature Goal: We need to make sure that people are getting the most of it. So, I would look to metrics of frequency of usage per type of Like, drilled down to type of post and content. The idea is to gather insights of what, where, and how people use the most of it.

   

The first thing I would do is clarify a few things:

- What does the feature do? FB users can like each other's posts. status updates, photos, comments, ads, etc. For the sake of this question, I will limit it to posts.

- What is the intent of the feature? In my opinion, it is to create a community feeling when people can show that they care. This is a one-click process and much easier than posting a comment and increases interaction between people on the platform. This can also be used to help people differentiate between posts more popular and the lesser popular ones.

This eventually drives engagement with the platform and revenue. But those are the secondary goals here.

 

User Journey

- As a user, I scroll through my feed and find something interesting. Or that I care about. I immediately like the post. I may also see a post by someone I lost in touch with and have let him know that I care by liking his post. I continue scrolling through my feed. 

- As a user, I have posted a photo on which I seem to have received likes. I am intrigued to see who has reacted to my post. I am motivated to post more such photos.

 

The basic metrics to track here would be:

- Number of likes: This would be more valuable if we could measure the number of posts posted/ viewed against the number of likes. Only measuring the total number of likes is not very meaningful. Knowing that the number of likes per post viewed went up by 5% is a useful metric.

- Clicking on the 'likes received' notification

- Going through the list of people who liked the post

- Checking what is driving the success - photos, videos, status updates, etc.

Since the intent is to increase the interaction among users, I would measure how the interaction has increased using the following metrics:

- Number of FB requests sent and accepted

- Number of FB messages exchanged

- Are people returning back likes? Driving feature referral on the platform

 

The increase in likes will drive people to post more, hence increasing engagement with the platform and that could be measured by:

- Number of posts created

- Number of posts shared

- Time spent on the highly liked posts over total time spent: This will indicate whether the posts that are liked more are making people spend more time with the platform.

 

There are drawbacks that we should keep in mind: These metrics are quantitative in nature. If the metric drops, it would be difficult to know if this was because of the nature of the posts, or a new feature that FB released and we must be mindful of that.

 

Since we did not include monetization here, I did not talk about ads. If I were to include it, I would talk about ads > how many likes they receive > Is that being converted to an increase in revenue for ad sponsors

Thank you for the question. I will start by defining what Facebook Like is. Then I hope to discuss how it serves FB’s mission and get some clarification about any ongoing or upcoming business initiatives. This should provide us some good context before we dive into the metrics. 

 

1 Define

• What is the Facebook Like feature? 

  • a simple way to engage in the online community. The like button is a reaction that content consumers choose after reading something posted by content creators. 

  • provides positive feedback to content creators, and hence supports FB’s mission to empower users to build communities and bring the world closer together. 

 

2 Clarification / Scoping

• Are there any strategic initiatives that I should keep in mind? Am I focusing on any particular emerging markets? Or shall I keep in mind about any new product development initiatives?

• Facebook Like is part of multiple products, for instance, FB Messenger, Newsfeed, FB Live, etc. Would you like me to focus on any particular new product or simply consider it across all products? 

 

Alright. To sum it up here - let’s say we’d like to look at Facebook Like at a high-level as a whole across global markets and all product lines. The company-level north star metric is engagement related - we’d like to see users have quality interactions. The more varied activities users choose to start doing, the better and deeper the engagement.

 

3 User Segments

At a very high-level, I’d like to discuss two segments. Let’s think about the core activities or user flows around Facebook Likes. As I hope to lay out the input activities related to the metrics we care about, just to understand the various scenarios in which Facebook Like is used today...

• Segment 01: Content Creators (Business / Marketeer, Leisure) ——  They share firsthand or secondhand  information in various mediums (image, sound, video, text, hashtags), either asynchronously or streaming in real-time. They may adjust the privacy setting to configure who can see their content. The content can be uploaded and shared or re-shared from and to multiple channels.  

• Segment 02: Content Consumers (Mostly Leisure) —— They discover content in several ways, and land on a piece of information they show interest in, then decide to tag, like or comment on such content. Both creators and consumers may interact via conversation threads and react to each other’s comments. 

 

4 Success Metrics

Now, I’d like to discuss a few candidate, feature-level north star metrics. Later I will touch upon where metrics might fail. I’d like to think about content quality and virality. Here are 5 candidate metrics in my mind.

 

 

I’d like to go with a post-level metric for the creator segment, and a user-level metric for the consumer segment. 

• Creator: average daily items posted with 3 or more likes

• Consumer: time to 2nd Like (monthly)

 

Based on the above, I think it’d be good to keep these guardrail metrics, potential trade-offs, risks in mind in the meanwhile:

• Guardrail: content quality and ranking algorithm fairness

• Risks: reports of abuse or copyright infringement

• Tradeoffs: Likes vs Comments ratio, Likes by Content Types

 

Thank you for the question. I will start by defining what Facebook Like is. Then I hope to discuss how it serves FB’s mission and get some clarification about any ongoing or upcoming business initiatives. This should provide us some good context before we dive into the metrics. 

 

1 Define

• What is the Facebook Like feature? 

  • a simple way to engage in the online community. The like button is a reaction that content consumers choose after reading something posted by content creators. 

  • provides positive feedback to content creators, and hence supports FB’s mission to empower users to build communities and bring the world closer together. 

 

2 Clarification / Scoping

• Are there any strategic initiatives that I should keep in mind? Am I focusing on any particular emerging markets? Or shall I keep in mind about any new product development initiatives?

• Facebook Like is part of multiple products, for instance, FB Messenger, Newsfeed, FB Live, etc. Would you like me to focus on any particular new product or simply consider it across all products? 

 

Alright. To sum it up here - let’s say we’d like to look at Facebook Like at a high-level as a whole across global markets and all product lines. The company-level north star metric is engagement related - we’d like to see users have quality interactions. The more varied activities users choose to start doing, the better and deeper the engagement.

 

3 User Segments

At a very high-level, I’d like to discuss two segments. Let’s think about the core activities or user flows around Facebook Likes. As I hope to lay out the input activities related to the metrics we care about, just to understand the various scenarios in which Facebook Like is used today...

• Segment 01: Content Creators (Business / Marketeer, Leisure) ——  They share firsthand or secondhand  information in various mediums (image, sound, video, text, hashtags), either asynchronously or streaming in real-time. They may adjust the privacy setting to configure who can see their content. The content can be uploaded and shared or re-shared from and to multiple channels.  

• Segment 02: Content Consumers (Mostly Leisure) —— They discover content in several ways, and land on a piece of information they show interest in, then decide to tag, like or comment on such content. Both creators and consumers may interact via conversation threads and react to each other’s comments. 

 

4 Success Metrics

Now, I’d like to discuss a few candidate, feature-level north star metrics. Later I will where metrics might fail. I’d like to focus on content quality and virality. Here are 5 candidate metrics in my mind.

 

 

I’d like to go with a post-level metric for the creator segment, and a user-level metric for the consumer segment. 

• Creator: average daily items posted with 3 or more likes

• Consumer: time to 2nd Like (monthly)

 

Based on the above, I think it’d be good to keep these guardrail metrics, potential trade-offs, risks in mind in the meanwhile:

• Guardrail: content quality and ranking algorithm fairness

• Risks: reports of abuse or copyright infringement

• Tradeoffs: Likes vs Comments ratio

 

How to measure the success of FB likes.

Clarification 

What does success look like? 

Engagement -- this can be non-revenue-related interaction among users, groups.

Conversion -- this is specific to marketers, 3rd party business partners and their ROI in ad sales.

Then, there is end user satisfaction? How well does the Like functionality answer the needs of the end user, does it serve their primary purpose of hitting like button? Does it drive towards FB’s organizational principle of bringing people together?

 

Let’s settle on conversion as the Like functionality for FB traditional, non-marketing users, is pretty established. It’s the example of how you make a short-term investment of liking something and then receiving rewards -- engagement among your friends. But what is not certain is how likes impact revenue generation and ROI for 3rd party vendors and marketers.

So FB likes is a very easy way for a user to express interest in a post, product, event or thread.

Once someone likes a post, they are notified when others they know also like a post/product/event. It helps them engage with other users. It also signals interest when certain products pique the interest of consumers to FB marketers and 3rd party vendors and individuals trying sale their items. To evaluate this, I would begin with the HEART framewok and add a weight measurement for prioritization purposes.

 

Functionality

User likes a product as opposed to a post.

That like is saved so they can review what they liked.

User can immediately decide to purchase the product.

User receives purchase confirmation email

User receives shipment confirmation

User receive a out for delivery notification

User receives a notification when it has shipped.

User is asked to complete a customer survey regarding their experience

Once establishing the above metrics, I think it would be fare to assess the success of FB likes as it relates to actual purchases within the app.

One huge deficit with FB is security. Amazon spends a lot of time/money on security. Users gravitate to that marketplace for that purpose, in addition to the 2 or 3 day delivery time-frame. As a user, I have had issues with FB. Sketchy merchants who do not deliver or they deliver two months after purchase. If FB or other vendors want to compete in this makretplace, they will need more rigorous controls. That is not answered within this question but it should be addressed.

1. Clarify the product feature: Thumbs up reaction to any posts. Used to engage with the post to show that you like what you are seeing.
2. Goal: We want to measure its success.
3. Give structure - Metrics and prioritize them. Discuss how we can improve FB Like. Generate hypothesis and talk about tests that we can use to validate them. Discuss pitfalls and trade offs of  the test. End with summarizing. How does that sound?
4. Metrics:
   1. Adoption: This will help us understand whether the users are aware of the feature or do we need to make it more explicit, say for new users with age group of 40+
      1. % of FB WAU that have used the like button at least once
   2. Engagement:
      1. # of likes/user/week
   3. Retention:
      1. % of repeat daily/weekly users that like a post
   4. Revenue:
      1. % of revenue coming from liking the ads
   5. Counter:
      1. DAU
   6. Prioritize:
      1. Engagement and retention as those would be the key metrics at an overall level considering the product lifecycle stage (it is not a new product and has been there for while). If users engage and are retained, it will bring revenue down the road.
   7. Hypothesis: Likes might not be as intuitive for new users (0-1 years old) with age 40+ years old. We can find this out if the adoption for that cohort is comparatively lower than the other cohorts. My hypothesis is, it will be.
      1. We can run an AB test that will show a small blue box pointing the Like saying "click here to like the post". Once the user likes it, user will not be shown again on the same day. The user will be shown once the next time he logs in until the user makes an X day streak. Below is the designing:
         1. Population: NC + Age group 40+
         2. Randomization unit: Network clusters of user ids in the same network to overcome the network effect as if one user id likes the post, it can be liked by others too
         3. Power analysis to know size and duration
      2. Pitfalls:
         1. Novelty effect: Users might like because of the pop up but then start ignoring the pop up or the engagement might go back down after X days streak in which case we can test by increasing X days to see if it brings better results.
   8. Trade off:
      1. Say engagement increases but DAU goes down
         1. Rule out possibilities of DU regressing because of other reasons like seasonality, competition, other product feature changes
         2. If DAU goes down because of Likes, we want to take a business call whether we want user accounts that are not engaging with the simplest feature of FB like
   9. Summary:
      1. We discussed how we want to measure the success of FB likes. In order to do so, we clarified the product feature first. Then we moved on to define metrics for each of the overarching goals, such as, adoption, engagement, retention, and revenue along with the overall counter metric. We prioritized on the metrics based on the lifecycle of the product. We also discussed how we can further improve the product for a specific cohort, designed a test, and discussed pitfalls and trade offs.

 

Me: Are we talking about Like feature in any specific product of FB? 

I: No. upto you.

Me: Ok. I want to think about this from FB's mission perspective. FB's mission is to connect people across the globe, bring them together and make the world feel like a community. Thinking from that angle, while most products of FB are well alinged to that mission, I also think FB News Feed being the primary FB product has most amount of users so if I were to satisfy FB's mission I think we should look to success measures for FB Likes on News Feeds product of FB. This is also becasue Likes generates Engagement from other users. Is that ok?

I: Yes, well said.

Me: One more question I have is Likes feature now includes also reactions within. When we say Likes here is it ok to assume that we are specifically focused on the "Thumbs up" and not other reactions? 

I: Sure.

Me: Part of FB's user value comes from user interaction and I believe Likes works as a fuel to the fire to user interaction. That user interaction also has a positive business impact, revenue. Since user interaction and user value are tied together, Engagement is what we should focus on. Is that in alignment?

I: Sure.

Me: Next I would like to think of users and then come up with few metrics for success as it relates to Engagement. 

I: Sure

Me: 

% of users using Like / Wk - here I would like to see metric trend downward. Surprising but this is becasue I think Reacitons generate more excitment and hence user interaction. So, it adds more user value. Having said that, I would wnat to also measure,

% of users using Reactions / Wk - in comparison to 1st one, I would like to see this measure tick up. 

# of comments per post compared with # of Likes /Wk - Hypothesis here is what the more Likes I get the more I am prone to commenting. If this holds true, I know that Likes it working. This is bit counter to what I said in #1 so I think we should be also comparing against # of Reactions. 

% of posts where Like has been used at least once / Wk - If Likes is used on more posts then it tells me my users are liking that feature. 

I choose to measure / Wk since for a mature business, QTR would be too long of a timeframe to measure things. Week worth of data is plenty amount of data considering world's population is 8 billion and probably around 40% of users use FB News Feed actively which is about 3.2 billion. 

To prioritize 1 measure if we were to, I would prioritize # of comments per post compared with # of Likes /Wk since inceased # of comments leads to more people reading the article or watching entire video. This in turn also helps drive business impact and also adds more to user value by enteratining users and providing them a better user experience.

Is my thought process in alignment with that of yours? any questions?

I: No. this is good. Thank you.

I would answer this question with the following structure -

1) Define about the Like feature

2) Mention the goal of the feature

3) Talk about the user journey

4) Prepare goal-metric table

5) Prioritise metrics

6) Summarize

Like button is a feauture where users can like content such as status, ads, comments, photos, links etc. When a FB user clicks the Like button, it appears in the News Feed of other friends.

The goals of the features are as follows -

Easy or low-barrier way of communicating to the post holder that you enjoyed the post. Helps in improving the content coming up on Facebook.

Increase targeted communications to the users by ad companies

User journey -

1) Scrolling through Facebook

2) Came through a friend's post or any ad or any campaign

3) Clicked on Like button

4) I send a positive signal to my friend. 

5) Friend is motivated to share his life's happenings/quality content on social media

Goal	Description	Metric
Adoption	Increase content production and consumption	Average Likes per user (High), average likes per session (High), session length (Low)
Engagement	Increase the user engagement with Facebook in terms of time spent, frequency of visit etc.	Average Session length (High) , Login Frequency (High), Share/post Frequency (High), No of posts (High), Likes by content type (Video, Photo, Articles, Posts, Web Pages etc) (High) Likes by user type (age, geography, interests, time on platform etc) (High), Time taken to write a post (Medium).
Retention	Minimise unlikes and increase user enagement frequency for similar content types.	% of Unlikes (High), % of likes followed by comments/shares (Medium)
Monetisation	Better means of targeted ads	Click through rate (High), Length of Ad Video viewed by Like Volume by user (High)

So the goal of the 'Like' feature is to give feedback to users on how was their content and keep them motivated to keep posting in future as well. It enhances the quality of content posted and content consumed on FB. Engagement related metrics would be the best measures to figure out the success of Like feature with avg likes per user as the fundamental meaurement metric.

 

 

 

Step 1: Explain your understanding of the product 

 

Facebook likes is feature/way in which users can engage with other people's content. Like basically shows users that particular group/set of people have appreciated or liked their content and gives them a sense of happiness/validation. Like also tells friends in your network what kind of content you have liked and engaged in. 

Facebook like can be used on stories, chat, videos and images on the timeline, content on timeline, comments, pages and groups as well.

The mission of likes is to connect people across the world in the most non-obstrusive/simplest way possible. Like as a feature has been around since many years. It was one of the first few features that was launched by facebook. 

The different kind of user groups that use this feature are as follows

1) Businesses/Organisations

2) Casual users

3) Influencers/ Content creators/Celebrities. 

 

Step 2: Ask clarifying questions to narrow down the scope of the question/product. 

-What do you mean by measure the success of facebook like? (Anything particular that you have in mind or any particular direction that you'd like me to go or can i decide on my own)

-Are we considering facebook like as a whole or facebook like as a feature used in particular scenarios? (Let's go with facebook like for the newsfeed/timeline)

-What is the goal that we have in mind? (You decide)

-Web based or mobile based app? (Mobile)

-Android or IOS? (IOS)

-Are we considering a particular region to begin with? (Yes, US only)

-Any particular user group that we want to target and keep in mind? (Lets go with content creators) 

 

Step 3: Define the goal for the product and explain your reasoning for the same.  

 

The mission of facebook(facebook likes) is to connect people across the people and bring them closer together, thus helping them communicate and collaborate.

Facebook like was launched over a decade ago as one of the few starting out features which is still going strong. Owing to it's huger userbase, facebook like has a user awareness and user adoption for sure. 

However facebook like may sometimes seem too impersonal because of which people might not use like that much. Keeping this in mind and the end goal, we will be focusing on 2 goals here which are 

Engagement and Retention. Thus increasing user engagement and retention (time-spent using this feature) will in-turn drive the business/revenue for facebook as a company.

 

Step 4: List down the user journey 

1) Go to the facebook app

2) Sign up/Create an account 

3) Login in to your account 

4) Scroll down the content newsfeed. 

5) If you find something you can like it for the first time

6) Like post/image/news/video etc.

7) Like again

8) Facebook shows relatable content based on the kind of content you liked

9)Friend whose post you liked feels encouraged and motivated to create more content

10) Done liking and scrolling through the post

 11) Log-out

 

Step 5: List down the metrics that you'll track for the different phases of the customer journey

For the scope of this question we will be listing down only Engagement and Retention metrics as other phases of the customer journey don't align with the goal of the product. 

 

Engagement Related metrics :

1) Avg # of posts/content liked per user daily

2) Avg # of likes per user per content type (image, video, posts etc)

3) % inc in time spent on facebook because of this feature. 

4) % inc in number of relatable posts discovered because of this feature

 

 

Retention Related metrics:

1) DAU, WAU, MAU of this feature. 

2) % of users who have liked over 30 images/videos/posts in the past 10 days

3)% of users who are using this feature even after 6 months of using it for the first time.

4) % of users who use like as a source of primary communication with new connections (they formed) on a daily basis

 

Step 6: Evaluate and prioritise the metrics based on RICE model 

 

Engagement Related Metrics 

#	Relevance to company's mission	Impact to end user	Confidence  in the metrics collected	Implementation  effort
1	H	H	H	L/M
2	H	H	H	M
3	M	M	L	L/M
4	H	H	H	L

 

Retention Related Metrics

#	Relevance to company's mission	Impact to end user	Confidence  in the metrics collected	Implementation  effort
1	M/H	H	H	M
2	H	M	M	M
3	M	H	H	L
4	H	H	H	L

 

 

Measuring the success Facebook likes

Feature description – Facebook likes is a feature that allows users to express their emotions about a particular post on Facebook in a non-verbal way. If someone likes or loves a post then they can just click on the like button and chose their emotion. This also helps Facebook decide the popularity and quality of a post and thus make it visible to other viewers it thinks would like the post.

Clarifying question – Is Facebook going to take any action based on these results? Assuming that it does measure different metrics and based on their outcome takes few steps to improve the feature. Firstly, I will note down different metrics for this feature and what would qualify as a success measure. Then I will suggest possible improvements if the metrics are not up to mark.

To measure the success, I would consider the following parameters

Usage of feature

1.         Number of active users clicking on Like
2.         Per centage users on average clicking Like
3.         Number of users using this feature after reading a post
4.         Number of clicks per day on average/ per user etc
5.         Frequency of clicks per user

An increase in the all the above would mean that Facebook likes is a much-used feature by users.

How the feature is helping Facebook (retention/ monetisation)

1.         Per cent of authentic posts with the greatest number of likes
2.         Number of posts promoted by likes converting new viewers

An increase these metrics means that the feature is bringing more authentic posts and hence users to the platform.

If there is a dip in any of these metrics, then Facbook would have to do an intrinsic and extrinsic evaluation to understand if users are migrating to other platforms and if so why. Or if there is a sudden surge in fake news circulating Facebook due to illegitimate likes on such posts.

 

Step 1: Ask Clarifying Questions

1) Are we considering the entire product or only a part of the product for this question to measure the success of Facebook Likes, or is it something that you'd like me to choose and assume

2) Do i have to keep any company's current objective in mind while defining and measuring the success criteria of Facebook likes or is it something that i can choose and assume

For narrowing down the scope of the question i will be going with the usage of facebook likes on the normal facebook newsfeed only.

 

Step 2: Talk about the Product/Feature

The mission of facebook is to connect people across the world. Facebook like is an easy, friction-less very less time consuming way of communication with others.

Facebook like helps a user to like and enagage with different piece of content (videos, articles, posts etc)

It gives a sense of validation and good feeling to the creator of the content, and they feel more motivated and inspired to create more content.

Facebook like has been out for a very very long time and has been adopted widely across the entire facebook ecosystem. I think there are hardly people who don't know or have not used the facebook like feature.

The broad target users for facebook like feature are generally

A) Content creators- Who create different types of content and share it with their network. They can like their content as well, and can like whichever content on their timeline which they find good.

They can be further divided into social media influencers and brands who keep a track of the like numbers to do targetted marketing on the audience based on the content type they like and to also track what kind of content works and doesnt work

B) Content readers- These are people who just consume content and engage with the comment by using the like, comment and share feature

They can also be divided into social media influencers, brands as well who like posts of their competitiors and any other kind of post that they like and find relatable.

Facebook like helps users see who all have liked that particular post or content, gives them an idea about the kind of content their audience wants depending on the number of likes their post got. It also helps facebook create an interest graph for the users by showing them similar related content. People can see everyone in their network who has liked a post and intrigues these users to check that particular post

 

Step 3: Identify and define the goal of the product/feature

Facebook like helps people engage with content and thus since users can see people in and outside their network who have engaged with their content, for this particular question i think we would go with increasing in user engagement as the main goal for facebook like feature.

 

Step 4: List down the user actions that are directed towards increasing user engagement

1) User logs in to facebook

2) User goes to the newsfeed and sees a lot of content

3) User finds some piece of content interesting and wants to give feedback to the content creator

4) However user doesn't have that much time to comment or share, so user just presses the like button

5) This indicates to the user who created that content piece that users are liking and engaging with the content and gives a sense of satisfaction

6) Content Creator and others can also see the number of people who liked the post which shows that they have similar thought process and gives them an option of sending them a message or a connection request

 

Step 5: List down the metrics that are directed towards the end goal of driving user engagement

1) Average number of likes per user per month- This is also a very broad level metric

2) Average number of likes per 1000 posts on a daily, weekly and monthly basis- Broad level metric that gives us a good idea about user engagement

3) Number of users who have liked atleast 10 posts in the last 1 month-  This metric sset-up a basic acceptance criteria and covers the adoption, engagement and retention aspect of facebook like. This shows how a user has become a recurring user of the feature and has engaged multiple times in a matter of a month

4) Average number of likes per post type on a daily, weekly and monthly basis- This will give facebook and content creators and idea of the kind of post that is working and the ones that aren't depending on the number of likes a particular type of post gets.

Also shows how the users are engaging with a particular content type

5) % of liked posts that - Since not all posts or piece of content is liked by the audience therefor this broad metric will give us an idea of how many people are adopting and using (engaging using) this feature.

 

Step 6: Evaluate the metrics and prioritize them based on certain criterias like  Relevance to the end goal and misison of product, Impact to the end user, Effort required to collect and evaluate these metrics.

Based on step 5 the metrics that we would choose to measure the success of Facebook like is

Average number of likes per user per month shows on an a monthly average basis how do users engage with content on facebook. This is again a broad metric hence we have considered this

and Average number of likes per post type on a daily, weekly and monthly basis as it gives a fair idea to facebook and the content creators about the type of content people are  engaging most with it, and facebook can advertise based on that as well which can further lead to increase in revenue for facebook

 

About the Feature:

1. Increase news feed quality
2. Help content creators post better content by providing feedback via likes
3. Increase revenue by targetting advertisements better

1. The user, while scrolling on his/her news feed, sees a post that he/she finds interesting, let's say a post by a stand-up comic
2. To express his/her interest, the user hits the like button
3. Facebook registers the fact that the user likes stand-up comedy
4. More posts related to stand-up comedy or comedy, in general, are shown in the news feed

1. Average Increase in # of likes per hour of scrolling (Limitation: Do not know if the post is shown due to previous like data or due to search/comment or interactions)
2. The average increase in interaction with posts recommended due to previous like data (Effort required to track like data-based recommendations separately, however, I would like to check the feasibility of this to the interviewer and modify the idea accordingly)

QUESTION 2: How would you measure the success of facebook likes?

 

Clarifying questions-

• I am assuming we are only considering likes not the reactions, right ?

 

Feature description: Facebook like is a feature which enables users to like post, video, profile picture and pages. This is validation for the  user who posts the content. For a marketer higher the likes the better the reach.

 

Goal of the feature: 

• Business goals: To drive engagement and also the revenue since higher the likes higher the reach and higher the revenue  from advertisers or the influencers 

 

Customer journey: Here customer journey starts from being aware of this feature and the next step is to use that is the engagement. And the user who posts pays for likes hence the revenue .

Metrics 

 

1. Awareness: 

• How many users know about this feature? 

# No of users who have liked the content once after the launch of this feature.

# No of users who have liked all including profile picture, video, post and page.

2. Engagement:

# Average likes per user in a day

#Ratio of profile pic likes : video like: post like : page like for a user ( This metric will tell us the purpose for which are using this feature the most-is it pics, video, post  page)

#No. of likes for different segments like entertainment, advertisements, news etc. ( This will help facebook or even advertisers to know which content to promote more often. Also this well tell which is that segment which is using it most )

 

3. Revenue

            #No of likes per post,picture, video page ( This will decide the revenue or the cost incurred to the user who post an ad or content)

 

Prioritization:

My goal of this feature is to  drive engagement and hence earn revenue so i would pick average likes per user in a day and no of likes per post for the ones which are for commercial purpose.

Assumption: only like (not reactions)

 

Facebook like is the feature which allows the users to express the feeling whether they like the content published on the facebook

 

Business goal: Facebook's mission is to bring people together and build the strong community and we would like to keep the users engaged on the facebook platform by showing them the content they care about which facebook is trying to get through the “like” feature. 

 

User goal: so the user goal is “ user should feel connected with the content they are viewing on the facebook” and also “ user should feel the assurance of their content which they view on the content based on if their content is liked ot not”

 

So Like feature is to understand whether content is engaging or not and I would measure the engagement metric here. Since “Like” is already a mature product then retention is the key which help me understand if my users are coming back and using this product

 

Who and where “Like” is being used

Creators/publishers - 

Using the like button to get the assurance if the content is likeable or not and how they

Viewers : 

: show the feeling through like button if they like it 

 

User actions and Metric

 

Different places users use like button

1. In the news feed

2. In the group, event, marketplace 

Metric which can help measure the success  of the feature 

1. DAU ( no of users using at least one like on the content)

2. Total number of likes

   1. No of likes per feed 

   2. Likes per content as well - type of content 

   3. Like in photos per group discussion 

   4. Likes by geography

   5. Like by device 

3. Retention 

   1. MAU - total number of users coming back and using like

   2. Engaged users = DAU/MAU - what % of monthly active users is coming daily

 

NSM 

• No of likes to understand if people are using the like feature to express the feelings

 Supporting metric

 

• No of comments per user

• No of reactions per user

To reach a measure the success for Facebook Likes, I will:

• Clarify the mission or goal of the feature (e.g. what success means) and its relation to the overall company mission. 
• Discuss a range of metrics to measure aspects of success, being mindful to explore the rationales and trade-offs for those metrics. 
• Briefly prioritise 2-3 key metrics and then 
  • Debug at least one in the time available  
  • Evaluate how the metrics and goals can be applied to decisions on improving the Like feature.  

Context - What is a Like, Exactly?

We can view a Facebook Like as a standardised form of action and communication between a content consumer and a content producer and also as a signal to other consumers. At a micro-level, to Like is to create a visual expression of a positive feeling or sentiment - from the person reading/watching content to the person or org who posted it. A Like is not strictly a form of voting, but given the value of a post gaining a high number of Likes being high engagement and virality it can be used as a way to count popularity or support in a sense. At the macro-level, certain user segments may Like some posts over others, potentially confirming explicitly stated political preferences for example. Subsequently it could then be possible to infer a person's political views based on the simple act of Liking a post.   

However, a single positive expression is not enough to convery the subtle complexities of positive sentiment. For example, people may "Like" a post as a way of showing support but if that post conveys bad news then a Like can be misinterpreted or appear insensitive. Given the rise of emoticons and gifs, a single positive expression is not enough. The need for more nuanced reactions that are as efficient as the original 'Like' button already lead to six new reactions now available (since 2016) 'under' the Like button on FB. 

What Success Means for Likes

All this to say that a Like is a valuable expression and action for Facebook as it helps to show how certain content can bring communities together, or divide them. For content producers, having analytics on Likes can provide insights as a form of feedback to ensure the quality and impact of future content. 

If we consider all six current reactions as a form of Like, their success can be viewed in terms of:

1. Usage shows that people are comfortable using a Like to express their views.
2. Effective Communication: in terms of semiotics, the succesful use of a Like (the signifier) is the ability to signify meaning effectively - conveying to the recipient precisely what the sender intended. We can break this further into:
   1. Universality
   2. Expressivity

The mission of Facebook is to enable people to build communities and to bring people together. Highly effective and efficient communication should mean there is less misunderstanding at the same time there is more engagement at a global scale. We can say then that the goals of usage and effective communication link directly to the overall company mission. 

 

Measuring success

Taken in the context of the goals and assuming the current six reactions available to users (Like, Love, Haha, Wow, Sad Angry), there are several metrics we could look at using:

 

1. Usage:
   Taken as a pure summary statistic, these are good top-line indicators, but they obscure the detail and can therefore be misleading to FB and to content producers. It would be ideal to have the flexibility to cut this data by the attributes of the post that being liked and the attributes of the audience who 'Like' it.
   1. Number of Likes: How many people use a Like on a post? How does that number change over time? 
   2. Frequency of Likes: How frequently do users Like posts in a session? Does this change over the course of a session or over the length of their time on the platform e.g. are long-term users more or less likely to engage. The flip side of that quesiton; are new users more likely to react to everything?
   3. Variety of reactions per post: How much variety is there in a single post? Are people using the full range of  Taken along with the number of Likes, it should be possible to get a clearer picture of the usage
   4. Errors: Do people make the wrong choice/or undo their Likes? Are there technical or UX issues that can be identified, perhaps by device type/OS, that are causing users to undo their choice of Like? How long after a Like is selected does the user undo the action?
       
2. Effective Communication:
   1. Universality: 
      1. Frequency of Likes by key audience segments:  Do all people use each type of Like reaction to mean the same thing? This would require identifying comparable posts across languages and cutlural groups, making accurate comparisons difficult. A data scientist should be able to cluster Like-types by key cultural and demographic factors given the size of FBs data. 
      2. Frequency of emojis by key audience segments: Emojis will reveal universality distinct from Like reactions that will provide an important context. Which emojis do people use, across FB and other platforms (Instagram, Whatsapp - if this is possible to access for a FB PM- I'm not sure??). Same quesations apply as above - are there differences in how audience segments express theirselves using emojis? Demographics will play a signifcant factor (i.e. generational differences, socio-economic background). 
          
   2. Expressivity:
      1. Text sentiment: If people are unable to express their views effectively, what do they write instead? Do they react with a Like then clarify further with a comment how they feel? 
      2. Number of Emoji in replies: Which emojis are used in replies that are not represented by a Like? Combined with Unversality, how does this vary by audience segments?
      3. Variety of Emoji in replies: This would help to understand if a high volume of emoticons is often a repitition or not.. e.g.
         💃 💃 💃 💃 , low variety but high numbers could suggest that repetition serves as a kind of adverb or modifier to express 'very'.. or 'much wow' 🤣. 

 

Prioritisation of Metrics

From Usage, I would prioritise the Number of Likes and Error metrics to give an overview but also provide some rate of error to improve the context. 

 

For Effective Communication I would proritise Like Frequency (by audience) for a top-line metric in Universality and Number of Emoji used in replies to give a high-level picture of how people are expressing themelves through emojis.

 

Debugging Metrics & Making Improvements

Assume the Number of Emojis used in replies has increased over several weeks compared a drop in the Number of Likes over the same period. This could be taken as an indication that the Like reactions are not offering users a universal form of expression. However, I would dig into which emojis are being used and over what time period. Is it seasonal (e.g. a festive holiday) or a social trend? By looking at audience segments and geo we might find that it's a passing social trend related to a single country or city for example. 

 

If the emojis being used were very similar to the available Like reactions, I would also work with UX designers to further analyse visual elements. For example, perhaps there has been a recent update to Unicode emojis - overall size, colour, clarity of facial / physical expression of the emoji vs Like reaction. I would then propose to A/B test design variations in the Like reactions to see if there is a significant reduction in the emoji that had previously been increasing. 

 

 

Facebook Like Success Metrics

 

So we are on the same page, I would like to describe this feature as follow 

 

Description 

Assume facebook has two user groups 

1. User group which publishes the content in the form of status updates, news etc 

2. User group which consumes the content 

 

Facebook Like feature let the consumer user group to like the content of the publishers. 

Publisher user group of the app can view the number of users who have liked their content and who they are

 

Customer Journey 

1. User is going through the feed items ( status, post, videos ) 

2. Users watch or read the feed items 

3. If User like the content then he or she likes the content 

4. Publisher of the content gets notified 

5. Publisher of the content can view who has liked the content 

6. Publish of the content can view how many users have liked the content. 

 

The goal of the Facebook feature 

 

To understand the goal of the Facebook Like feature, we need to understand what it is helping or solving for the users and the business 

 

Goals of Facebook LIKE feature from Business perspective 

1. Like would add a signal to the quality of the content published by the publisher so that best quality content can show up on the top or bad quality content can be degraded

2. Gauge the interest of the consumer so that similar content can be shown up to increase the engagement 

 

Goals of Facebook Like feature from the publisher perspective 

1. Encourage the publisher to publish more content by knowing him how many people have liked his content 

 

The goal of the Facebook Like feature from the consumer perspective 

1. Reduce the friction or time it takes to appreciate the content 

 

Important Business Metrics 

1. % increase in user engagement 

   1. Like should improve the quality of the content and hence increase the user engagement 

2. % increase in the user content ( per day/week) 

   1. Like should increase the user content as publishers tend to roll out more videos, status, posts 

3. % increase in the sponsored ads clicks 

   1. Like, should improve the digital ads targeting. 

 

Important Product Metrics 

1. % increase in the post activities 

   1. This should increase as Like make it very simpler for the consumer to appreciate the content 

2. The average number of posts per user 

   1. It should increase if Like feature has an effect on the content publisher

 

 

 

I would have some clarifying questions e.g. Is it the like button within the facebook application ? If no specific preference i would go with the assumption it is within the scope of FB application

Describe like - It is a fundamental feature of facebook. Kind of makes the application more expressive. sort of expression that the user would have for the post / entity that they would see on their news feed. It kind of ties back to FB overall mission as well to 'bring people together and build communties' . Like feature makes people more expressive so ties back to the org overall mission.

Measure - To measure the effectiveness of the like feature could segregate this into Explicit and Implicit measurement

Explicit  

likes per user or likes per session

No. of comments / post 

Implict

You can see who else liked the post ; it could lead to connecting more people together . New friends added

new group converstions started or new groups created

New followers

No. of posts / user (user who posted would be motivated and happy)

The primary objective of the feature is to feel users more connected and expressive and hence more engaged. So i would take engament as the primary metrics and track on Explicit measures.

 

 

 

 

 

What is the feature?

A facebook like is a way that the FB  user shows his support or endorsement for a post/pic/video/page etc. 

What is the Goal of the feature?

Goal of facebook is to help users connect with family and friends. Goal of facebook Like is to help users show support/endorsement of the posts in their news feeds.

What is the User Journey?

Users go though the news feed and click or touch to Like on a post of wait for other reactions like Happy, Sad, Funny, Love, Amazed etc. Depending upon Facebook news feed algo logic, this is shown to other people in their facebook network.

What Metrics will measure facebook likes?

1. No, of Likes- absolute and per user

2. No. of Likes per post- differemt for photos, videos, links etc.

3. No. of different reactions- per user

4. No. of likes/reactions vs. time spent on facebook

5. No. of likes vs ARPU

6. No. of likes per user on friends' posts, celebrity posts, sponsored posts

7. No. of users who like and comment on the same post- split of like then comment and comment then like

8. No. of users who like and share the same post- split of like then share and share then like

9. No. of likes as a function of the position in news feed

10. No. of likes as a function of time since login

11. No. of likes per user as a function of the country of the user

12. No. of likes as a function of platform( mobile vs web)

13. No. of follows after likes

14. No. of likes as a function of the video % watch time

 

Facebook likes is a plugin that allows users to manifest interest in content. It also creates engagement between liker and like. For creators is a way to receive feedback. And allow FB to understand the preferences of users that can be used to target marketing and to improve facebook product.

 

Goals:

For content creators is understand the preferences of their followers.

For Facebook is understand what content the users prefer in the platform, and where to put marketing efforts to increase revenue in CPM cost per impressions.

 

Journey:

a. User opens the app, scrolls the feed, see content that it is appealing and hits the like button

b. User opens messenger.. see a message/share from a friend and gives a like.

c. User opens Facebook and search for a page and subscribe or like the page.

 

Metrics:

a. What is the average time that a user spends in the app before liking some content? How many like it gives per every time it login in? How many posts has to scroll before liking something?  Does s\he interact in other ways with the content that received the like (e.g. comments, share)?

b.What percentage of times does the user gives a like when receiving content through messenger? What kind of content that it’s obtained through messenger receives more likes? Does s\he take additional action with this content (e.g., replies to the sender, click links, etc.)?

c.How down has to scroll in the search results before subscribe\like a page? How often s\he interacts with the content od the liked pages when it appears in the feed? How often they search for the content of these pages? How often they click monetizable content?

 

Priority metrics and counter metrics

For the case a:

1.What is the average time that a user spends in the app before liking some content? Acquisition metric.

2. How many like it gives per every time it login? Retention metric, do they like what they are receiving in the feed?

3. How many posts has to scroll before liking something?  Another retention metric

4. Does s\he interact in other ways with the content that received the like (e.g. comments, share)? This is the monetization metric; we want to increase impressions.

For b:

Does s\he take additional action with the content (e.g., replies to the sender, click links, etc.) that receives through messenger? This is an activation metric, they are receiving content that they like, but it’s necessary that they take action outside of messenger.

For c:

1. How often s\he interacts with the content of the liked pages when it appears in the feed? This is our retention metric.

2. How often Do they search for content of these pages?  This can be a counter metric because even if they are searching for content means engagement, it also means that Facebook is not displaying this content naturally in their feed.

 

 

 

Context: What is the role of likes feature in context of overall facebook as a product?

Facebook's number one metric is growth of its network. The more users active on a social network online, the more valuable it is. Facebook introduced the ability for users to show interest and appreciation by liking anything shared on the network. It is human nature to look for appreciation and validation of one's actions - in this case sharing with a group of friends or broader audience. Facebook like provided a way for people to react to content and provide the authors - how much the shared content was liked by the audience. The more likes - the more validated the author felt and the more they were enouraged to share content. This led to more content being shared. At the same time, the user's likes were a signal about their interest on types of items - and this could be used to make sure the news feed is as relevant and personal to the users as possible. In general, both from people who shared content and people who liked content - the social network newsfeed became more interesting due to "good" quality content being shared and the right content being shown to users based on their preferences. The more interesting the news feed, the more likely users were going to stay with facebook. Therefore, this leads to the broader answer: the goal of like was to make facebook users find their newsfeed super interesting and continue to increase usage of facebook from which we can derive success metrics:

1. User engagement - did the introduction of likes lead to a more interesting newsfeed across the board?

If the newsfeed was indeed becoming better over time due to better content and better personalization due to likes, we should more engagement with newsfeed 

- %engaged users = % users engaging with at least 1 newsfeed item every session - this is the most important metric to track as this is the primary aim of the feature.

A derivative metric is: %re-shares = reshared items / shared items daily, weekly, monthly - we know that people usually share with their immediate network. This strengthens the connection but new connections grow faster if the content is re-shared.

2. User adoption - did increased engagement lead to deeper adoption of Facebook as product 

If likes encouraged someone to share more often, we should see more content being shared per user on the network, thus increasing its value across its entire user base. # content items shared / user - would therefore be a key indicator of success and contribution.

If likes inspired more users to share, then we should see growth in users sharing content relative to overall population - content sharing users / Facebook population daily/weekly/monthly. In any community, there is a certain share of people who are active content creators / discussion initiators and those who are passive followers. The more content authors relative to the population the more healthy the network.

3. User retention - did increased adoption contribute to long term health of the network with user retention (lagging metric)

The newsfeed is the primary user experience for facebook. A poor newsfeed experience will cause churn. Introduction of Facebook likes thus is going to directly impact user retention or churn - % users becoming inactive on a weekly or monthly basis, sessions/uu - daily/weely, returning uu / weekly or monthly. If we were introducing Facebook likes and running an A/B experiment, we would look to see impact on user retention. 

4. Network Growth - did the social network as a whole keep growing due to increased engagement, connections

The more users find compelling content, the more they want to share and the more people discover the source of the content and want to be associated with the content. Thus increase in number of connections / user is another indicator of impact.  

 5. Revenue as Guard rail metric- Facebook needs to make sure of its overall health as a business - thus revenue is an equally important component. Introduction of likes fundamentally changes the product experience and impacts user engagement, thus the revenue earned. Therefore, we should track revenue / per active uu or a proxy. We want to make sure that revenue potential of sessions are not decreased due to change in newsfeed.

6. Impact to brand - Facebook brand could be adversely affected due to introduction of likes and consequent impact on newsfeed quality. Therefore, this could be monitored directly with inline surveys (NSAT) or other clear disengagement signals such as opt-outs.

 

 

scope

Before I start Id like to clarify the scope of this problem

1) What is like feature : Like lets you quickly 'Like' a photo, comment, video, reply or page

2) When you say success do you mean post launch evaluation or long term or both?

3) I'm assuming this is applicable to both New and existing users - I'd cover the metrics separately as some metrics may be unique to each cohort

What is it doing?

What is Facebook? Facebook is a social network that lets you form virtual connections with people you may already know and want to connect the world. 

In many ways Facebook works like the real world - it lets you form connections with people you know/meet and let you stay in touch easily. Sharing is real world equivalent of talking points/things you share with your friends and likes from a customer perspective lets you express +ve sentiment easily.

From a business perspective it lets you express yourself to friends quickly and frictionlessly since typing is tedious and makes you think a lot more in comparison.

who are our customers?

Facebook users

- Content creator 

- Content consumers 

Needs

As content creators I'd like feedback on my posts  so I know how my friends like my content/I can do more of it

As content consumers I'd like to give feedback so I can tell my friends i like their content

Likes solves for both the customers

From a business perspective this should translate into higher engagement so here's the metric I would be looking at :

New user

Key metrics

Business

1) Impact on retention : AB test to determine if there's any impact on D7 retention/7 day engagement for users who can like posts vs those who can't

since creators might come to check likes on their posts, consumers may use more often because they enjoy the experience more

2) Impact on key actions : Are the key actions easier to reach due to introduction of likes? 

With improved engagement Key action definition may have changed. ( For reference Key action is an action that indicates user has discovered value and is now very likely to be retained)

3) Adoption rate: How many people use like in W0

This would give us 2 feedbacks

 - is the experience easily discoverable and intuitive

- do people like using it 

4) Second order effects 

Are more people posting?

Are people posting more often?

Are these posts leading to conversations?

Customer facing

User feedback : What do user have to say about the feature on Play/App store , social media channel and Internal customer feedback

Existing users

Business

1) Impact on 7 day/30 day engagement  : Are people using app more often

since creators might come to check likes on their posts, consumers may use more often because they enjoy the experience more

2) Impact on timespent : Are people spending more time

since creators might come to check likes on their posts, consumers may use more often because they enjoy the experience more

3) Adoption rate : what % of WAU are using it 

Tells us about discoverable/intuitiveness of UX

4) Second order effects:

Are more people posting?

Are people posting more often?

Are these posts leading to conversations?

Customer facing

Customer facing : What are users saying about it