How would you measure the success of Facebook Likes?
+1 vote
3.8k views
in Metrics by | 3.8k views

8 Answers

+7 votes

Describe

  • Like button offers low friction way to signal interest
  • Inside FB, provides validation back to content creator
  • Across web allows user to bookmark content they are interested in
  • Allows FB to build expanding interest graph for each user
  • Creates virtuous engagement cycle, both liker and like-ee

Feature goals

  • Engagement
    • More consumption and production of content
  • Richer user insights
    • Surface more personalized content
    • Offer advertisers better targeting

Journey

  • I'm scrolling through newsfeed
  • I see friend's video
  • No time to comment
  • Click Like
  • I feel good about sending positive signal
  • My friend gets endorphin hit
  • More like-able content shows up in my feed
  • My friend is motivated to post more content

Success metrics

  • Is the feature discoverable?
    • Focus groups
    • Likes per session
  • Are users using the feature as intended?
    • % Un-likes
    • % of Likes followed by a comment
  • Is usage of the feature growing?
    • average likes per user
    • Likes per session
  • What is driving usage of the feature?
    • Likes by content type (video, photos, articles, webpages, groups, etc)
    • Likes by user segment (age, geography, time on platform etc)
  • Does the feature increase engagement?
    • Average session length by Like volume by user
    • Post frequency by like volume
    • Content interactions by like volume
    • Session frequency by like volume
  • Does the feature increase user value to advertiser?
    • Ad clicks by like volume by user
    • Length of ad video viewed by like volume
by (27 points)
+6 votes
Like is the fundamental feature of facebook. I would start this question by first confirming that the interviewer only wants to measure success of Like feature used in Facebook application(mobile, web etc.) and not the ability to include Facebook Like button in social components in websites. That is one of the big ways in which Facebook Likes is used in collecting data throughout the web.

Let us assume that this question is only for the Facebook Like feature in the facebook applications.

What is Facebook Like? - It is the most important way of user expression in facebook to denote the user's engagement with a post/entity on Facebook

Success of facebook Likes is based on the user adoption of using this feature and continuing to express in the platform. Therefore, engagement metrics are the cornerstone for this features success. Some of them are as follows -

Direct Feature Engagement Metrics due to Like

a. Number of Likes/User or Brand/Day

b. Number of comments, shares per post

Indirect User Engagement Metrics to track

c. Number of 1:1 communications as a result of a Like

d. Number of new connections or recommendations generated as a result of a Like

e. Number of  followers as a result of a Like

f. Number of group conversations and Increase in engagement due to Like

Platform Metrics - Likes increases Posts , Articles, User and Brand popularity which in turn drives recommendations and better engagement for customers. Likes also helps Facebook with data point about user interest and therefore better ads to be served. Some example metrics are as follows -

g. Time Spent by the user in the application

h. Number of Ads/user

i.  Ads Click through rate

Metrics Prioritization - The metrics prioritzation is based on metrics which drive direct engagement, indirect engagement and finally business and platform metric improvement. Therefore the order would be - a, b,c,d,i,ge,f,h
by (44 points)
+1
I like the structure of the answer and your clarification questions. You could also ask to clarify if the Like is in chats (e.g. liking a Facebook Messenger Message) or shared content (e.g. liking a friend's shared article). Enjoyed reading the answer.
+2 votes

Framework i'll use to answer this:

  1. Define feature
  2. Define goal of feature
  3. User journey
  4. Metrics

Define feature: Ability to 'thumbs up’ a post from another user - including original text/images, as well as responses. AND to thumbs up ads/ company pages on Facebook

Goal of feature: Reduce the barrier to user interaction on FB by supporting ‘one-click’ interaction. Thereby drive more interactions and overall more engagement and retention/stickiness + ideally more ad revenue longer-term

User journey:

Adding a like:

  1. User on their timeline
    1. Sees a post - thumbs up
    2. Sees an ad - thumbs up
  2. User on a 3rd party facebook site: Sees company they like - thumbs up
Receiving a like:
  1. User on their timeline: sees a response
  2. User not on FB: gets notified of a like

Metrics:

  1. Adoption
  2. Engagement
  3. Monetisation
  4. Retention

Since primary goal of feature is to increase engagement, will start here.

2. Engagement:

We want to understand uplift in engagement coming from this feature. Engagement levers:

  • # users
  • # sessions
  • length of a session [less important, will ignore]
  • # ‘Interactions'
    • Primary: users who give/receive likes
    • Secondary: increase in other interactions following these likes

Uplift metrics:

  • # users: % users who have only interacted through likes and not otherwise
  • # sessions:
    • % users /sessions which have / only have likes
    • % users /sessions triggered by viewing what someone has liked
  • interactions - primary:
    • % interacted/all posts which have only received a like
    • Likes as % of all interactions with a given post (track this over time: if decreasing, then examine the other features that are replacing it)
  • interactions - secondary:
    • likes of individuals: % interactions generated following a like => ‘network/re-engagement effect’ of likes
    • likes of ad/company: uplift in traffic from user’s network following a like => upsell potential

Other metrics to consider:

1. Adoption:

Two dimensions to adoption - measuring both (absolute and as % total) is a baseline:

  • Like: Users who have ‘given’ a like
  • Liked: Users/posts which have been liked

3. Monetisation:

Relevant for likes on ads/companies.

  • Levers: ad views, click-through, revenue
  • Metrics:
    • Uplift in all levers for ads which have received a ‘like’ vs not
    • Uplift in ad views & revenue from users who receive likes vs not?

4. Retention: Ideally feature keeps more users on FB over time, which will eventually translate into more monetisation. Levers are similar to engagement - frequency of sessions and interactions. Want to look at cohorts of users who receive/give likes and see how this affects their session and interaction frequency.

by (18 points)
0 votes

a. Customer goal: Human emotion has a wide range and variety. The Like button limits the range of emotions that someone can express while engaging with content on their news feed - provide customers with more options in addition to "Like" to express themselves.
b. Customer goal: Increasing social validation - By making it easier to engage with the news feed through reactions, you increase the likelihood of activity across friends and increase the feeling of social validation.
c. Business goal: Increase engagement of users on the news feed- By engagement I mean - average number of shares, posts, likes, reactions, comments per visitor during a 30 day period
d. Business goal: Increase retention: Increase the rate of repeat visits (daily, weekly, monthly) per user

by
0
I think your answer lacks the structure that's needed to answer a Metrics question. Have a look at the article on how to answer a product manager Metrics question here.
https://productmanagementexercises.com/how-to-answer-a-metrics-question
0 votes

I am preparing for an interview and have been reading many sample answers, but this is my first attempt at answering a metrics question so I would really appreciate your feedback.


First, define the "like" feature:

The like feature is a way for users to interact with content on FB. By content, I mean videos, pics, statuses, shared posts, events, pages, comments, etc. It's implementation is simple and straightforward: you see content and you can click the "like" button. You can also see others who have liked the post (even if they are not your FB friends).


Now, outline the goal(s) of the feature:

  1. As with most FB products/features, one main goal is to increase engagement between users by sharing their "feelings" and "thoughts" in a simple and easy way.
  2. The other prominent goal is user retention - perhaps this feature entices users to keep coming back to interact.

Some Metrics:

  1. Average number of content likes per user per week
    • We can even go further to track this value for particular content such as videos, statuses, comments etc. to see which type of content is achieving most likes
  2. Since it would be nice to know how the "likes" feature effects other types of engagement, we should measure a percentage defined by the number of content that a user has liked AND interacted in some other way (comment, share) divided by the total number of content the user has liked (regardless of other interactions or not).
  3. To measure retention, we should calculate the percentage of users who have liked at least one item per week for the past month(s).

I think that is about all I have so far. A question I have is, is it necessary to segment between power users and "regular" users to crunch these numbers? Is that always a key practice when analyzing metrics?

Thanks!

by (20 points)
0
Your structure is great. I have some feedback:
- The user retention is a great goal. Perhaps I would have described it in more detail and said users come back to see how many likes they have received or who have liked their content.
- I'm wondering how the first metric reflects success of failure of the feature (given that goal is engagement with facebook). I would have listed a more generic metric such as "time spent on site among those who like vs those who dont"
- I didn't follow exactly how metrics 2 and 3 help you measure if you're achieving your goal. Can you elaborate more?
- I would have included a couple more metrics that would help you measure your success in achieving your second goal. For example, does more likes result in more frequent posting or more returns?
- If you end up listing more metrics as I described in my above point (I would personally list at least 5), then I would do some evaluation of metrics based on some criteria (e.g. easiness to collect data) to show my prioritization skills.
You can pick a particular segment if you like but it wouldn't be necessary. If the engagement metrics are up, then the feature is a success.
Good luck!
0 votes

Great answers. I'd add a few additional points:

  • Another business goal here is to get a signal from users as to the type of content they like (FB could analyze whether it's something inherent in the content, or the identity of the poster), so FB can surface more like-able and personalized content which should result in higher engagement by users
  • The question is not forthcoming on *when* this measurement is taking place. If we were to think of a world before the like button, the best strategy may be to roll it out in a mock AB test. This is complicated since it would be weird for two friends who interact a lot to be in two buckets, but not impossible (e.g. by region). 
  • Another advanced trick might be to analyze the text of comments using NLP. For example, people might write things like "I didn't feel like liking this post was appropriate so I'm writing this comment instead" or "OMG, yes!" combining insights from the text and crossing them with whether or not the content was liked can provide big insights into how users use the Like button and whether or not it is a successful feature
by
0
Your answer is missing the right structure. Please have a look at the interview guide on how to answer a product manager metrics question.
https://productmanagementexercises.com/how-to-answer-a-metrics-question
0 votes

scope

Before I start Id like to clarify the scope of this problem

1) What is like feature : Like lets you quickly 'Like' a photo, comment, video, reply or page

2) When you say success do you mean post launch evaluation or long term or both?

3) I'm assuming this is applicable to both New and existing users - I'd cover the metrics separately as some metrics may be unique to each cohort

What is it doing?

What is Facebook? Facebook is a social network that lets you form virtual connections with people you may already know and want to connect the world. 

In many ways Facebook works like the real world - it lets you form connections with people you know/meet and let you stay in touch easily. Sharing is real world equivalent of talking points/things you share with your friends and likes from a customer perspective lets you express +ve sentiment easily.

From a business perspective it lets you express yourself to friends quickly and frictionlessly since typing is tedious and makes you think a lot more in comparison.

who are our customers?

Facebook users

- Content creator 

- Content consumers 

Needs

As content creators I'd like feedback on my posts  so I know how my friends like my content/I can do more of it

As content consumers I'd like to give feedback so I can tell my friends i like their content

Likes solves for both the customers

From a business perspective this should translate into higher engagement so here's the metric I would be looking at :

New user

Key metrics

Business

1) Impact on retention : AB test to determine if there's any impact on D7 retention/7 day engagement for users who can like posts vs those who can't

since creators might come to check likes on their posts, consumers may use more often because they enjoy the experience more

2) Impact on key actions : Are the key actions easier to reach due to introduction of likes? 

With improved engagement Key action definition may have changed. ( For reference Key action is an action that indicates user has discovered value and is now very likely to be retained)

3) Adoption rate: How many people use like in W0

This would give us 2 feedbacks

 - is the experience easily discoverable and intuitive

- do people like using it 

4) Second order effects 

Are more people posting?

Are people posting more often?

Are these posts leading to conversations?

Customer facing

User feedback : What do user have to say about the feature on Play/App store , social media channel and Internal customer feedback

Existing users

Business

1) Impact on 7 day/30 day engagement  : Are people using app more often

since creators might come to check likes on their posts, consumers may use more often because they enjoy the experience more

2) Impact on timespent : Are people spending more time

since creators might come to check likes on their posts, consumers may use more often because they enjoy the experience more

3) Adoption rate : what % of WAU are using it 

Tells us about discoverable/intuitiveness of UX

4) Second order effects:

Are more people posting?

Are people posting more often?

Are these posts leading to conversations?

Customer facing

Customer facing : What are users saying about it 

by (143 points)
0
Hey Rahul,
Great structure! I have some feedback on how to improve the answer:
- Think "users" not "customers". Remember you're a product person who cares more about the experience and usage. It's fair to assume higher engagement results in more revenue & profit.
- Don't need to describe what Facebook does. Both you and the interviewer know what it does. If you're describing anything, describe the particular feature / product (in this case Likes) you're evaluating
- Your user groups should be as distinct and exclusive as possible. Content creators and content consumers are not exclusive since a user can belong to both groups. I'd go with frequent and occasional.
- What's the goal of the product? You should try to clarify this right away and use it as a guide as you answer the question
- I would only differentiate / categorize the metrics after listing them (by walking interviewer through different stages of a Likes experience. See answers and feedbacks above) and evaluating them based on some criteria
All the best!
0 votes

Context: What is the role of likes feature in context of overall facebook as a product?

Facebook's number one metric is growth of its network. The more users active on a social network online, the more valuable it is. Facebook introduced the ability for users to show interest and appreciation by liking anything shared on the network. It is human nature to look for appreciation and validation of one's actions - in this case sharing with a group of friends or broader audience. Facebook like provided a way for people to react to content and provide the authors - how much the shared content was liked by the audience. The more likes - the more validated the author felt and the more they were enouraged to share content. This led to more content being shared. At the same time, the user's likes were a signal about their interest on types of items - and this could be used to make sure the news feed is as relevant and personal to the users as possible. In general, both from people who shared content and people who liked content - the social network newsfeed became more interesting due to "good" quality content being shared and the right content being shown to users based on their preferences. The more interesting the news feed, the more likely users were going to stay with facebook. Therefore, this leads to the broader answer: the goal of like was to make facebook users find their newsfeed super interesting and continue to increase usage of facebook from which we can derive success metrics:

1. User engagement - did the introduction of likes lead to a more interesting newsfeed across the board?

If the newsfeed was indeed becoming better over time due to better content and better personalization due to likes, we should more engagement with newsfeed 

- %engaged users = % users engaging with at least 1 newsfeed item every session - this is the most important metric to track as this is the primary aim of the feature.

A derivative metric is: %re-shares = reshared items / shared items daily, weekly, monthly - we know that people usually share with their immediate network. This strengthens the connection but new connections grow faster if the content is re-shared.

2. User adoption - did increased engagement lead to deeper adoption of Facebook as product 

If likes encouraged someone to share more often, we should see more content being shared per user on the network, thus increasing its value across its entire user base. # content items shared / user - would therefore be a key indicator of success and contribution.

If likes inspired more users to share, then we should see growth in users sharing content relative to overall population - content sharing users / Facebook population daily/weekly/monthly. In any community, there is a certain share of people who are active content creators / discussion initiators and those who are passive followers. The more content authors relative to the population the more healthy the network.

3. User retention - did increased adoption contribute to long term health of the network with user retention (lagging metric)

The newsfeed is the primary user experience for facebook. A poor newsfeed experience will cause churn. Introduction of Facebook likes thus is going to directly impact user retention or churn - % users becoming inactive on a weekly or monthly basis, sessions/uu - daily/weely, returning uu / weekly or monthly. If we were introducing Facebook likes and running an A/B experiment, we would look to see impact on user retention. 

4. Network Growth - did the social network as a whole keep growing due to increased engagement, connections

The more users find compelling content, the more they want to share and the more people discover the source of the content and want to be associated with the content. Thus increase in number of connections / user is another indicator of impact.  

 5. Revenue as Guard rail metric- Facebook needs to make sure of its overall health as a business - thus revenue is an equally important component. Introduction of likes fundamentally changes the product experience and impacts user engagement, thus the revenue earned. Therefore, we should track revenue / per active uu or a proxy. We want to make sure that revenue potential of sessions are not decreased due to change in newsfeed.

6. Impact to brand - Facebook brand could be adversely affected due to introduction of likes and consequent impact on newsfeed quality. Therefore, this could be monitored directly with inline surveys (NSAT) or other clear disengagement signals such as opt-outs.

 

 

by (73 points)
Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
To avoid this verification in future, please log in or register.
Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
To avoid this verification in future, please log in or register.