Define the success metrics for videos on the Facebook Newsfeed.

Question

Define the success metrics for videos on the Facebook Newsfeed.

Asked at Meta (Facebook) over a year ago

910 views

Asked at

910 views

How to answer Metrics questions

Interview Guide

Top Meta (Facebook) interview questions

What is your favorite product? Why?89 answers | 263k views
How would you design a bicycle renting app for tourists?62 answers | 82.5k views
Build a product to buy and sell antiques.54 answers | 66.8k views
See Meta (Facebook) PM Interview Questions

Top Metrics interview questions

How would you measure the success of Facebook Likes?40 answers | 61.3k views
What metrics would you use to measure the success of the Save feature at Facebook? 39 answers | 87.2k views
How would you measure the success of Gmail?23 answers | 17k views
See Metrics PM Interview Questions

Invite members
Invite by email

Add another

rahul_g · Answer 1 · 2021-04-23T06:50:43+0000

Clarification

When you say Videos on Facebook Newsfeed

- are you talking from launch perspective or BAU state today when its already live - assuming the latter

- App or web or both -> assuming both

- Videos sit inside the newsfeed and is one of the many sub components /we are not talking about watch here

A little background about the company/product

Facebook's Mission is to connect the world and provide community building tools.

Newsfeed is probably the most used product outside of login flow since all users land here by default

From a product perspective Facebook's strength is it network size, network size in turn benefits from higher engagement and retention

Newsfeed helps drive engagement/retention by helping you discover relevant content. Content can be in a number of formats and in general video is consider a higher engagement format which explains why videos are on newsfeed.

User journey

Users login into FB app or side
Checkout there feed
- bounce off
- Engage
When engaging, the depth of engagement varies - users may view something briefly but scroll past it as it's not interesting enough or the snippet provided all the enough that she/he needed
Or they might click on a news article or photo or post or video
in case its a video then user clicks on the video
- Watches a % of it
  - Likes or comments or shares or saves it
- Drops off or watches another video
Continues scrolling the feed
Maybe comes back again

Product goal: Video is an engaging format for users and should results in more engagement for newsfeed and hence for engagement for Facebook. More engagement should imply more retention as well.

I want split into 2 metrics: User facing and product metrics

I'd keep tech metrics as a defensive overall metric

From a user facing perspective I would want to measure -> Additional contribution to timespent via the video format as a % share of timespent per user on average. This presents a clear unbiased picture on how much videos is contributing.

I would want to also measure what % of users have a video in their feed and what % engage with it

At a product level I would want to look at product health metric

- What % of users are exposed to video

- CTR rate for video

- Bounce rate - what % of users bounce of even before 10% of video is viewed

- I'd want to put some guard rails around this metric to prevent outliers from introducing error into the metric (not sure videos that are too short/have a definition of 10% of video or x mins of video whichever is lesser)

- Average and median length of videos viewed before user stops

- % of users watching more than 1 video in a session/day (unbounced traffic)-> Indicator of quality of recommendations

- Like/comment/share/save rate for videos -> quality indicator for videos

As primary metrics I would want to track : mission statement driven metric and product health metrics + counter metrics/defensive metrics to ensure we have the complete picture

Mission metric: Additional contribution to timespent via the video format as a % share of timespent per user on average.

Product health metric: Timespent/user on video who engage with videos.

The problem here is that both are lagging metrics and it implies that in case we'll only know about the engagement once that happens + it tells me nothing about distribution/engagement rate of product(for example CTR on video may go down but more posts may showup so time spent may remain unchanged)

To balance the above I want to look at the following

-> % of users exposed to video post

-> CTR on video posts

in addition I want to share the highest quality engagement - this can be a leading metric for video engagement so Id wnt to measure

- bounce rate

- Like/share rate

- Median timespent for videos for unbounced traffic

Finally I would look at video playback start time/fail rate as a tech health metric along with buffer issue frequency per min(how often does the video pause to buffer)