Imagine you were in charge of Facebook Watch. What metric would you want to measure?

Questions 

• Are we considering the current version of FB watch or imagining a state before it was launched

• Assumption - removing microsoft fb watch since its a different app altogether

• Assumption - not considering watch party or watch together

•  

Product / Users / Value

• FB watch or  FB video as it's called right now is the FB on demand video platform. It consolidates all video content into FB. Its a different tab in FB and there are sub tabs for For You / Reels / Live / Popular / Following 

• How does it play with FB mission - FB mission is to build communities and get people together, a video watching experience certainly is well aligned to this mission. Finding common video content is one way for people to get together.

• Its a fairly mature product which has been around since 2017. In terms of competition it faces competition from Youtube / Tiktok . 

• Business goal 

  • The business goal here is to deepen engagement, increase retention and optimize revenue.

• Users

  • Video creators

    • Video creators create a FB page

    • Video creators acquire followers

    • Video creators upload a video on “Whats on your mind”

    • Video creators set visibility of video to public

    • FB algorithm evaluates eligibility of the video 

      • Reels shared to a public audience.

      • Produced by a Page with 5,000 followers or more.

      • Published by a verified profile with 50,000 followers or more.

    • Get views / reactions / shares 

    • Get followers

  • Viewers

    • Go to the videos tab

    • Get video recommendations on the “For you” Tab

    • Scroll through video feed

    • Watch video 

    • Engage with video by liking / sharing / sending in messenger

    • Follow the creator

• North star metric 

  • # of videos watched per user per day

• Goals 

  • Engagement

    • High engagement ensures people spend more time on platform, maximizes opportunity for ad revenue

      • Video views

      • Video completion rate

      • Interactions per video

  • Retention

    • This is important since we want to make sure users are not moving away to other platforms 

      • Creator retention rate 

      • Viewer retention rate

      • Increase in creators

  • Monetization

    • With a mature product, can contribute significantly to facebooks revenues

      • ARPU

      • CTR

      • Swipe up rate

  • Creator brand loyalty

    •  

  • Innovation

  •  
•  

• Metrics

  • # of videos uploaded per user

  • # of videos uploaded on verified profile

  • # of videos uploaded on pages

  • # of videos shared on reels

  • # of videos added on video feed 

  • # of videos watched

  • # total time watched

  • Average time watched per video per user per day

  • Average time watched per user per day

  • Average time watched per video

  • # of new followers

  • # of new followed 

  • # of interactions 

  • Total time on facebook

• Prioritization 

  • # of videos added on video feed 

  • # of videos watched

  • Average time watched per video per user per day

  • ARPU

• Tradeoff

  • We want to make sure there is quality content ; content creators shouldnt prioritize longer content at the expense of quality

• Counter metrics

  • Drop off rates

    • If users dont like the content they can exit the video midway. 

    • % of users who stopped watching video midway

  • Bounce rates

    • If users get disinterested due to long videos they might drop off the platform

    • No of users who close the app immediately after watching videos

first i would like to clarify the question.Here it's not given why we are measuring the metrics.So i will repeat the question as I am in charge meta(Facebook) Watch.and i have to choose the metrics for the success of this feature.

Facebook watch feature allows you to see video based on your interest it also allows you to give reactions and comments on them and also you can connect with other people in same time.

Goal:- goal of the feature was to give a competition to YT and allow users to have more options for viewing and posting thus increasing the ads monetization.

Metrics:-

Activation-

A1.% of people who have used this feature at least once during a period.

Engagement:-

B1.Average time spent on this feature per user.

B2.avearge time spent on this feature per category(action,sci-fi,news,cartoons).

Retention:-

C1.% of user used this feature this month vs previous month.

Monetization:-

D1.number of ads clicks per user.

D2.number of ads seen by user.

Referral:-

E1.% of people who have seen the video when their friend shared video.

For the success of this feature I would like to focus on B1,C1 & D1.

1. Describe the product

Facebook Watch is a section on Facebook that allows the user to view, share, react and comment on videos uploaded by creators and publishers. Watch is available on mobile devices globally, and on desktops in certain countries.

2. Clarify the scope of the question 

Awareness - Number of users clicking on facebook watch/are aware of this feature

Acquisition - new DAU for FB Watch, % of new FB users Vs new DAU for FB Watch

Usage - DAU, Duration spent on FB Watch

Revenue - Click through advertisements

Engagement- Comment, copy links, sharing videos

Facebook’s mission is to give people the power to share and make the world more open and connected. Through Facebook watch users can share their video content - live or recorded  and help them connect with other users or followers of their page. 

For this exercise I would like to focus on acquisition as I believe that most facebook users don’t use this feature as they mostly go to Instagram which provides the same functionality in some capacity. FB is mostly used by its users for groups, sharing content/getting updates on their friends and commenting on articles.   If more facebook users start using Facebook Watch it can help with making FB more sticky for its users.

3. Walk through the customer journey - Acquisition 

• Clicks on FB Watch if aware of the feature/made aware by a friend

• Scrolls through videos

• May or may not click on a video/comment/share depending on whether the use is intrigued by the video caption

• May click other tabs such as live videos, shows, search, save

• Exit to some other facebook feature or keep engaging

4. Choose a goal

• Acquisition- 

  • Clicks on FB Watch if aware of the feature/made aware by a friend→

    • Number of FB users clicking on FB Watch

    • % of new FB users Vs new DAU for FB Watch

  • Scrolls through videos  → 

    • Number of videos clicked

    • Number of videos watched till the end

    • Number of new views per video

  • May or may not comment/share depending on the impression the video leaves

    • Number of new views per video vs number of shares/video

    • Number of new views per video vs number of comments/video

  • May click other tabs such as live videos, shows, search, save

    • Number of new DAU users Vs Number of clicks for live videos, shows, search, save

  • Exit to some other facebook feature or keep engaging

    • Time spent on FB watch during first visit per new DAU

Counter metric - Number of users landing on FB Watch for first time and not returning to FB Watch

5. Evaluate the metric

Use the RICE format to evalute the following 

6. Summary

The metrics I would track to measure the success of facebook watch are -

• Number of FB users clicking on FB Watch

• % of new FB users Vs new DAU/MAU for FB Watch

• Number of videos clicked

• Counter metric - Number of users landing on FB Watch for first time and not returning to FB Watch

Before I establish the goals for FB watch, I’d first like to align our understanding of it. FB Watch is a sub-product that sits as a tab on the main FB app for watching videos across multiple genres and lengths. A user can scroll endlessly through videos or watch a creator going live. Consumers engage with them however they'd like - like, comment, share - both within the app or externally.

It’s an ecosystem with two components

1. Consumers, who have diverse preferences.
2. Creators, that can be either individuals or media production houses. Larger creators can also choose to run ads within their videos.

Similar players in this space include:

1. YouTube
2. IGTV

When I think about an NSM, I try to think of that one goal, that would bring value to all the components involved in the ecosystem. In this case, the NSM would be

which I will break down into L1 metrics described below.

1. Active users

   1. Daily Active Watchers (DAW), defined as those consumers who watch at least 1 minute of video a day. These can be calculated in terms of

      1. # Number: Measuring it in terms of number is helpful to see if the overall count is going up or down.

      2. % Percentage: Measuring it in terms of % with respect to the overall ecosystem helps in knowing if the user base for FB Watch is growing/declining wrt to the FB’s overall user base.

      While we’ll also track Weekly, Monthly cadence for active users, we’d have Daily on the dashboard since consuming video content is a daily behaviour.

   2. Weekly Active Creators (WAC), defined as those creators (either individuals or media production houses) that produce/list at least 1 video a week.

2. Engagement

   1. % of DAWs consuming at least x minutes of video a day, and/or like, comment, share the video.

   2. % of WACs producing at least x # of videos in a week.

      In both the above, x is >1

   3. L2 metrics

      1. % consumption of videos across different video genres
      2. % completion of videos across different video lengths

      3. # of minutes spent in the Watch tab reading comments

      4. Ratio of DAWs that watch on the Newsfeed vs those that watch on the dedicated tab

   4. # of creators with x mins of audience attention/week

3. Retention

   Defined as the user that does the basic activity that constitutes them as either a DAW or WAC. The two retention metrics calculated are:

   1. % of DAWs that come back on:
      1. D1, tells us how easy or difficult it is to navigate and interact with Watch
      2. D7, tells us how good or bad the algorithm is
      3. D30, tells us if we have enough depth and breadth of content for maximum engagement
   2. % of WACs that come back on:
      1. D7, tells us if creators find it easy to upload content along with the required meta details
      2. D30
      3. D90, tells us if the creators find enough value to continue on the platform. Value is measured in terms of either engagement or revenue derived from content uploaded

1. # of videos reported

2. Less engagement on other sub-products within the FB app.

Clarifying Questions and goals:
1. What is the key goal of FB Watch? Why FB wants to offer FB watch? Why FB wants to enter in entertainment industry? How is it linked to their primary vision - connecting people? Any specific geography, user demography FB is focusing on?
2. Which stage of product lifecycle FB watch is in? What is the key goal at the current stage of product lifecycle?

Metrics:
General Metrics: Awareness, Acquisition, Adoption/Activation, Engagement, Retention, Profitability
Metrics to Monitor as per the stage of product lifecylce - Awareness, Acquisition/Activation, Engagement
Go deep in each of the metrics to segregate further. 

Awareness - What are the primary channels of awareness? What is the view rate, hit rate, registrations for free trials?

Acquisition/Activation - What is the no. of actual registrations through each of the channels? What is the % of renewal rates? what is the churn rate?

Engagement - No. of watch hours per user per day? No. of parasite users and their watch hours? What is the values for these metrics in competitive products? we can dig further by analysing which content is more liked/disliked by users etc.



 

Clarification:

Are you looking for any specific success metric? Engagement metrics : A user experience metric or a revenue focused metric?

Interiewer: Upto you pick one or two of your choice.

Facebook watch: Allows users to view videos from various groups or public figures or pages and also allows users to create videos and post. The organizations vision it aligns to is people connect with people through engagement with the videos by viewing, sharing, liking, commenting and mentioning friends and family.

Let me quickly go over the User journey:

Facebook watch is available as a separate tab.  User logs in to fb -> User is on the home page, with the news feed, tab and other stuff that the home page has -> User looks at the facebook watch tab -> Clicks on the facebook watch tab -> Views the videos posted from various groups or public figures -> User can search for the videos of choice as well and looks for the relevant search results -> Views the videos -> Likes, shares or comments.

Based on the above user journey Goals I would set are :

1.      Drive user engagement by first making more users click on facebook watch.

2.     Increase avg eyeball time on the videos.

3.     Increase the engagement in with liking, sharing and commenting.

4.     Increase the engagement with the ads

5.     Search should provide very accurate results to the users’ search.

Metrics that would drive the above goals:

1.     CTR of the facebook watch tab

2.     Avg eyeball time spent on the video

3.     CTR of like, share and comment

4.     Avg eye ball time spent on the ad

5.     CTR of the ad

6.     Avg number of searches for a particular video by the user

7.     The same video metrics after the search results pop up

8.     Search performance metrics (Internal)

9.     DAU, MAU

Top Success metrics to be picked:

1.     CTR of facebook watch tab

2.     Avg eyeball time spent on a video

Some of the strategies that would drive the above metrics:

1.     Video quality control based on users’ bandwidth. So that people from various geo locations with various kinds of internet speeds would be able to watch the video without problems in streaming.  In that way the avg eyeball time on a video as well increases, avg eye ball time on the ad increases, user engagement in clicking the ad, liking sharing and commenting on the videos also increases.

2.     Make the search platform more powerful so that users can find very accurate results

3.     Show the user relevant videos based on the user’s interests and behavior

4.     Provide more relevant ads to the users based on users’ behavior with ads

Mission: Let's first restate what Facebook Watch's mission is - Facebook Watch is a destination that people can visit to find the videos that matter to them, the video creators they want to see, and other fans that they can connect with.

Users: Based on the whole journey there are multiple users in the watch ecosystem 

1. Video creators - Creators of videos can have different personas and range from celebrities, organizations, or individuals. 
2. Video consumers - These consumers are the biggest fans of the content being published
3. Friends of consumers - These would be friends who get an invite from the biggest fans to watch videos along with them especially when there is a watch party

Metrics: I would like to categorize my metrics into 2 broad themes

User related metrics - rationale of why this is important

Product growth or engagement

1. Weekly number of creators on the platform - Provides an idea into the top line of this feature to understand how many content creators we generally have on the platform. This should be segmented by cohort (Individuals, celebs, large organizations) along with regional split.
2. Weekly number of videos posted on the platform - How much content is posted by these creators
3. Average weekly views of the videos - Measure of engagement for the consumers
4. Average weekly shares/comments/reactions - Measure of engagement for the consumers
5. Number of Saves to watch later - Provides directional signal on consumer engagement
6. Weekly Number of follows - Consumers retention for creators
7. Number of friend invites & accepts to watch party - New product to try and further increase engagement

1. Hide clickthrough rate - Depicts how many times we are not showing relevant content to users as they clicked hide.
2. Feedback provided by consumers - Working assumption that feedback is generally used to provide negative information, will help improve the relevance

Product health metrics 

1. Average video load time - Benchmark to competitors and see if it requires improvement
2. Average video or audio errors - Reduce errors to mitigate for bad user experience

1. Create a score with weights attributed to each of the engagement related metrics i.e. shares, reactions, comments etc.
2. % of consumers clicking hide and providing feedback