You’re the product manager for Facebook Live. You notice usage is down 20%. How do you go about finding the root cause?

Context

First, let's just quickly recap what Facebook Live is and how it fits into the Facebook ecosystem. Facebook Live let's users livestream themselves to users on the Facebook platform similar to Twitch. While there is gaming content on Facebook Live there is also other content like streamers exploring the world around them or showcasing their talents such as music or cooking. Streamers can chat with their viewers. Facebook's Mission is to bring the world closer together and Facebook Live let's streamers connect with their followers in a real and meaningful way that is more authentic and interactive than pre-recorded videos.

Strategically speaking this feature is primarily aimed at engagement as a way of retaining Facebooks large existing user base in hopes they don't defect to other platforms like Twitch, YouTube, or Tiktok which all feature a Live capability.

Clarification

First off, let's just clarify what exactly the question is saying. "Usage is down" is a fairly ambiguous statement. Usage amongst who, streamers, viewers, or advertisers? We need more clarity in terms of just what metric or set of metrics we are looking at to determine usage is down. For the sake of this problem, let's just assume viewer usage is down. A few metrics that could indicate Facebook Live usage amongst viewers are as follows:

1. Total Number of Minutes Watched - This gives us an absolute measure of how much Live content is being consumed
2. Average Number of Minutes Watched per user - This is a good relative metric whereas metric #1 is an absolute metric which will wax and wane with FB usage overall
3. Average number of Interactions(comments, likes, subscribes, donations) per stream - Interactions are a deeper form of engagement whereas viewing minutes is more surface level engagement, are they both down?

• Is this decrease in usage universal across geographies or do certain countries or cities have more or less of a drop in engagement?
• How long has this drop been going on for? Was this a sharp sudden decrease or was a slow gradual decling that is now at 20%?
• Is this endemic to a specific platform like mobile vs desktop? or browser vs app?
• Is there a particular demographic or user segment for whom this decline is more or less apparent? Ex, teens aged 15-18 or live music streams.

• Overall Facebook Usage Declines - If there are less Facebook users in general there is going to be less Facebook Live usage as well
• Livestreaming Ban / Censorship - It's possible a country like China, who has a history of censorship, might become unnerved by the amount of viewership some of these livestreams get and ban them to maintain control. Imagine a foreign Russian speaker streaming the rest of the world's views on the Ukrainian conflict to Russian citizens.
• Stricter Community Guidelines - Maybe we as an organization decided to impose stricter community guidelines amongst our streamers and whole content categories like Hot Tub Streams are no longer valid.
• Product Cannibalization - Facebook is a very mature product jam packed with a variety of features all competing for users' attention. It is possible a newly introduced feature like Stories is taking engagement away from other features like Live. Alternatively, other existing features could have just been given more prominent positioning within the UI.
• Competition - Streaming is a competitive space like we mentioned above. A new competitor could of launched or an existing competitor could've gained marketshare. For example, a popular streamer like Ninja could've previously been using FB Live but then negotiated an exclusive contract with YouTube.
• Bugs - If we notice engagement drops isolated to specific browers or software versions it's possible something tehnical could  be not working right which would negatively impact users and their engagement.

• I should've stated an initial assumption is that our reporting is accurate. So many times have I seen bugs in the reporting or collection of the metrics themselves. Nothing feels worse than finding out the data you've been using to guide your decision making is invalid to a certain degree.
• 20% is a pretty large drop so I'm assuming that is statistically signifcant and not part of the natural variance in user behavior or the metrics that monitor them. 

Clarifying qs:

Is this a sudden drop (might be due to a SEV) or did it happen over multiple days/weeks? Has the drop been driven by a single market/region (Russia ban etc.)?

Has the drop been driven by a certain demographic (e.g. Young Adults)?

ASSUMING THE DROP IS NOT SUDDEN BUT HAPPENED OVER THE PAST MONTH

Ok now let’s understand the metric. What does usage mean? Fb Live is a product that exist within the FB app and allows creators to express themselves by sharing live content (both short or long form video). In order for a user to be counted towards Fb live MAU, they need to go to the Live video tab and (1) broadcast a video or (2) watch, like or comment on a live video. It’s a multi-sided ecosystem, relevant for creators, viewers and Fb alike. So what does usage mean in this context?

It can mean a drop in:

- MAU for creators (people who broadcast video)

-  MAU for viewers

- avg number of videos created per day/month

- avg number of videos created per user per month

- engagement e.g. avg num of views per video, avg num of interactions per video, avg num of comments per user per month

Is it ok if I assume it is a 20% drop in the number of monthly Live video created on the platform compared to the previous month?

Let's understand what impacts that metric:

• A drop in creators? Competitors like TT or even IG Live

• A drop in the avg num of views/comments per week might have prompted creators to post less

• Time spent per user per month in Live ? Has it decreased or not? 

• A change to the UI: maybe we are testing a new UI and it’s less straightforward to people how to get to FB Live. 

• Self cannibalization: Reels? In order to investigate this we would check if avg time spent per user per month in FB overall has stayed the same? What happened to time spent in Reels?

So let’s assume that avg time spent on Fb per user per month has been consistent. Then root cause is likely due to changes in app (e.g. Reels launch)

A. Clarity –

1. Waow! 20% is a huge drop! Is this gradual? Or Sudden?   

2. Where is this drop? Views, Likes, Subscriptions or User profiles

3. Is this drop to a certain user demographic? Certain Market?

4. Is their a drop in any other related metrics?

Framework – To find the root cause this drop, i would require data on external and internal causes

Solution –

External Data Needs -

1.     Work with the PR team if there is anything released that gives an image change

2.     New competitor in the Market or a competitor has reinstated itself like Google Plus or Orkut and the users are deflected towards exploring the new competitor?

5.     Data or privacy concerns with facebook? Any critical bug that has been found and publicised?

6.     Change in any other metric? Like report abuse

Internal Data Needs -

1.     Server issues

2.     Any release of a feature that caused issues in the exisitng production environments?

3.     Critial Defects that are still pending for resolution

I would be needing a meeting with my companies execs, and relavant sr. managers and managers, where I will -

1.     Present the root cause analysis and solution

2.     Additional needs – resources, escalations, designs etc.

3.     Timelines/roadmap around my solution

4.     Execution Map of the solution – Teams, Risks, Mitigation rollout plan