Facebook Newsfeed engagement is down 10%. How would you report this issue to the CEO?

A standard metrics question and potential structure for answering it.

Clarification

1. Does the down happened all of a sudden on one day during specifc time or it has gone down gradually over the period of time

2. Does it shows any pattern like fall in the commerical feed/personal feeds shared by the network on any specific period of time?

3. Does it happens only on mobile version or on all kind of web version

Description of feature

Newsfeed helps the users in getting the updated/most recent posts and share there perspective on it by the way of likes, comments, emojis

Possible reasons for the fall

Internal Factors

1. Posts are not getting synced due to technical issues

2. Delay/latency observed while scrolling the pages

3. Users are not even able to login in the facebook may be due to account lock

4. A change in the UX/UI design or the slight modification in the feature which may result in poor user experience or the users are not use to it

5. Quality of content has been deteriorated

6. A defect/bug

External Factors

1. Accounts have been hacked for particular set of users

2. People not initiating the new posts due to some event during the specific time like world cup match, election results, festivals

3. Users have been diverted to some other means like newspaper, book reading or a trip planned due to sudden discounts offered by the companies that guage the attention towards other alternatives

4. Poor internet network restricts the users to login into facebook

5. New competitors are attracting the facebook users may be thru large scale referrals

Let’s that the user interface for the video player was recently changed on mobile. We would then ask questions to explore this such as, “What exactly was changed in the UI and was any drop to be expected as a result?”

After discussing with the interviewer, we learn that the UI change involved making the “Send video to device” button two times larger, and reducing the “Full screen” button by half its original size. The interviewer also explains that the “Send video to device” button can be used to play a video that’s on your mobile phone on another device such as a Chromecast, Roku, SmartTV, etc.

This is interesting, but we need to investigate further. At that point, we could form a hypothesis and say something like, “Have we noticed a change in the frequency at which the ‘Send video to device’ and ‘Full screen’ buttons are being used on mobile? Maybe mobile users are having a harder time tapping the ‘Full screen’ button now that it’s smaller, and are tapping the ‘Send video to device’ button by accident because it’s too big? Tapping ‘Send video to device’ would end the session and therefore decrease the average time per session. And if the smaller ‘Full screen’ button is harder to tap, it might frustrate some users and lead them to end their session early.”

Let’s imagine that the interviewer confirms that this is likely one of the drivers of the decrease. Before making any final conclusion, we would continue exploring the remaining elements (i.e. Internal External factors) to confirm that they aren’t also a driver of the change we are observing. Let’s imagine we quickly discuss this with our interviewer, and they confirm there’s nothing significant to report related to those remaining possible factors.

Now that we’ve stepped through our framework, we can conclude by saying  I would explain to the executive team that our time spent watching per session has dropped by 5% month-over-month on mobile because of a change in the video player UI, which has negatively affected the user experience. Mobile users are having a hard time tapping the new, smaller ‘Full screen’ button and are ending their session prematurely. They are also now more often tapping the larger ‘Send video to device’ button unintentionally which also ends their session early.