You are a PM for Netflix homepage and notice that while there is a lot of hits on the homepage (users are scrolling and browsing the content a lot. They are even watching the trailers and snippets.), there is a very high drop-off rate. While the time spent on the homepage has gone up, the content consumption is declining. How would you go about understanding this more and fixing this?

Question

You are a PM for Netflix homepage and notice that while there is a lot of hits on the homepage (users are scrolling and browsing the content a lot. They are even watching the trailers and snippets.), there is a very high drop-off rate. While the time spent on the homepage has gone up, the content consumption is declining. How would you go about understanding this more and fixing this?

Asked at Intuit over a year ago

885 views

Problem Solving category

Asked at

885 views

How to answer Problem Solving questions

Interview Guide

Top Problem Solving interview questions

Invite members
Invite by email

Add another

Joe_riley · Answer 1 · 2022-07-31T21:04:01+0000

Context

Let's just briefly recap what the Homepage is, why users use it, and how it fits into the bigger picture. The Homepage is the first thing users see when they launch the Netflix app, it displays recently watched and trending shows as well as suggestions. As a company Netflix's goal is to entertain the world and the Homepage helps further that mission by helping users discover entertaining content.

Clarification

Before we dive into trying to solve what's happening let's make sure we properly understand what we're observing. The total number of hits, trailer watches, and time spent on the home page has either increased or remained consistently high. I'm assuming this is on a per user basis as we mention next that there is a high drop off rate meaning users come to the home page browse and scroll then exit the app leading to overall content consumption going down. I'm assuming we're measuring content consumption in terms of minutes watched.

How much has the drop off rate increased and how much has content consumption gone down? This is kind of essential. If the drop off rate has increased by 2% but content consumption went down by 50% I would start looking at other areas of the product for an explanation.

I'm going to make two additional assumptions here:

We're confident in the accuracy of our metrics. Nothing feels worse than discovering the data you've been using to guide your decision making is innacurate.
The metrics we're looking at are from a large enough sample to be statistically significant.

Honing in on the issue

Now that we have a better understanding of what we're observing let's see if we can't narrow it down or isolate it further by asking a few questions:

What is the time range for these observations? Has the increase in drop off rate and decrease in content consumption happened in parallel? Did it happen suddenly or gradually over time?
Is there any particular city, country, or region where what we're observing is more or less severe?
Are our observations uniform across platforms? (Mobile app, desktop brower, smart TV,)
Does it vary by any particular demographic or user segment? Maybe younger users have less of an attention span and exit more often than older users.

Potential Causes

Based upon the answers to the above questions, we can propose some potential causes for why we might be seeing what we're seeing and how we would validate or invalidate it:

Are users browsing the Netflix home page to figure out what they might want to watch at a later time? For example, a user could be on the subway home from the office and browsing what they would want to watch later in the evening -> We could validate this by examining if these shorter micro sessions are happening at times that particular user would not be watching Netflix.
The quality of our content catalog could have gone down. Users may be looking for something to watch but not finding somethign interesting so they go watch cable or open another streaming service -> Validate by looking at the average ratings of our recent content additions.
Users are consuming content at a faster pace than we are adding it. Even if we are adding quality new content users may just be binge watching shows over the weekend then not finding anything wortwhile to watch afterwards -> Examine the rate of content consumption among those are now dropping off.
Decline in the quality of our recommendation engine. Maybe we tweaked our algorithm and it's performing better in some areas but worse in others. -> Do any algo adjustments line up with the timeline of metric changes we're obersrving?
Recent UI changes to the Home page. We could've made some less than optimal changes to the Home page that could be resulting in analysis paralysis for our users. If given a large menu I will spend 10 minutes optimizing the best dish to order. -> Look at recent releases for UI tweaks

Summary

The Netflix Homepage helps users find new entertaining content to watch. We've observed that users are spending more time on the Homepage but dropping off at a higher rate and we want to try and isolate this futher by looking at a few different elements such as: timeline, geography, platform type, and user segments. When we're able to isolate what we're observing further we'll have a better sense of just what the cause is but we've gone head and proposed several pontential causes and ways to either validate or invalidate them.

Anamika Chaudhary · Answer 2 · 2022-05-12T19:29:56+0000

Clarifying questions -

Time spent on home page has increased - Yes
Time spent on videos (daily) has reduced - Yes
Videos started per user (daily) - has it reduced? - Yes
Videos completed per user (daily) - has it reduced? - Yes
Has it reduced suddenly or gradually? - gradually
What kind of % reduction are these - 1%, 5%, 10%, >10% - 5% over a week
Is it happening on a specific platform? - No
Is it happening in a certain geography? - No
Is it happening for a certain set of users like new users vs old users? - New users (acquired in the past month)
Did we release anything significant recently? anything on the recommendations side? anything on the trailers screen? - No, only minor releases, nothing impactful
Are there any bugs in terms of starting the videos? - not that we have found
Are users finishing trailers - trailer completion rate is lower for these users vs old users

Summary till now - Users are exploring content, but not consuming enough content, consumption has been gradually reducing over a week, and we are seeing this trend mainly in new users who have been acquired recently within a month.

More clarifying questions - Has our marketing strategy changed? Are we targeting a new user persona which we didn’t have earlier on the platform? - Yes, we have launched a cheaper mobile plan in developing countries

Hypothesis -

We have launched a light subscription plan for mobile users in developing countries, but the content on the platform that’s been recommended to them is not suitable. For this reason, they are spending a lot of the time on the home page, exploring content, opening them, trying to watch trailers, but are not engaged and hence completion rate on trailers is lower and content consumption is also lower.

1. If we can recommend the right content to them, the content consumption for these users would go up

2. If we can make it easier for them to find the right content for themselves, the consumption would increase

Test -

Show more local language content shelves at the top for these users
Show content that is popular in their local area
Voice search for finding content
Translate app into local language
Ask them to sync contacts, and suggest content based on what their friends and family people are watching

Thursday · Answer 3 · 2022-04-29T14:28:32+0000

Clarification

1. The fall is sudden or gradually happening in a certain timeframe

2. The customers who leave are the premium cutomers or the non premium customers

3. When we say high drop rate? is it happening on the premium paid content or based on casual search

4. Are these droppers never turning up or they join scroll the page and left out frequently

5. Is it happening to any specific region?

Describe the product- Netflix provides premium content for the entertainment like movies, shows to the viewers. There are various competitors in this area like Amazone Prime, Disney hotstar and other OTT platforms but the quality of content is premium in the channels like Netflix thats why it is liked by the viewers across the markets. One of the weakness is Netflix is still not prominent amongst the middle class people in the developing coutries due to high premium price and mostly the good quality content is displayed in english language/hollywood movies. People are acquaitted to the other cheaper OTT platform and even set top boxes due wide network and the availability of content in the local native language.

Reasons of fall

1. The scrollers could be mostly the people who are non premium customers who are interested to see the content and to expore the same but when it comes to prices they leave the screen and shift to other media sources.

- Talk to the users and understand the root cause behind it and nature of users . Are they premium non premium users , unique or non unique users

2. There is sudden rise in price

3. Some restrictions have been imposed or the regulatory policies have been changed which is restricting the use of content to people of certain age like kids below 14 years of age

5. Content is not catching attention. Hence, people scroll it, check it and drop off

- Interview the users to understand what kind of content they are watching, thier demographics tastes to understand likings and disliking

6. Is there any other competitor has come in the recent past providing better services at low cost

- Investigate with the secondary sources on the new rivals

7. Some technical issues that has locked the account of the users which does not allow them to move ahead after the homepage

- Check root cause of the problem with the tech team

8. Poor internet connectivity which disallows the users to see good quality long videos at strectch

- Talk to the users and identify which set of internet providers provides what quality of internet service in that area. Is it compatiable to the system being used by the viewers to watch Netflix

After interviewing to the users and to the tech team and the service providers from the different angles I got to know that there is flaw in the UX design which is casuing compatibility use for the users who are viewing thru tablets vs those are watching thru televisions. When the users tries to enter in the main screen to fully view the content they tend to drop off from the screen due to inadequate size of the close button which looks normal in the bigger screeens but covers off in the small screen gadgets and the user inadvertetntly hit the close button and drop off the main content. We revisited the user journey thru the various gadgets and catch the problem and fix the issue with the UX team.