You noticed a drop in user engagement on recommended videos in Youtube, what metrics would you track to guide your next steps?

Before, I proceed, I would like to clarify a few terms:

When we say “recommended videos”, are we referring to the videos seen on the homepage? → Yes

When you say a drop in user engagement on recommended videos, do you mean users are not at all clicking on such videos & watching or they are clicking but not watching even 10 seconds or they are not commenting/sharing/reacting to such videos? → Not commenting/sharing/reacting to such videos

I will reiterate my understanding of the metric being highlighted here: For the recommended videos, if in a month, “x” number of engagements were seen per user, now that number has dropped to “0.9x” or “0.8x” per user.

I am assuming that it does not mean that the number of users engaging with such videos have dropped? Correct? → Yes

Is this drop gradual, sudden or seasonal? → Gradual

What’s the time frame of reference like is it week on week or month on month? → Month on Month

Is this drop specific to any geo? → No data Is this drop specific to any OS (android vs iOS) or device (mobile vs desktop vs TV) or native app vs web application? → No data

Just laying it out so YouTube is a 3-sided marketplace with creators creating the content (videos), viewers consuming or watching those videos and the advertisers.

I will just quickly walk through the user journey once:

1. User opens YouTube (logged in)
2. User sees a personalized feed of recommended videos on their homepage
3. User clicks on one such video
4. a. User does not watch the video b. User watches the video
5. a. User engages with the video b. User watches the next video c. User goes back to the home page and watches next recommended video or some other video d. User closes YouTube

Metrics that I am planning to track for the next steps are the user actions before and after the user engagement step:

1. Daily, Weekly and Monthly active users/viewers of YouTube (if the highly engaging viewers have dropped, user engagement will automatically drop)
2. No. of recommended videos in the personalized feed - can happen that because of algorithmic change there are not enough recommended videos to watch
3. No. of recommended videos clicked/user (week on week and month on month) - user cant engage if he/she is not clicking on the videos
4. No. of recommended videos watched/user (week on week and month on month) - user cant engage if he/she is not watching the videos
5. On average, no. of times the next suggested video is watched after watching/not watching a recommended video - if the algorithm to show next suggested video has been improved, viewers are no longer wasting time in engaging with the recommended video

The above metrics should be tracked for every geo.

The above metrics should be tracked across OS (android vs iOS), device (mobile vs desktop vs TV) and native app vs web application.

The above metrics should also be monitored for different topic categories.

Also, the engagement should be split between commenting, sharing and reacting/liking.

Problem statement - Drop in user engagement for YT recommended videos

Questions -

What do you mean by engagement? 

• Drop in views?
• Drop in likes?
• Drop in comments?
• Drop in shares? Drop can be in anything among these

• Web
• Mobile
  • Android
  • iOS

Were there any recent competitor product launch in the online content viewing space? No

• Do we have any data on the user feedback around the same, hintng possible churn to that platform?

• Avg screen time per user on YT
• % time spent on recommended videos
  • DAU/ WAU/ MAU
  • Video quality metrics (Analyse for short, moderate and long videos)
    • % videos viewed atleast once from recommended videos
    • Avg video content viewed at one go
    • % users coming back to watch the in progress videos
    • % videos watched completely
    • % videos liked
    • % videos shared
    • % videos commented
  • Video ad related metrics
    • Avg no. of ads per video
    • % users dropping off after each ad
    • % users clicking on any of the ads
  • Customer feedback
    • Avg customer ratings on recommended videos
    • Customer complaint rate
      • Split % of issues faced