How would you decide whether or not doubling the amount of video content in the home timeline was a good idea?

 I am considering twitter case here.

Product Description

Twitter is a social media platform for users to publish their thoughts, interact, share content, and read breaking news in real time. it generates revenue through advertising services and data licensing.  

Clarifying questions:

Are we talking about device specific?  Mobile or desktop or both? Both

Are we talking about particular geography or worldwide? Worldwide

Are we aiming for any particular metric such as increasing userbase or engagement or revenue? depend upon you

Adding video to timeline increase engagement, reach to user and followers. Here engagement I am considering

·         share, like and comment

·         watching videos in full screen, turn on the sound and enabling high definition.   

Adding Video Increase engagement means increasing revenue. If Engagement is still decreasing then causes would be

Internal factors

·         Content of the video

·         Category of the video

·         Instream ads make irritating to user

·         Any recent upgrade of software such as video viewer

·         More videos make user irritating

External factors

·         Seasonality such as election campaign, any special event etc.

·         Similar videos are shown on another site such as YouTube, Facebook etc.

·         User behaviours have changed

Hypothesis and Suggested action

Here I am assuming Content and Category of videos as root cause. After that I will do A/B test on doubling the video and placement of video to see the engagement among user.

·         Content and category of video does not appeal to user.

Content: we will educate content creator with good quality of video and content.

Category: we will ask user about Broad Categories such as news, sports, etc. and make it narrow down such as in case of sports cricket, hockey, etc.

·         A/B test on Doubling Amount of video

Hypothesis: Doubling amount of video on twitter timeline increase engagement.

Experiment Group

Control Group: Timeline with video from followers as usual with tweets

Variable Group: Timeline with doubling the amount of video such as trended, Ranked video with tweets

Success Metrics:

1.      Total time spent on twitter

2.      Time spent on watching the video

3.      Engagement such as Like, comment, share on video

Trade-off:

Promoted/Ranked video sometimes make irritating to user.

·         A/B test on placement of video

Hypothesis: Placement of Video based on Ranking will increase engagement.

Experiment Group:

Control Group: Placement of video in reverse chronological order

Variable Group: Placement of video on the basis of ranking order

Success Metrics

1.      Total time spent on twitter

2.      Time spent on watching the video

3.      Engagement such as Like, comment, share on video

Trade-off:

Ranking placement of video sometimes make awkward to user due to wrong sequence related to same event.