Let's say that Facebook would like to change the user interface of the composer feature (the posting box) to be more like Instagram. Instead of a box, Facebook would add a "+" button at the bottom of the page. How would you test if this is a good idea?

Question

Let's say that Facebook would like to change the user interface of the composer feature (the posting box) to be more like Instagram. Instead of a box, Facebook would add a "+" button at the bottom of the page. How would you test if this is a good idea?

Asked at Meta (Facebook) over a year ago

2k views

Asked at

2k views

How to answer Metrics questions

Interview Guide

Top Meta (Facebook) interview questions

What is your favorite product? Why?89 answers | 263k views
How would you design a bicycle renting app for tourists?62 answers | 82.5k views
Build a product to buy and sell antiques.54 answers | 66.8k views
See Meta (Facebook) PM Interview Questions

Top Metrics interview questions

How would you measure the success of Facebook Likes?40 answers | 61.3k views
What metrics would you use to measure the success of the Save feature at Facebook? 39 answers | 87.2k views
How would you measure the success of Gmail?23 answers | 17k views
See Metrics PM Interview Questions

Invite members
Invite by email

Add another

Amit Aggarwal · Answer 1 · 2020-07-27T08:08:04+0000

Hi Himani,

My feedback to your answer.

Your answer talks much about the metrics that should be used to measure the success of the "+" button feature and very less about "how will you test it to know if + is good". You have touch base those points (month to month ) and user demographics however they are shallow.

I would say - this question is suppose to be answered around "how would you test if + button is a good feature or not ". As a clarification question- you could have asked what does "good" implies here. He may have answered with the metric like "Increased number of total posts". Once you have this metric, you can start working out how will you test the feature

Here is a quick gist of what could have been done

Problem - "how would you test if + button is a good feature or not "

Amit - What does "good" implies here

Interviewer - "Total increase in the number of posts"

Amit - Do we have any constraints like on resources

Interviewer - No you do not have any constraints on the resources

Approach

"how would you test if + button is a good feature or not "

1. Figuring out the testbed segments

    1.1 Users from one most active user country like US, OR India

             Hypothesis - Due to the dominant of users of these countries - Feature success much depends on them

    1.2 Users from mix of most active users countries like US AND India and China

          Hypothesis - Due to different cultural and social behaviours of the countries, the feature success vary significantly

Assume, we are going ahead with our 1.2 hypothesis

2. Figuring out the customers in the segment

2.1 Users from mix of most active users countries like US AND India and China

        2.1.1 - Select customers which have a definitive pattern of posting the posts over last few months

                Hypothesis - Due to the definitive post pattern, it would be easy to see if these users have same of the declined post after rolling out the "+" feature button

        2.1.2 - Select the users who have no pattern in posting the posts over last few months

               Hypothesis - Due to non post pattern, it would be easy to see if these users are posting as it ( the + button feature ) has made it convenient to post the posts

Assume, we are going ahead with our 2.1.1 hypothesis

3. Figuring out the number of users in the segment

Do the same thing - here we have to make sure you have chosen enough users which makes the test results statistical significant.

Here statistical significance number matters a lot as Facebook as a huge audience base of more than 1B. So you need to figure out on how many users would you like to test this feature rollout.

So we will roll out the features

Himani Singh · Answer 2 · 2020-07-23T05:27:25+0000

FB’s mission is to bring the world closer together and to build a community by allowing users to connect to friends and family, to share what’s happening around the globe and to express their views and opinions on what matters to them.

Instagram mission: To capture and share the world’s moments.

The basic difference between how user posts on FB and IG is:

- On FB users can share content from across the web, they can express themselves in words alone, share and re-share photos and videos from other social media platforms.

- IG users mainly focus on sharing photos with less emphasis on words. IG provides a platform to boost user’s individual creativity and does not facilitate sharing content from other social media platforms e.g. youtube etc. Though it allows users to provide youtube links.

What is the goal of composer (post) feature on irrespective of the platform?

The goal of this feature is –

1. Engagement – which aims at increased content production and increased content consumption

2. Retention – so that user continues to come back to the platform and create and consume content.

3. Revenue – User engagement helps in providing richer user insights thereby driving more personalized content in user feed and also more targeted advertising which can be monetized.

FB wants to change the text box post option to + CTA which will be placed at the bottom of the page just like IG.

The key metrics to measure are:

1. Discoverability: How easy or difficult it is for the user to discover the new “+” feature for posting?

- Number of users clicking on the + / post CTA button.

-Number of posts shared per user per session

-Number of photos shared per user per session

-Number of videos shared per user per session

2.Engagement:

-Number of comments per post per session

-Number of reactions per post per session

-Number of re-shares per post per session

-Number of clicks per post per session

-Average session length per user

-Post frequency

Each of the above metrics needs to be measured from 2 aspects:

1.Content type – which content type is creating more engagement using the + option compared to the existing one?

- User demographics – which user type is engaging more using the + option compared to the existing one?

I would then conduct an A/B test with the control: existing post feature and treatment: + post feature.

I would conduct the test for almost a month so that my test is not biased.

If the results of the A/B test are leaning towards people of a certain age group say 20-25 who are engaging more frequently through a specific content type e.g. photos, does this drive the overall engagement on FB and also does it increase the overall active user percentage? Also does it improve the overall monthly user retention rate? If yes, then it is a good idea.

Maduri Narasimhan · Answer 3 · 2022-04-06T02:44:50+0000

Assumption: I am assuming here that there is sound reason as to why we want to test this idea. Otherwise I wouldn't want to break what's already perfectly working. Posts are everywhere, so I would restrict the test to FB newsfeed first.

Understand users : I would work with Data team to gain an understanding of the user base on FB and Insta:

1. What are cross section of users that use both FB and insta?

2. What are the users that only use FB but not insta?

Hypothesis: I would generate a few hypothesis around the user segments and expected behaviors:

1.For the intersection user segment, I expect to see very minimal post declines with the user experience change as the user experience is not new.

2.For the user segment that are on FB but not on instagram, I expect to see a greater decline in posting and therefore overall engagement declining in the short term (4-6 weeks)

3. For the user segment that are on FB but not on instagram, I expect to see a minimal decline in posting after 4-6 weeks of launch

4. For both user segments, I would see in a marginal increase in posting after 6 weeks of launch

Choosing test group for A/B test:

1. Users that are in the high frequency posting category

2. Users of top 3 countries that have maximum posts

I would run tests for all of the above hypotheses. If the number of posts increase after 6 weeks of launch, I will scale it, otherwise kill the test.

Ruby Payal · Answer 4 · 2020-08-04T10:31:41+0000

Facebook is considering to change its posting box to be more like instagram-

A plus button at the bottom instead square box at the top of your feed as well as on your own profile

How to know is it good or bad idea?

A/B testing which will help us randomly serving visitors this Plus button at the bottom and the following metrics would be used to judge

Visibility and accessibility : # % of people who clicked on plus button
Usability: # % of people who used to plus button to post
Impact: # % change in the frequency of post ( with respect to their earlier frequency of post using composer box)

# Change in the Avg ratio of Text:Video:Text and video to measure change in content type of posts using this plus button

OR

Could come up with a post highlighting the new feature and measure the metrics for the post- No of likes, comments, share . Could also seek feedback from consumers in comments. Although not advisable instead should go for A/B testing.