Instagram is launching video calling. How would you test and measure the success of it?
0 votes
in Metrics by (12 points) | 415 views

4 Answers

+1 vote

Product Desc- Instagram Video calling is an option on top within Instagram messaging, giving the opportunity to have a video conversation with friends over audio. It acts as a quick platform tp connect rather than going on alternative apps for initiating a video call.

FB Mission- To build community and bring ppl closer together and Instagram video calling helps ppl to connect and build community through having them have video calls and take the communication with a personal touch.

Goal- To increase engagement

Customer Journey:

Open conversation with a friend- access option to initiate video call -> it connects to a friend and once the friend accepts- you have a video call.

Hypothesis: Launching VC will increase engagement ( time spent per user on Instagram)

Test: 1. Number of ppl prefer VC rather than just text

2. Time spent per user on Insta increases post-VC launch

3. Number of users not opting in for VC 

Platform Health Metrics:

  1. #video calls per user per day-->Conversion
  2. CTR of VC feature--->Awareness
  3. time spent on video calls / total time spent on Instagram per user per day--Engagement
  4. %increase in time spent on Instagram post video calling feature rollout
  5. %increase in messaging conversations per user per day-Engagement
  6. #missed video calls per user per day
  7. Average time spent per video call per user per day--Engagement
  8. Drop rate/ of video calls--Retention

North Star Metric: number 3 and 4 as it aligns with our goal for user engagement.


Counter Metrics:

  1. Average time spent per video call per user per day-- if that is too high that could be because quality may be poor and user on opposite side is unable to hear and it may take longer to keep a call going.
See less
by (20 points)
0 votes

Instagram is launching video calling. How would you test and measure the success of it?

By video calling are you referring to a feature that lets you call users individually/privately unlike live which anyone can opt into 

Assuming you want to identify metrics for the success here WRT launch which means a focus on adoption


Instagram is a photo/video sharing app - its mission is to capture the world's moment and gives you the tools to do that

We are launching so whats the goal of the launch - adoption vs engagement , Id say former(adoption focusseD) since it takes time to build habits 

Video calling 

Video calling allows you to video call your contacts , as a reciever youd want this to come from someone in your network so assuming its only for 2 way follow relationships

Is it opt-in by default or do you have to opt-in manually, assumings its former

Whats the goal here

Allow people to closely connect with people in their network in a private fashion and help build a community 

users needs


Share a beautiful moment with reciever 

Connect with receiver regarding something they shared

Generally catch up with reciever

good quality stream


No spam, I dont want to be called by people I don't know 

Good quality stream


Assuming this is only enabled if the user pair is following each other 
Since this is a social feature assuming that there are enough users on it already and the flag for the feature was kept off
Can users opt-out? assuming they can since this is not messaging platform at its core

Goal at launch: drive adoption and trial (Short term)

Given that IG is not the default when it comes to video/calls/messaging ( though it's moving towards that) assume that this is opportunity driven use case rather a daily use case so retention may not be a meaningful metric 

I would breakdown North star metrics into the following : 

Product (engagement focussed): Mins spent on live calls (with a graph for the following) = Unique users x sessions/user x avg time/session

launch focussed(adoption focussed):  % of eligible users(support version + live call enabled) using calls in a meaningful fashion (actually made a 2 way call)


Health metrics - these are metrics I'll use to understand the health of the product


Discovery Funnel : What % of users discover the feature , what % of these users actually tried it

NPS score: Gives me a sense of the users are feeling when they use the product

% users opting out:  This is critical in terms of understanding if the defaults we've chosen make sense or do we need to make changes because you dont want to annoy users

Spam rate: What % of calls are spam - ie rejected multiple times. 

Avg call length: Gives you a sense of depth of engagment this is generating

Overall timespent metrics for IG: Are the users spending more time on IG because of this feature

P1 - retention focussed

Weekly rolling retention:  users using this feature in a meaningful way(call actually connected) on a weekly basis

Drop off rate: users who tried for the first time(callers only) and dropped out the next week 

See less
by (165 points)
0 votes

Instagram is launching video calling. How would you test and measure the success of it?



  • Improved User experience (Engagement time)

  • Increase adoption / repeat usage (retention) 

  • Assumption - Since, instagram is a free product (goal is not monetization). Hence, not focussing on it. 



  1. # of video calls made per day

  2. % of sessions w/ video calls made

  3. Average time spent on video call



  1. Unique daily users making 1 video calls / DAU%

  2. Avg. calls per day = Total calls per day / DAU



  1. Frequency clustering - # of video calls made in a week

    1. 0-1

    2. 1-3

    3. 3-5

    4. 5-10

    5. 10+

7. Weekly retention cohort - users who made call on 1st day, made call on 2nd, 3rd, 4th and so on to the next day.


North Metrics: 1, 3, 6


Counter Metrics - video calls per day can increase (for a while) as user want to try the product or due to patchy network and user making multiple video calls in patch. 


See less
by (79 points)
0 votes

There are few questions that I have before i dive deep into this?  

Can I describe the video calling feature as below:

Video Calling Feature dscription: Ability to make a video call to your contacts by simple tap of a button - (Sounds right)

1. Why is the video calling feature being launched? - is it to increase engagement on the platform? - (Yes)

2. Is the video calling feature being launched around the world or are we testing it in certain location? - (Across the world) 

3. Is it being launched for all the platforms? Web as well as mobile & for all OS? - (All)

Vision: Vision of instagram is to give users an ability to create professional looking photos & share it easily.

GOAL: The objective is to increase engagement on the instagram platform and hence the revenue

Let's dive into the metrics we would want to track & success criteria for each of these metrics. Initally we would target high level metrics. I would divide the metrics in the following categories:

Discoverability (Growth & acquisition)

- Feature Penetration (%)  i.e. users who have used this feature atleast once post launch  Success criteria - Target % of Instagram's base. WhatsApp's video calling data can be used as a proxy. Highest target value will be % penetration of what WhatsApp has achieved.

New users added per month i.e. first time users of video calling feature each month. Success criteria - Positive Growth compared to previous month


- Active Users of Video call (DAU, WAU, MAU), Success criteria:  Positive Growth compared to previous period

- Total calls per Active user -  Success criteria:  Positive Growth compared to previous period

- Duration of calls per Active user - Success criteria:  Positive Growth compared to previous period


- Quality of Service - QOS - Success Criteria - Should be excellent  (Industry defined standard for VOIP)

- Calls drop % - Success criteria - Less than X% (Industry defined standard for VOIP)

- Average Rating for the version on which the feature was launched- Success Criteria - >4

Returning users

-% Returning users for monthSuccess criteria: Positive growth compared to previous month


- Ads revenue per session (in which video call occurred) - Success criteria: Should be equal or more to the average Ads revenue per session

See less
Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
To avoid this verification in future, please log in or register.
Your name to display (optional):
Privacy: Your email address will only be used for sending these notifications.
To avoid this verification in future, please log in or register.