How would you evaluate the success of Instagram Stories?

Question

How would you evaluate the success of Instagram Stories?

Post and review answers and feedback to answers in the comments section of this post.

Asked at Instagram over a year ago

17.4k views

Asked at

17.4k views

How to answer Metrics questions

Interview Guide

Top Metrics interview questions

How would you measure the success of Facebook Likes?40 answers | 61.3k views
What metrics would you use to measure the success of the Save feature at Facebook? 39 answers | 87.2k views
How would you measure the success of Gmail?23 answers | 17k views
See Metrics PM Interview Questions

Invite members
Invite by email

Add another

Utkarsh Binay Singh · Answer 1 · 2021-03-06T18:47:28+0000

Step 1: Explain your understanding of the product

Stories by instagram is a feature that help users connect/communicate with one another (people in their network) in the form of content (stories) which disappears after 24 hours. It is the most non-obtrusive form of communication.

Stories was launched by instagram couple of years back and has seen a great user adoption. Instagram being part of the facebook ecosystem, for this reason the mission of instagram is to connect people across the world and that's what it does with stories as well.

You can put text, add images/put pre-existing images, video, audio, filters, hashtags, location on your story. One can communicate with others by either share the story with everyone on their network or with particular people in their network.

The different kind of user groups using Instagram stories are as follows:

1) Brands/Organisations: Companies use stories as a way to promote their brand or promote any kind of product or for giveaway

2) Celebrities/Power users/Content creators/Influencers: These are people who have a huge following and post content (using post, stories) on a regular basis. These users post as well as consume other people's stories.:

3) Casual users: These are people who don't create any stories they just watch other people's stories. Don't use/view stories on a frequent basis

Step 2: Ask clarifying questions to narrow down the scope of the product/question

-What do you mean by evaluating success? (Is there any particular thing that you have in mind or any particular direction that you'd like me to follow or can i decide on my own and go ahead)

-Is there any defined success criteria across which we will have to evaluate the product? (Yes or no?)

-Any particular user group that you have in mind or do it for everyone? (Power users)

-IOS or Android? (IOS)

-Only instagram stories or some other features as well we have to consider? (Only stories)

-Any particular region that we have in mind or not? (US)

Step 3: Define a goal to further narrow down the scope of the product/question and give your reasons for the same

Since instagram is one of the largest social media platforms in the world, being part of the facebook ecosystem, user adoption and awareness is not a problem for this feature. Even though there's a lot of awareness about stories and a lot of people have used this feature as well,but we see that a lot of people are not using this feature actively.

Thus for the scope of this question we will focusing on 2 main goals namely engagement and retention as these goals directly tie down to the mission of the company which is to connect people across the world and bring them closer to each other. These 2 goals will also lead to increase in usage time which in-turn will lead to increase in revenue for the company and increase in value for the end user.

Step 4: List down the user journey.

1) Go to the instagram app

2) Sign-up/ Create an account

3) Login to the/your account

4) Create your first story

5) Save your first story for future use

6) Delete your first story

7) Share your story with everyone or people of your choice

8) Check other people's story

9) Reply to other people's story

10) Reply to comments on your own story

11) Log out

Step 5: List down the metrics for the different phases of the user journey

For the scope of this question we will be considering only Engagement and Retention related metrics as calculating the other metrics across Referral, Revenue. Adoption, Awareness doesn't make sense as they are not aligned towards our goal.

Engagement Related Metrics.

1) Avg # of stories created and viewed per power user

3) Avg # of replies per story and story type

4) % Inc in time(session) spent on instagram because of stories

5) Avg time spent per active user on stories

Retention related metrics

1) % of total users who have created more than 10 stories in the past 2 months

2)% of total users who are creating stories even after 6 months of creating their first story

3) DAU, WAU, MAU of this feature

Step 6: Evaluate and prioritise the metrics based on RICE condition

Engagement Related metrics

#	Relevance to company's mission	Impact to end user	Confidence in metrics collected	Implementation Effort
1	H	H	H	L
3	M/H	H	M	M
4	M	M	M	M/H
5	M	M	M	L/M

Retention related metrics

#	Relevance to company's mission	Impact to end user	Confidence in metrics collected	Implementation Effort
1	H	M/H	M	M
2	M/H	M/H	M	M
3	H	H	H	L

Based on the above criterias of User Engagement and User Retenetion be the goal of the product we have prioritised 2 metrics that we will track and measure the success of instagram stories with.

Primary metric

Avg # of stories created and viewed per power user

Secondary metrics

DAU, WAU, MAU of this feature

Step 7: Summarize your answer.

So for this question, i started off explaining instagram stories, defined it's mission, came up with the goals of the product based on the company's mission. We chose user engagement as user retention as the main goal for this product/feature.

Evaluated metrics across these goals and finally came up with a primary and secondary metrics which we will use to track the success of the feature keeping the goals in mind

Maria Ech · Answer 2 · 2020-03-20T19:50:46+0000

Instagram metrics questions call for a strong structure. For this particular question, here's what I used.

Understanding of the feature

As far as I'm aware, Instagram Stories allow users to snap photos and videos and upload them to their profile for others to see for 24 hours. It also allows users to customise those photos/videos with filters, music, etc. Other users can see those stories and comment on them or share them with other users.

User goal

By using this feature, Instagram users seek to share "instant" moments with friends and other followers and see what other friends or interesting people are doing. This is very aligned with Facebook's mission of giving folks tools to build community and bring the world closer together.

Business goal

I would say that this feature is trying to achieve 2 possible goals:

Engagement: make users use the feature more
Retention: have users keep coming back to the platform

If I had to focus on a single one, I'd go with engagement.

Actions

I'm going to list some of the actions users have with this feature. First of all, I have 2 types of users for stories:

Poster: users who post stories
Consumer: users who view stories and interact with them

I think both of these users are important for the feature to make sense as people can only consume and interact with stories if there's other people creating them. Users can then:

Post story
View story
Interact
1. Share
2. Comment

Metrics

Now I'm going to think about some metrics that can help me understand whether my team is reaching the business goal and what is the health of the feature.

Goal oriented
- Average # of stories posted by user (daily or weekly)
- Average # of stories viewed by user (daily or weekly)
- Average # of stories commented (daily or weekly)
- Average # of stories shared (daily or weekly)
Product health
- Distribution of type of stories posted (photo / video / quote / etc.)
- % of stories with filters
- Distribution of filters used

Prioritisation

To select my top line metrics I would say I'd want at least one of each "bucket" (goal and product health). From the goal oriented bucket I would want one metric that will allow me to see if the poster is engaged and one for the consumer.

Average # of stories posted by user (daily or weekly)
Average # of stories viewed by user (daily or weekly)

And for product health I would go with Distribution of filters used so that it will give me a good idea of what users are engaging the most with so we can focus on making it better.

So, to summarise, my metrics would be:

Average # of stories posted by user (daily or weekly)
Average # of stories viewed by user (daily or weekly)
Distribution of filters used

And these will help my team understand if we are driving more engagement with our efforts.

PM Team · Answer 3 · 2018-08-12T22:26:53+0000

The Instagram story feature let’s Instagram users take a picture or video and post it for all of their followers (if account is private) or for anyone on instagram (if account is public) to see. User’s can customize their stories by adding filters, text and interactive elements like tags of other Instagram account, links to their website or polls. Users of Instagram can view each others stories and scroll through all of the stories from the people they follow or use the discovery feature to watch stories from people they don’t yet follow on Instagram.

Goals this feature has:
Awareness – letting users know that stories are versatile – that they can be made interactive and that you can view stories from any public IG profile.
Acquisition + Market-Share – this feature helps FB compete with SnapChat and acquire users from SnapChat + acquire brand-name influencers from SnapChat
Activation – getting users that have never or seldom used IG stories to create or view their first story or five stories.
Engagement – getting users to view, engage with and create more stories daily + getting brand-name influencers to choose IG as the platform of choice
Retention – measuring what effect engaging with stories has on other engagement on IG such as increased following, interaction/messaging between users, etc.
Revenue – measuring how increased engagement with stories = more sponsored stories viewed –> more conversions for advertisers –> increase in revenue for FB from advertising

Different user personas of this feature:
1. Users active on IG but not on viewing or posting stories
2. Passive users that view stories but don’t post
3. Active regular users that view and post (under 5K followers)
5. Facebook advertisers – brands that create IG story ads

I’ll focus on two use cases:
1. Users viewing stories
2. Users creating stories

Why? Because having a large and active AUTHENTIC user base of both users that view stories and create stories is important to Instagram. Authentic users (not brands) are most likely to have habitual interactions on IG and more likely to buy from advertising on IG because they are accustomed to using IG for a personal reason (communication with friends) daily.

Customer Journeys

Viewing Stories:
1. Login to / open IG
2. Look at the top where IG stories are located (possible engage with other aspects of IG)
3. Click on 1 story to view
3 – A: Exit viewing stories
3 – B: Continue viewing other stories
3 – C: Engage with viewed story (answer poll, click on link, swipe up to view more)
4. Exit IG or continue engaging with IG (follow, comment, message)

Metrics for “Viewing Stories” use case:
1. % viewed story within a day of signing up for IG [Activation]
2. % of all IG users that view 1+ stories daily [Engagement] + growth of this #
3. % of all IG users that interact with 1+ stories daily [Revenue] + growth of this #
4. Average # of stories viewed per day per user [Engagement]
5. Average # of stories interacted with per day. Interaction = voted on poll, swiped up to view more, clicked on story link [Engagement]
5. % of all IG users that view 1+ sponsored story daily [Revenue] + growth of this #
6. % of all IG users that interact with 1+ sponsored story daily [Revenue] + growth of this #

Creating Stories:
1. OPTIONAL: take photo or story on phone
2. Login to / open IG
3. Click add story
4. Create story by taking video/photo OR post from phone
5. Customize story
6. Post
7. Look at stories to see # of views

Metrics for “Creating Stories” use case:
1. % created story within a day of signing up for IG [Activation]
2. % of all IG users that create 1+ stories daily [Engagement] + growth of this #
3. Average % growth in followers after creation of story per user (segment users by average with 5K followers) [Retention]
4. % of all IG users that are also advertisers and create 1+ sponsored stories daily [Revenue]

Top Metrics:

The metrics from that list that I’d choose to focus on are:
1. % of all IG users that view 1+ stories daily [Engagement] + growth of this #
2. % of all IG users that create 1+ stories daily [Engagement] + growth of this #

Both numbers will let me know the overall health of the IG stories feature and if more people are getting value out of it by viewing or creating stories. Growth in these numbers also directly impacts the # of sponsored stories IG users will see and increases the % of users that interact with sponsored content. Increasing interaction with sponsored content increases the trust advertisers have in IG ads and increases FB ads revenue.

si5PZCa · Answer 4 · 2021-11-15T17:31:48+0000

Question: You are the PM of Instagram stories. What goal would you set and how would you measure success?

Are we referring to the product as it is today or when it was launched?

Stories is the feature where people post videos/pics only for 24 hours, right?

FB mission: provide tools to build communities and bring the world closer together.

IG Stories mission: allow users to connect more easily by sharing casual updates about their day

What is the goal? Increase engagement with stories feature as this is a mature product (maybe also increase revenues).

User segments:

1. People who post the story

a. “regular” ppl who wants to update their friends about their lives – biggest segment which I believe has the lowest engagement.

b. Celebrities / Influencers – have followers/fans they want to update

c. Advertisers – Ads stories

2. People who view the story

Looking at IG stories as a mature product I assume it has good engagement from influencers and advertisers hence I will focus on monitoring engagement for the “regular” ppl segment.

User journey:

1. Login IG

2. Know that I can create a story

3. Click on “create story”

4. Know how to use all features in the story creation tool

a. Pics

b. videos

c. add text/emojis

d. reference a post

e. reference other’s posts

5. Know how to share the story – maybe also in FB

6. See comments and reactions about the story

Metrics categories:

· Awareness

o Users with first story view per day

· Activation

o Users with first story created per week

o Total users who posted a story / total IG users (%)

· Engagement

o Users with 2nd+ story created per week

o Average # of stories per user

o Frequency of story creation – DAU/WAU story creation

o Average engagements with stories per user by segment

§ Pro (100 <)

§ Regular (10 < posted stories < 100)

§ Beginner (posted stories < 10)

o Ave # of story items per user

· Retention

o Total users who haven’t posted stories in the last 14 days (but activated)

· Monetization

o Ave $ per account (total ads income from stories/total accounts)

· Referral

o IG stories shares in FB

Evaluate metrics based on mission, value to user, level of accuracy, effort to collect

Selected metrics:

Top level:

o Total users who posted a story divided by total IG users (%) – shows general adoption of the stories feature.

o Average # of stories per user – gives a general idea if we are trending in the right direction overall.

Others – will focus on engagement from “regular” ppl with the goal to see them improve their engagement and retention:

o Average # of stories per user with less than 1000 followers

o Users with 2nd+ story created per week less than 1000 followers divided by users with less than 1000 followers

o Ave # of story items per user with less than 1000 followers

o Frequency of story creation – DAU/WAU story creation per user with less than 1000 followers

o Total users who haven’t posted stories in the last 14 days (but activated)

Gunjan Bhardwaj · Answer 5 · 2020-05-28T01:09:17+0000

Instagram stories is a way for Instagram users to provide an update about small or big events in their life more frequently than the posts and in an easier way than the posts. The users can share a short video/a picture/a text/even a post as part of their stories with their followers.

According to me the goal of Instagram story feature was to increase user engagement on the platform. With posts disappearing within 24 hours, a user needs to keep posting more regularly to engage his/her audience. This will increase the number of times a user has to open the app. Although, a user will open the app to post a story. He/she may end up performing more actions than just posting the story.

Also, the business users on the platform use stories as a way to acquire customers through links and swipe ups.

Considering these 2 use cases of Instagram stories, I will talk about only the most important metrics i.e. engagement metrics and acquisition/conversion metrics for business users.

Also, considering that a social networking platform thrives through the existence of 2 types of users - content creators and content viewers. The more the views a creator gets, the more he/she is motivated to keep his/her audience engaged. I will talk about metrics for both kinds of users.

First, let’s talk about how does a user creates a story. User Journey - Creator:

User locates the stories button and clicks on it
User selects media from her gallery/clicks a picture/takes video/writes text/takes a boomerang (I will stick to these options for the moment as I don’t remember other options available in the feature)
The user modifies the post using filters such as text font filter, background filter, geolocation tagging, GIFs, stickers, tag other users, etc.
User selects the audience for her post - all followers or only close friends
User posts the story

Engagement Metrics for creators:

# stories created per user per day

Avg # of filters used per story per user: to understand whether introducing filters has encouraged people to create more stories or have they no effect in story creation behavior of a user

Next, let’s talk about the journey of viewing a story. User journey - Viewer:

User sees the story scroll at the top of his/her profile
User clicks and views a story
User reacts to the story
User clicks to view the next story
User swipes to view stories of next followed user

Engagement Metrics for viewers:

# stories viewed per user per session

Avg # reactions for stories per user: to understand whether the viewer is active or passive

# stories viewed per session vs # posts viewed per session: to understand whether introducing stories feature has negatively impacted viewing of posts by the users

# swipes for next users: to understand whether there is a preference for specific friends/ connects by the viewer so that Instagram can work on prioritizing stories of a few users over the others

Also, not to forget the business users who have become the major way of monetizing Instagram product for Facebook.

Important metrics for business users:

Click through rate for per link per story: to understand how many users did the business acquire through Instagram

Similarly, I will also track the click-through rate for “Swipe Up” option per story

The more conversions, the more Instagram earns through the platform.

PM Team · Answer 6 · 2019-02-02T14:53:04+0000

Let's talk about what is IG Stories.

IG Stories give users the possibility to post images and videos for their followers and also edit their posts with filters, tags, links, etc. Followers can see the stories. Users can also search for Stories based on location and hashtag.

Users can reply to stories.

Stories disappear after 24 hours and moved to Stories Archive, which is only visible to user.

Now according to this description, what is the goal of stories? I think the goal is to engage users more with IG. User engagement means for IG more Ad Impressions and more Ad clicks.

So having this goal in mind (user engagement), lets go through the user journey.

Discovery/Awareness: User sees other friends are posting Stories, or sees the button to post a story herself.
Engagement: There are different actions that can be defined as Engagement
- Posting stories
- Replying to stories
- Visiting Stories Archive
- Searching for stories
- Viewing other people stories
Retention: User returns to post stories, user begins to follow new people who post stories
Convertion/Monetization: User gets the habit of watching stories and so watches also stories from brands. User does not notice the difference between monetized stories and non-brand stories

Since our Goal was increasing engagement, let's concentrate on the engagement phase. But before that. Why do people spend their time on IG? They are curious to know who is doing what, they want to show to their friends what they are doing ===> they want to be entertained and socially accepted.

How do they get to find who is doing what? They search for people or content, they follow people.

How do they show to their friends what they are doing? The post images, videos

How are they entertained? The more authentic the post is, the more entertained the user is. The content should be aligned to user's interests and earlier behaviour, or it can be targetting the user herself, e.g. a video of this user posted by a friend

How is the user socially accepted? The user gets feedback on what he posts, through likes, comments, and replies (in case of stories)

I also need to remember I have different types of users:

Brands --->Their goal is to market their goods/services. They want to create a community of people who love their goods and hence spread the word in the online community.
Power users ---> Invest more than 3 hours a day on IG. Their life is defined by what they see and the feedback they get on IG.
Passive users: They interact with IG less than 30 minutes a day. They just watch the posts/stories, but never comment or post anything.
Influencers ---> Users with big audience that target a certain industry, for example beauty, music or food.

I will concentrate of power users and passive users. Power users are already spending a lot of time on IG, but do not have as many followers as Influencers. On the other hand, they follow many people. My hypothesis is that if they produce authentic content, the content begins to be interesting enough for passive users as well so that they interact.

So once again:

BIZ Goal: Increase user engagement

User Goal: Be entertained and socially accepted

------------------------

So let's go back to the engagement phase. What were the actions that the user does in this phase?

Posting stories
Replying to stories
Visiting Stories Archive
Searching for stories
Viewing other people stories

----------------------------

How to increase engagement? By giving reward.

Reward:

What is the reward I get by posting a story?

Reply ---> Social acceptance
More Followers ---> Having more followers means you are not alone and you are socially accepted. User gets the feeling of importance (even if the importance is defined in her small world of followers, for the user, she is the center of this network)

Other users can be rewarded by being tagged in the story. Being tagged is a reward because it means I am connected to poster and poster has cared about me, plus, it potentially could mean I belong to a social group/activity/event and hence I am socially accepted ====> Things I am tagged in activate more joy in my brain.

-------------------------

Power Users

Some metrics to measure the engagement of power users:

Creation

%Stories vs. %posts ----> (share of stories, are we cannabalizing other prodcuts?)

Content

My goal here is to simply understand what content is posted as story more often. This helps me to understand what are the trends in using the story. In addition, I can correlate this information with the #visits and #replies to post, to sort stories and give a higher ranking to stories that are more engaging.

%stories which are live video
%stories that are video vs image
%content (e.g. selfie, nature, animal, family photo)
%stories with friend tags
%stories with hashtags
%stories with filters
%stories with location

Retention/Engagement

Creation

%users who post 1+ story/day + growth---> Users want to continue using stories or they have abandoned it
Growth in the avg #views per story in 1 month ---> Users find the concept of Story interesting and want to be entertained with it
[brands]%stories with paid impression (stories from brands/advertisers)
[brands]%stories with paid links (stories from brands/advertisers)

Viewing

%user who view 1+ story a day + growth
Avg #stories viewed per user /day + growth
Avg #stories interacted with per user /day + growth
Growth/decline in the %stories that get 0 views in the 24 hours that they are visible
Growth/decline in the %stories that get 0 replies in the 24 hours that they are visible
Monthly growth in %users who have replied to at least one story in that month ---> Reward to poster
Monthly Growth in the avg #replies per story ----> Reward to the poster. Is the content interesting enough that the user answers to it?
Growth in the #followers for users with more than X stories a day. (X can be the average #stories for power users) ---> Having more followers is a Reward for story poster. Since the followed person posts a lot of stories, the followers are exposed to more stories and might also find stories interesting and begin posting them (needs to be checked if it is true or not)
%users who watch a story after they were tagged in it --> Reward Do users react to this reward?
%tagged users who replied to story ---> Reaction to Reward ---> Tagged user Rewards the poster by replying
[brands] %stories which had 1+ clicks on paid links (stories from brands/advertisers) ---> Revenue for IG and advertisers
[brands]%stories with paid impression with were viewed 1+ times (stories from brands/advertisers) ---> Revenue for IG and advertisers
[brands]%users who followed a brand/advertisers after watching a story / day + growth ---> Revenue for IG and advertisers
[brands]Growth in #followers for brands/advertisers with more than X stories a day (X can be the average #stories for brand) ---> Revenue for IG and advertisers

-----------------------------------------------------

Passive users:

I also have a metric to know if things have changed for passive users. It is more challenging to get the passive users for this product, because of the nature of product: The story disappears after 24 hours, and passive user is active less than 30 minutes.

Monthly Growth in #users who have posted their first story
Monthly Growth in #users who replied to a story for the first time
Monthly Growth in #users who for the first time followed a user after watching her story

-----------------------------------------------------

Prioritisation

I will prioritise based on my goal for users: Entertainment and social acceptance. I define entertainment with authentic content, and the social acceptance with the reward system mentioned above.

To find what content attracts more audience (which might be a sign of authenticity) I use the measures defined in section content and try to find a correlation between features of a story (e.g. content, hashtags, links, video, image) and the amount of engagement (views, replies, followers) it creates. If I combine this information with user behaviours/habits/interests, I can rank stories in a more smart way.

To measure entertainment, I will concentrate on section Retention, Creation 1-2 and Viewing 1--3, 6-10

To measure revenue, I choose Creation 3,4 and Viewing 11-14

---------------------------------------------------

Summary

I described why people might want to post stories and what is their goal: Begin entertained and socially accepted. I described how to extract features out of the content to find entertaining content. I also described the reward system (reply, tag, followers) which would correlate with social acceptance feeling.

I skipped the passive users for now. When I have assured high content quality then I can concentrate on acquisition of passive users and I follow my progress by metrics mentioned for passive users.