You released a new feature on Facebook Stories that led to 5% decrease in number of active creators but a 10% increase in the number of daily active users. Would you roll this feature back?

Are these sudden changes or have been observed for quite some time?

Are the change in the metrics observed only after the new feature release or were happening before as well?

Could this be some seasonal change?

By active creators we mean the ones who creates posts, upload stories, videos etc and by active users we mean the ones who log into to Meta daily irrespective of whether they perform any activity.

I will structure the answer by considering it from technical, user profile/activity and external events/competition perspective.

Technical:

1. Has there been any change in the way the metrics are defined and measured?
2. Are these changes specific to any device such as mobile vs web or to any OS such as Android vs iOS?
3. Are these changes specific to the app version which contains the new feature?
4. Has any other feature of Meta been upgraded or changed from backend or UI perspective?
5. Are there any bug reports on story creation or story uploading process?
6. Are there any reports of the application crashing whenever try to create or upload stories or other posts?

 User profile/activity:

1. Are these changes specific to any region?
2. Are these changes specific to any user segment?
3. The increase in the daily active users à Do these users just scroll on Meta or view and react/comment to/on stories and other posts?
4. Has there been any change in the following metrics?
   • Views and engagements (reactions and comments) on stories
   • Views and engagements (reactions and comments) on posts/other content
   • Avg time required to create and upload stories
   • Avg time required to create and upload posts/other content
   • Avg no. of posts, stories and other content per creator
5. The decrease in the active creators à Did these creators mainly create stories or create other content as well? Were these creators celebrities, influencers, organizations, or regular consumers? Were these creators creating mainly paid content?

External events/competition perspective:

1. Is there any new competitive application in the market?
2. Is there any news of any competitor app providing more monetary benefits to creators?
3. Is there any government/political regulation advocating creators to move to competitor application?

If I observe that story creation and upload process is flawless and there are no major bug reports and all the health metrics mentioned are fine or improved, I will mostly attribute the decrease in active creators to competition and will not roll back the feature.

If I find some bug reports, instead of rolling back the feature, I will ask the team to fix the bugs and make the creator journey as smooth as possible.

In both the above cases, I will continue to track the 2 metrics.