You're the PM for Facebook Pay. What goals would you define and what metrics would you track to measure your team's success?

Start answering this Facebook metrics question by understanding the problem.

Understanding the problem

First off in order to establish a common understanding let's briefly describe what FB Pay is how and users use it. FB Pay let's users add their payment information to their FB account for use within FB, Messenger, and Instagram. From my understanding the primary use cases are users can purchase items, make donations, and send their friends or family money. 

In order to properly set the stage for this problem, it's important to view Facebook Pay in the context of it's parent company and their mission. Facebook's mission statement is to bring the world closer together and give people the power to build community. FB Pay helps further this mission statement by making it easier for people to exchange money within their community. 

By adding more value to users, simplified easy payments, at no additional cost, FB Pay further increases user engagement and stickiness with the FB platform. 

Metrics

There are a ton of different metrics we could use, so one of the ways I like to stay organized is by grouping them by different steps in the user lifecycle. It is also worth calling out that there are both users who send payments and users who receive them. Our metrics should accomodate and account for both types of users.

• Awareness - # of people who have signed up and entered their payment details
• Activation - # of people who have sent or received a payment
• Engagement - Average number of payments sent or received per user per week, number of users who have sent or received at least two payments in a week, number of users who have received 10 payments in a week (for stores /businesses),  Average size of a payment being sent or received
• Retention - % of people who have sent or received at least two payments this week who also did the same in the week prior
• Monetization - Will ignore, this is primarily not about making money. I do want to give a nod to the fact that there is value in knowing what our users are doing with their money.
• Referral - Will ignore, this isn't as important as the above steps. 

Prioritization

I think all of the above metrics have their own place, but if we were to focus on a single metric I would suggest we choose an engagement metric. Engagement is a great measure of whether or not our particular product or feature is adding value to users. If users aren't engaging with FB Pay then they don't find it useful and we aren't meaningfully simplifiying payments. 

The other situation in which users could not be engaging with FB pay is if they don't know about it. However, I would suggest we start with measuring engagement and then work our way backwards if need be.

Out of our proposed engagement metrics, I would prioritize monitoring the average number of payments sent and received per user per week. I like this metric over some of the other ones because it measures engagement on a per user basis where as the other proposed metrics measured absolute values aka pure # of users. The issue with measuring an absolute is that it will eb and flow with the total number of users on the platform independent of how well our product or feature is performing.

Downsides

While I'm confident our chosen metric above is a great indicator of success for FB Payments, no metric is perfect and it's worth calling out its downsides.

It is definitely more of a high level metric and doesn't give us insight into the various use cases or demographic breakdowns. For example, 90% of all transactions could be coming from ecommerce purchases on Instagram and we wouldn't know that with just our single metric. Other variables could include, business vs personal use, gender, age, device type, platform, transaction type, etc.

As part of a platform, we are inherently competing with the other features for user attention. We're assuming that FB Pay will add value to users and make them engage more / spend more time on Facebook's offerings. We want to make sure we aren't negatively impacting the overall health of the other platforms.

Summary

In order to measure how well FB payments helps facilitate users sending and receiving money we are going to measure the average number of payments sent and received per user per week

Here's my thoughts on answering this Facebook metrics interview question.

First, I'll like to start with what is FB Pay, FB's mission, and how FB Pay ties into it.

What is FB Pay: FB Pay allows users to send money across apps like Messenger, FB, IG, and WhatsApp, and shop on FB Shop and IG Shop.

FB’s mission: To give people the power to build community and bring the world closer together.

FB Pay’s mission: Make commerce more convenient, accessible, and secure for people on the apps.

FB Pay's mission aligns with FB's mission by simplifying payments and allowing people to send money to other people and shop online more easily across the apps.

Next, I'll like to define the goals for the product in terms of why users might want to use it, how they might benefit from it, and the business/strategic benefits.

User goal:

• People already use payments across the apps to shop, donate to causes and send money to each other. Facebook Pay will make these transactions easier and more seamless.

Business goal:

• Engagement & retention: Give people more value they can use FB for, and increase the stickiness of FB
• Increase online checkout conversion rate and sales and reduce time to check out on FB Shop and IG Shop
• Build a trusted online payment brand
• FB may eventually monetize the product by charging a transaction fee for FB Pay payments

Next, I'll like to go through who are the FB Pay users, and a list of desired user actions.

Users of FB Pay:

• Senders
  • Peers
• Receivers
  • Peers
  • commerce/businesses

User actions:

• Set up Facebook Pay app-by-app, or choose to set it up for use across apps by adding a payment method
• View payment history, manage payment methods (update or delete a payment method)
• Send payment to other people on one of the apps using FB Pay
• Make a purchase on IG Shop or FB Shop using FB Pay
• Transfer money received into a bank account
• Contact customer support via live chat for help

Now I'd like to turn each user action into a trackable, measurable metric broken down into the following buckets (Acquisition, Engagement, Retention, Conversion), which can affect the user & business goals.

Acquisition:

• number of users who set up FB Pay per app (FB, Messenger, IG, Whatsapp)
• the conversion rate of FB Pay (number of users who sets up FB Pay/number of users who clicks on FB pay) per app

Engagement:

• number of daily transactions using FB pay
• % of users who deleted their payment methods (did not subsequently add a payment method)
  • this is a proxy for % of users who “un-enrolled” from FB Pay
• % of users who contacted customer support via live chat
  • track contact reason, resolution time
  • we want to ensure that we understand the reasons for contact and that we’re able to resolve their issues
• Engagement metric of FB pay user (weighted scoring based on actions)
  • payment actions: send money, transfer money, make a purchase
  • view actions: view payment history
  • interact actions: replacing an existing payment method with a new method, updating the exp date
  • delete actions: deleting a payment method, which is correlated inversely to engagement
• % of FB Pay users with a high engagement rate
  • Work with data science to determine a baseline of what represents a high engagement rate

Retention:

• Monthly active FB Pay users
  • sent at least 1 payment/made at least 1 transfer or one purchase in 30 days

Conversion:

• Check-out CVR% and avg. $ spend of FB Pay users compared to non-FB Pay users on IG Shop and FB Shop
  • unplanned purchases
  • repeat purchases
  • frequency
• % of unsuccessful payments per week broken down by
  • user-driven errors: card declines due to insufficient funds, changed card numbers, or expired cards
  • system-driven: system outage

North start metric (P0):

Given FB Pay has already been launched for more than 2 years, I'm making the assumption that we already have a considerable amount FB Pay users on the platform, so I'd consider engagement as the primary goal of the product.

I'd like to use the number of daily transactions using FB pay as the north star metric.

Secondary metrics (P1):

I’d also like to monitor the number of monthly active FB pay users to ensure the health of FB Pay user base.

As a secondary goal, we’d like to increase the checkout rate and average spend of users paying with FB Pay on IG Shop and FB Shop to promote unplanned purchases and repeat purchases more frequently so I’d also like to track CVR% using FB Pay as a secondary metric.