The Facebook Messenger usage is down. Figure out why. What would you do?

1) Describe the Product

So, just to make sure that we are on the same page. Facebook Messenger is that product that enables people to message each other, make calls, video-calls, and create rooms. Moreover, it is also integrated with MarketPlace. Is my understanding correct?

2) Clarification

• What do you mean by usage?

  • Are people logging in but taking no actions?

  • Are people not even opening the app?

  • Is avg session time decreasing?

  • How is it measured?

Let's assume that the interviewer lets me assume what usage means. So, I will take the path that avg session time has decreased. 

• How much is down? 1%, 10%, 30%?

• Is it gradual or sudden? If gradual, for how long?

• Is it happening in a particular geographic location?

• What about the user segment, is it specific?

Let's assume that the interviewer said that it was about 15%, a gradual drop over the last months. It's specific to South America, particularly Brazil.

--> Ok, so now we have more information. Just to recap where we are. We are trying to solve the 15% drop of avg session time on Messenger in Brazil over the last months.

--> Because it's a gradual and significant drop and specific to a region, I have some clues in my mind of what could be:

• External

  • Have we noticed any competitor entering the market or launching a new feature/product?

    • Yes. The Treasury Bank launched a new way of paying people, called "PIX". It's a free, 24/07, secure, and instantaneous form of transfering money to one another.

  • How do people use this PIX?

    • People can use their Bank Apps to send a PIX.

• Internal

  • Have we launched new features directed to Brazil?

    • Yes. We launched Whatsapp Pay over the last months.

3) Hypothesis

Ok, so I believe I have my hypothesis of what the problem can be.

Brazilians were using Messenger Pay to transfer money around, but with this new technology coming up from the government and our launch of Whatsapp Pay, they stopped using this feature on Messenger, thus decreasing the usage of the product.

In order to further check this hypothesis, I would look at some metrics:

- Number of transactions with WhatsappPay per week

- Avg session time on Whatsapp per week

- People who decreased avg session time on Messenger, compare this time decreased to the time they used spent, and check if it was related to Messenger Pay

- If possible to correlate users on whatsapp and messenger, check if the avg session time was transferred

- Follow the usage of PIX at high-level with Brazilian Government

Does it all make sense? Do you want me to focus on something else or dig deeper on another point?

4) Actions

Well, we should investigate further why people would trade Messenger to Whatsapp, because I believe that both features are kind of the same.

However, we could think about partnering with PIX to integrate on Messenger in order to get more credibility, so that users do not need to change platforms.

Give me feedback on this structure for answering this Facebook problem solving question.

1.Clarify

Could you explain what you mean with down and usage?

We can have different scenarios:

a)User are not opening the app.

b)User is opening less often the app.

c)Users are opening the app, but they are not replying to messages.

d)Users are opening the app but they are not sending messages, starting conversations.

2.Context questions

More clarifying questions:  Have we seen seasonality in this behavior? Is this happening in all the regions? Demographics? Have we made any changes to the product? Is this happening in all the devices? Do they stop using the Facebook app?

3.Isolating the problem

Assuming that we answer No to all these questions; we can think that the drop is due to a change in user behavior and not to external components.

More questions for every scenario:

a) Are they interacting with their friends on Facebook? Do we see an increase or decrease in the interactions?

b) Are they receiving notification? Do they open the app at a particular time of the day?

c) Are they ignoring all their contacts? What is the content of the messages that they don’t answer? How often are they receiving messages?

d) Are they still interacting with their friends on Facebook? Are they opening the conversations? Can we check if the type messages and then they discard them?

4.Hypothesis

a) Users are using Facebook app but not opening messenger, and they are interacting with their friends Facebook. I hypothesize that they are having conversations in the comments.

b) Users are opening the app less often, hypothesis is that is something wrong with the notifications.

c) Users are not replying messages and assuming that they are receiving notifications, the hypothesis is that they are receiving less engaging messages.

d) Users are not sending messages, and the hypothesis is that they are having problems with the messenger app to send messages.

5. Metrics.

a) I hypothesize that they are having conversations in the comments. I will measure if there is an increase:

in the time they spend on Facebook, in the number of posts that they publish, in the comments, the use of comment threads, and mentions. Also, I will measure if they are replying comments more often.

b)    Users are opening the app less often, my hypothesis is that is something wrong with the notifications.

 I will compare the number of notifications that they are receiving with the number of notifications that they used to receive. I will check the percentage of time that they are active when they open messenger. The rate of messages that they reply when they open the app.

c) Users are not replying messages and assuming that they are receiving notifications, hypothesis is that they are receiving less engaging messages.

I will check the percentage of messages that they are replying. The type of  content on the messages (pictures, stickers or text) that they are responding or ignoring (maybe they are receiving some kind of spam by analyzing the content we can figure out). I will compare the characteristics of the relation with people who are sending the messages: for example maybe they are not replying to users that they just added or have few interactions with them (perhaps these are spam creators), but they are responding messages to people who have been their Facebook friends for a long time. Also the frequency of the messages that they receive and don’t answer, which can indicate spam too.

d)    Users are not sending messages,  the hypothesis is that they are having problems with the messenger app to send messages.

I will check the percentage of conversations that the open and they send a message.

How many times they started a message and then discard it, and the time that they took to compose a message and then just discard it. If they searched for a feature for the message and if they found it and included it in the message, or they didn’t find it and then discarded the message. What is the kind of content that they are sending (pictures, videos, stickers, texts)?

6. Wrapping up:

1. For the hypothesis that they are interacting with their friends on Facebook, I will check the behavior in the Facebook app to test my hypothesis. Once that I confirm the theory I can go to case “d” to review: what is the reason that they prefer Facebook over messenger?. Maybe there is an attractive feature that is not available in messenger.
2. For the hypothesis of notifications, I focused on the notifications. I want to prove that if they receive enough notifications, they will reply to messages. If I find that even when they are receiving notifications, they don’t answer messages I can jump to the case ”c.”
3. For the case c, I want to find what makes the messages less engaging. I can figure out if there is a lot of spam or maybe the messenger features are not attractive enough.
4. For case d I want to check if there is something that is preventing to send the messages. If I find that they are taking a lot of time to compose a message and then they just don’t send it,  I would think something is crashing in the app, and indeed they tried to send the message. Also, I want to make sure that they find all features and elements they want for their messages.