Utilization went down by 45% on app XYZ in Italy for the month of August. Give a reason why and draft a plan to fix it.

1) Check out the reviews and ratings.

2) Take direct feedback from users.

3) Find out what other related/ non-related apps has the unusual pattern in that month.

4) Analyse the growth cycle of the product. If the product has crossed its peak phase, try to come up with some new features.

5) Find out the page analytics and check the drop point of the app.

6) Check out if you rolled some offers which has expired or covered all the required user base.

7) Look into the market news in that duration and see if there's something related.

8) Find out which metrics is getting changed: hit rate, CPC, bounce rate, conversion rate etc.

9) Analyse what services your website is providing and if there's some issue in any of the services in earlier month.

If nothing helps, ask the internal teams what they changed, that might have impacted the result.

•  What is meant by utilization rate?
  • User sign-up
  • User activation
  • # login/day
  • # hours spent/user-day
• Can you please tell me little bit about the xyz app?
  • What value does it add to the user's life?
  • Who are the target audience?
  • Who are the competitors?
  • Is it a local app, or a global app?
  • Which all platforms is xyz available?
• Any significant changes happened to the app recently?
  • New feature introduced
  • Existing feature removed
  • Any free service made paid
  • Any security incidence
• Any significant change(s) brought in by competitors?
• Any new competitor - what are their USPs?
• What are the users/ customers saying - on app stores, on review websites?
• Any significant changes in the platform ecosystem of the app?
• Any change in the geo-political situation? (esp. if it is a travel app) --> however, if we investigate starting from internal factors to external factors, probably we would detect the problems, and would not have to factor geo-political situations.

Honestly, I think the question is quite vague. So the discussion can take any direction based on the responses to the exploratory questions.

Let’s assume utilization means engagement. The problem statement is vague so I’d ask some questions to narrow down the root causes:

– Is it down 45% YoY or MoM?
– Is the same app available in other countries and how are they performing?
– Are there similar apps that are showing a decline in publicly available proxy metrics (eg: # of app ratings being submitted every week)
– What’s changed in the industry? Any new competitors?
– Is the decline from new customers or repeat customers?
– What source of traffic is causing the decline?
– Is the decline in select components of the app?
– Where in the conversion funnel do you see higher than expected drop offs?
– What do you see in customer feedback? What is the nature of customer service calls in August?

Hi Bijan, a great detailed reply. However, dont you think you should have started by asking what is meant by utilization, because it could be engagement as you mentioned in your last paragraph. It would be good to start there and then hypothesize about each option.

My initial guess is that this is too big of a drop in a short time to be caused by competitors and given that Apple and Google don’t launch their new OS in August, it’s unlikely that it’s caused by an update to an OS or launch of a product by major competitor such as Apple.

First, I’ll take a look at list of updates and changes recently made to the app to get a picture of what things has changed internally. Maybe one of the changes have broken a flow in the app or has impacted the user experience of the product. The app reviews can also help me see if there is a trending complaint about some aspect of the product. Let’s assume that there was’t many complaints and there weren’t any major releases or updates that could have caused this.

Then I’ll check to see if this is related to seasonality of the product usage. I’ll look at historical data to see if I see a similar drop in previous years. Assuming there wasn’t any drop in the previous years, I’ll look at other apps within Google to see if the issue was isolated or across the spectram. If it was across the platform, I would know that it is probably not related to my particular app and I would look for more macro-related issues. Examples would be telecom provider changing data plans or having outages. Now let’s assume the issue is isolated.

I’ll first look at top of the funnel to see if there have been any drops in the number of app daily opens. If there isn’t, I’ll look at the engagement rates to see if I get any hints on the issue. Numbers to look at could be time spent, number of certain tasks completed, etc. For example, if I find out the number of tasks completed is same as before but time spent has come down, it might be an indication that an update has made the app more efficient.

Le’s assume that the drop is in engagement. Then I will look at more metrics related to engagement and particular flows that are contributing the most to the user engagement and see if I see any changes in user behaviour in those flow. Hopefully, this will help me spot the problem.

Would love to get people’s feedback on this.