Google Maps had a 10% drop in daily active users. What do you do?

How to answer this problem solving question.

Clarifying Questions:

1. Google Maps have several parts, are we talking about the maps app or the API product that powers maps for other businesses? - Consider the maps app
2. Is this change sudden or in comparison to a longer period of time? - over a week
3. Is this specific to any particular platform - web, android, ios, android auto etc? - Across the platform
4. Is this specific to any geography? in the United States
5. Can I assume that active users open the app and perform any activity in the app - Yes

Thanks for clarifying the questions. I want to quickly discuss the usage of the Google Maps and will evaluate various external and internal factors due to which the usage might have dropped.

Google Maps Usage - casual users

1. Going from point to point and using directions
2. Checking out a place on the maps, perusing the reviews, ratings and general information about the place

External Factors

1. Ecosystem Related Changes
   1. Regulatory reasons - The app was banned in specific countries or geographies, which led to sudden reduction in these areas
   2. Lockdowns or Shelter in place orders - Like COVID19, is there a pandemic or a similar disaster event that has caused people to shelter in place
   3. PR Issues - Have there been any news maligning Google Maps or an issue caused because of the product, which resulted in stopping usage
   4. Internet Loss - Is this triggered by a blackout or other reasons because of which certain areas could have been cut off from the internet, not just from Google Maps
2. Seasonality
   1. Holidays - Is it a big holiday when people prefer to stay at home
   2. Adverse Climate - Has there been heavy snowing or a climate event that resulted in a reduction of usage

Internal Factors

1. Analytics Related
   1. Misfired data capture - Data is not being captured accurately due to faulty instrumentation
   2. Logging Issues - Data is not being logged accurately due to an issue in the analytics stack
2. Product Related
   1. New Feature - A new feature is confusing the users
   2. Changed User Experience - New user experience is causing issues
   3. Version Support - Have we stopped supporting certain versions of the product, due to which users stopped using the product
   4. Compatibility with New OS launches - Have there been any updates logged to various operating systems due to which Google maps is functioning incorrectly
3. Engineering Related
   1. Software Bug - Bugs are preventing proper usage
   2. Crashing pages - Pages are crashing due to various reasons - over consumption of memory etc.
   3. Performance Issues - Latency - Yes

Problem Validation

We realized using the analysis that the product usage has reduced significantly since the app wasn't loading fast enough for users on the app. To test this, we can check the page loading times of the various pages and where there has been a significant increase in the median loads during the past week. Secondly, we can also try to establish a correlation between users experiencing latency and turning dormant the next few days.

Investigation

Once this is established, I will further dig down to see which users are facing a latency issue - by platform, by geography, by age group, by action on the app etc.

After identifying the exact situation for the latency, I will jump back into the product to examine why there is a load happening. Some reasons could be:

1. Ingesting new points of interest data - this is the first time load for the user
2. Maps haven't been resized properly leading to longer load times
3. Search indexes haven't been fully prepared, leading to slow searches
4. Certain software package has been refreshed and was being downloaded at run-time

Solutions

Depending on the challenges, we will implement solutions to see if there is a marked improvement.

1. Checking the points of interest data for poor data or resizing the data set
2. Scaling down the images to load faster
3. Re-indexing all the data
4. Downloading the packages along with the app itself

Once the issue is fixed, I will send re-engagement notifications and emails to the users to try out the app again.

Summary

We examined various internal and external factors that could be causing the drop to identify that due to latency in the product, users were dropping off. We investigated various reasons why this could be happening and identified solutions that could help get the daily active users back up again.

Clarification: 

• How are DAUs counted I.e. what is defined as engagement action ? —> Means app opens and some action within the app

Approach

1. I will first ask a series of questions to identify the potential reasons to why this is happening
2. I will then formulate my hypothesis around the reasons and determine ways to address
3. I will outline actions we need to take to prevent this from happening in the future as well as determine if we need to make any internal / external communications

Root Cause

• TIME: How long has this drop been happening. 
  • Day, week, month? 
• GEO: Which region does this drop impact
  • Global, Regional, Country specific?
• PLATFORM: Which platforms does it impact
  • Web or Mobile. If mobile iOS or Android? 
• SEASONAL: Is this a seasonal change 
  • Has this happened in the past during specific times of the year of is the first time it is happened
• GOOGLE IMPACT: Is this only impacting Google maps or does it also impact other google products?
• INTERNAL
  • APP UPDATE: Did we update the app and introduce a bug?
    • Find latest app update version date and determine if that overlaps with the start date  drop in DAUs
  • FIRMWARE UPDATE: Did iOS or Android release a new version ?
    • Find out when firmware update date corresponds with DAU drop
  • MARKETING: Did we change / pause marketing for Google maps across any channel?
    • Collect information on marketing campaigns for Google maps and see if there has been any change spend / user acquisition / user engagement on channels
  • OUTAGES: Did we experience an outage in any part of the system / services powering google maps?
    • Look at logs and uptime of services Powerades
  • GOOGLE APPs: Did we make any changes to other google apps which were referring to Google maps / opening Google maps
  • METRICS: Was any update made to the metrics collection for define DAUs? 
    • Did we adopt a new way of counting DAUs or did we change our suite,
• EXTERNAL
  • EVENTS: Have there been any major events that have impacted our user base 
    • Natural disasters, Wars, Pandemics, Internet outage, Power grid failures
  • COMPETITOR: Did a competing product offering launch in the market? 
    • Competing product taking away marketshare from our product. 
  • PR: Did Google face a backlash in the press over its anti-completive behavior or some other public snafu
    • Users intentionally deleting the app to boycott the company

Actions and Communication

Based on the responses we would formulate and hypothesis around why this has happened. Thereafter, we can take steps to fix this issues based on what the issue is. For example, 

• If app bug - Push and app update and increase level of testing / QA. Do a post mortem
• If infrastructure failure - Work with SREs to identify ways to proactively alert of issues / logging metrics and ensure we build in redundancy in the product 
• If Google apps - Work with other PMs to ensure that they are aware of the drop due to updates to apps on their end. Work with them to find a feasible solutions and prepare some numbers on how the drop impacts the company overall and find a way to revert to the  old process
• And so on…

Based on what the issue I would prepare a post mortem with the relevant stakeholders’ inputs and share with internally with the right groups.

OK. Let's start with some clarifying questions and understanding:

Google Maps is a web mapping service. It offers satellite imagery, aerial photography, street maps, 360° interactive panoramic views of streets (Street View), real-time traffic conditions, and route planning for traveling by foot, car, bicycle, air (in beta), and public transportation.

In what timeframe did the report show a drop in 10% of daily active users? Is this within one day, week, a month?

Is this a onetime drop?: Assume one time

Is this worldwide or only in the US?: Assume Global

On what platform?: Assume on mobile apps only

Seasonal pattern? No

Specific user segment? No

A decline in other google products? Assume: No

Is there any AB testing going on? Assume: No 

Summarize the problem statement: Google Maps had a one time drop of 10% in daily active users worldwide on mobile platforms, globally.

We will look at external and internal factors that could impact the drop.

External Factors:

• A sudden global event that is causing people not to want to travel such as a WW Pandemic
  Next steps: Researching external sources that could give more information
• A new product/feature in a product that has been launched
  Next steps: Researching the competition and news externally but also internally such as Waiz 

• The way we track metrics may be broken?
  • Next steps: are logs collected correctly, has anything changed how we collect metrics?
• HW / SW issues?
  • Next steps: Checking for outages in the datacenter / infrastructure upgrades / software upgrades / platform upgrades
• New Feature / Bugs / UI rolled out?
  • Next steps: checking with other PM's if any new features have been rolled out? Checking with QA on bugs. Checking if any A/B testing is going on?
• Changing user behavior?
  • Next steps: Checking with data if there is a change in user behavior apparent. Checking with news sources for possible explanations of user behavior if there is an indication for that.