As a Product manager at a food delivery app, you observe a downward trend in number of orders fulfilled. How would you identify the root cause??

• Assuming an app like Uber eats

• Is the drop in # of orders received or fulfilled? Received

• Sudden or gradual? Gradual over the week (-5%)

• Everywhere or a single city? Single big city

• 3 way marketplace: Drivers, Restaurants, customers

Lets look at some of the internal and external factors that might be responsible:

Internal factors:

• App downtime? Bugs? Slowness?

• A/b test?

• Major changes in App? Sign in issues?

External Factors:

• Long weekend or holiday season? Restaurants might be closed

• Competitors gaining share?

• Bad PR?

Since this is a 3 way market, lets look at the major factors which affect the # of orders and build hypothesis:

• Registered restaurants or their availability to take orders (total available restaurant hours)

• # of drivers and their total hours

• # of DAU/MAU 

• Average wait time for order

Hypothesis 1: There might be a drop in no of restaurants or their availability for takeaway 

• Will check for no of registered restaurants and how long they were available to take take-away orders. 

• Will also check if their prep time has increased. Any major increases might lead to less orders

• Will specifically for our biggest restaurants if there is any major changes in # of orders they received

Hypothesis 2: There is less availability of drivers

• Check of # of available drivers available last week(# of drivers and clocked hours) 

• If we see a drop, check if a competitor is hiring drivers or if there was a major festival/holiday

Hypothesis3: Less users using the app 

• Will check for the DAU for the app. If there is a drop, it might indicate overall adoption going down. Reasons can be long wait times or unavailability of favorite restaurants

• If DAU hasnt decreased, will check for decreases in restaurant viewed per users. Any drop might again indicate lesser availability of favorite restaurants

Segmentation:

We can also segment the problem in different ways and look for root cause

1. Time of the day: Check if the drop is coming from specific hours or specific days. And check for the major factors listed above during that time period

2. # of orders served per day by restaurants: Bin restaurants by the # of order they serve and look for drop starting with the biggest restaurants. Its possible handful of restaurants are responsible for the drop

The framework I would use to answer this problem solving question.

Assuming 

• by food delivery app you mean uber foods or postmates 
• Im the PM for the food delivery app 

Observations: the drop mentioned is in order fulfilled, by fulfilled we mean order that were delivered to the customer.

questions:

What is the quantum of the drop? assuming 5%

Over what time - it says downward trend so assuming its over a week or more 

It doesn't say whether fulfilled rate - is fulfillment rate ie fulfilled/orders placed down as well? assuming yes since orders placed would have raised an alarm soon 

Approach  outline:

- I will first validate the drop

- Confirm if any other metrics are impacted 

- Isolate drop to either 1) Internal or 2) External suspects

drop confirmation 

- is the data complete/is there any data lag or corruption

- are the instrumentation events firing correctly? are they being captured correctly

- are there any other metrics affected? order fulfilled dropping would mean that there should be decrease in bottom line as  few orders getting fulfilled means higher cancellation rate 

other metrics:

- is there a spike in cancellations? where is the spike,  is it by customer or auto cancelled

- Is there an increase in  order return rate?

Assuming there is an increase in cancellation - check cancellation source and reason split, if there's no change there then spot check open ended comments entered as cancellation reasons

assuming at an aggregate level it'll be difficult to check above trends as they will get diluted 

Will continue to check the above paired with following cuts

• App version
• Platform
• Geography 
• Orders split by type of payment mode 
• Delivery time distribution 
• Breach rate
• Were there any major backend changes during the period that affect the fulfillment flow

is there any major external event that could explain the cancellation increase like snow storm/extreme weather that could affect the above across major regions?

I should be able to identify the root cause basis the above investigation