15% off membership for Easter! Learn more. Close

Orders cancellation increased by 15%. How do you diagnose the root cause?

Asked at Stripe
5k views
eye 5k views eye 5k views
Answers (5)
crownAccess expert answers by becoming a member

You'll get access to over 3,000 product manager interview questions and answers

badge Gold PM

How to answer this problem solving question.

Steps

1. Clarify the Q

2. Brainstorm explanations

3. Evaluate probability of explanations

4. Sum & recommend most viable option & provide solution

1. Clarify the Q

Who/what are we talking about? Deliveroo (like a Seamless for Europe)

At what phase of making an order did users cancel? Customers completed an order (made the purchase) and then cancelled

Any specific city/region or urban/suburban/rural? Nope

Any specific user demographic (elderly, millenials, singles, families)? Nope

Any specific order size (small orders for 1-2 people, larger meals for 4-6, catered meals for 10+ people) Nope

A specific restaurant/cuisine? Nope

Specific platform; app or website? Both

Over what period of time? The past 2-3 months

What other metrics have changed, for example, has the total number of orders increased as well? Are users cancelling and then reordering at a different restaurant? Other metrics remained stagnant

2. Brainstorm explanations

1. COVID concerns creep in and make customers feel uneasy about having food cooked and delivered by strangers, how do they know that the chefs and deliverymen are safe and healthy? 

2. Customers decided to eat in/cook 

3. Customers decided to order something else (different restaurant/cuisine)

4. Customers decided that they are not hungry

5. Customers decided to eat at the restaurant (outside dining)

6. ETA is too long

7. Customers wanted to add/change something in their order, but don't see the button to do so, so they cancelled and start again

8. Something came up and they won't be home (they have to go to the hospital, a friend invited them to socially distance, they forgot they have an appointment)

3. Evaluate probability of explanations

1. 100% legitimate, but would they not realize this until after they complete the entire check-out process and make multiple decisions (browsing restaurants, cuisines and menus)?

2. Possible, but people who order in are often feeling lazy/busy and it might be unlikely to have a sudden mood change/open schedule that make them decide to eat in/cook.

3. Possible, but then the number of orders from Deliveroo would increase, unless they are ordering from a competitor or directly from the restaurant, but it's unlikely that the restaurant will have efficient delivery options.

4. Possible, but it's rare that people will go through the process of browsing restaurants, cuisines, and menus and make a purchase before realizing that they're not hungry. Looking a food pictures alone can make people feel hungry.

5. Possible, but people are likely taking precautionary measures and opting to stay home. It would be interesting to learn if the individual restaurants are experiencing an increase in outdoor diners or experiencing an increase in cancelled reservations.

6. Possible, but may still be quicker than cooking something themselves. Also, it's likely that Deliveroo shows the ETA before customers complete the purchase, so this may not be a viable option.

7. Unlikely, although I've personally never used Deliveroo, I'm confident an "change/add to order" button is located prominently on the app/site, because the more that customers purchase, the better it is for Deliveroo & their partnered restaurants. 

8. Possible, these things happen, but will it warrant a 15% increase in cancellations?

4. Sum & recommend most viable option

I'd check in with some restaurants that are experiencing many cancellations and inquire if outdoor dining is increasing to see if customers are choosing to eat there instead of at home (#cabin fever). 

Option 1 & 8 seem the most possible from the list. To mitigate these concerns moving forward, Deliveroo can highlight a few things on the order confirmation page including:

1. The safety precautions they are taking 

2. Thank the customer for their patronage during these times 

3. Emphasis how quickly their fresh, delicious and hot food will be delivered outside their front door. 

4. Provide sample activities to do while customers wait, (depending on the ETA) (examples: an episode of Friends, a 30 minute Peloton digital workout, a NYT crossword). 

This can delight users by making them feel safe and confident about their decision to use Deliveroo and may be less likely to cancel. 

 

 

 

Access expert answers by becoming a member
4 likes   |  
1 Feedback
badge Platinum PM

Overall the answer was a very nice take on the situation.

Things that you did well:

  • You followed the structure clearly and precisely just the way you mentioned upfront.
  • Covered a vast amount of situations.

Areas of Improvement:

  • Most of your hypothesis was based on customer behaviour. Hwever, there could be issues within the platform itself. Probably some bug in the recent update cancelling orders automatically or something similar. Mentioning something related to above shows how vast your thought process is.
  •  Providing Summary will also add value to your response. 
0
Get unlimited access for $12/month
Get access to 2,346 pm interview questions and answers to give yourself a strong edge against other candidates that are interviewing for the same position
Get access to over 238 hours of video material containing an interview prep course, recorded mock interviews by expert PMs, group practice sessions, and QAs with expert PMs
Boost your confidence in PM interviews by attending peer to peer mock interview practices, group practices, and QA sessions with expert PMs
badge Gold PM

 

Clarifying questions:

What do we sell? everything

At what phase of making an order did users cancel? Customers completed an order (made the purchase) and then cancelled

Any specific city/region? No

Any specific user demographic? No

Any specific order size? No

Specific platform; app or website? Both

Over what period of time? The past 2-3 months

Was this increase sudden? Or has it been occurring gradually? gradually

What other metrics have changed, for example, has the total number of orders increased as well? Are users cancelling and then reordering? Other metrics remained stagnant

 

We can interview some people who cancelled their orders to find the exact reasons, but before that my assumptions are:

-Something changed in the process and buyers see the shipping time after their payments

-something happened in the shipping process and we informed buyers that there would be a delay

-add/change button didn't work, so they have to cancel and reorder

-Our competitor accessed our data and offer a discount to our buyers

-there was a bug that cancelled the orders randomly

-we added a new feature to cancel and return the order after receiving it

 

Access expert answers by becoming a member
0 likes   |  
Get unlimited access for $12/month
Get access to 2,346 pm interview questions and answers to give yourself a strong edge against other candidates that are interviewing for the same position
Get access to over 238 hours of video material containing an interview prep course, recorded mock interviews by expert PMs, group practice sessions, and QAs with expert PMs
Boost your confidence in PM interviews by attending peer to peer mock interview practices, group practices, and QA sessions with expert PMs
badge Bronze PM
Clarifying questions

1) What kind of orders - suppose Ubereats

2)Have cancellations increased all of sudden to 15% or there was gradual increase - suppose sudden

First of all, I would like to check if the cancellations is coming for particular food chains or is it accross the board. Assuming its across the board.

1) Check if there has been any change made to UI which makes it confusing and users are prematurely placing orders in checkout path?

2)Analyze wait times for cancelled orders and overall orders to see if the wait time has increased or if the wait time was more than the estimated arrival time or if the estimated time was changed multiple times. This could lead to users getting frustrated and cancelling orders.

3) Check the feedback or help data to analyze if there are have been any recent complaints from user leading to order cancellation.

4) Did the order get accurately placed & accepted with the restaurant (may be check if the order api's are working fine? At times, the order might get placed with delivery services but the order might not be accepted by restaurants which could cause user frustration and cancelling orders.

Looking at all the options -#2 could be one of the probable reason for cancellations. I would give dicounted ubereats coupons to users experiencing delay meanwhile work on increasing the driver pool to ensure timely delivery.
Access expert answers by becoming a member
0 likes   |  
Get unlimited access for $12/month
Get access to 2,346 pm interview questions and answers to give yourself a strong edge against other candidates that are interviewing for the same position
Get access to over 238 hours of video material containing an interview prep course, recorded mock interviews by expert PMs, group practice sessions, and QAs with expert PMs
Boost your confidence in PM interviews by attending peer to peer mock interview practices, group practices, and QA sessions with expert PMs
badge Platinum PM
The way I would like to approach diagnosing the problem is better understand the problem. This will result in few questions and after that I will take a few momments to brainstorm areas that I would like to diganose. (cool).

Q: Order cancelations have gone up but for what business? (retail)

Q: What type of retail? I mean, groceries, other items you can get from grocery stores, clothing, shoes, etc.? (well pretty much everything since we are Super Target so we kinda sell pretty much all of what you mentioned).

Q: I want to undersatnd device type, OS, worlwide vs. local, if local specific areas, etc. So any of these an issue? (well it's regardless or device or OS, and we are in US so we focus on data for US and it's entire country overall).

Q: When you say order cancelations, do we mean people are placing the order and then canceling or people have added things to cart and then cancels or right before paying / actual checkout people are canceling? This is more of a funnel question but it goes towards understanding the problem. (good question. We see that people are canceling after paying).

Q: Has this dropped in specific time period? (yes, we have been noticing this since about a month). And is this what we generally notice every year or just this year? (just this year).

Me: So then we don't have seasonality at play here.

Q: Is this for specific segment of customers? If so, I honestly right now don't know what that might mean but just wondering. (no just about all types of customers).

Q: Ok you said all types of customers. Do we have any variance measurements amongst different demographics? I mean, maybe for certain age group of nationality we have low variances while for others we see high variances. (no).

Q: alright, I think I understand the problem. Now I would like to explore factors such as 1)changes (ui, some other chnges to our app or website, etc.), pricing strategy changes, logistical nightmares resulting in cancelations, news report, etc. I don;t know if we will get through all of these and maybe we would explroe additional areas but let's start with these for now. (sure. What do you want to take up first?)

Q: Well, let's start with some changes to our product. That's something we can immediately eliminate or hone onto sicne those are potentially easier to explore and take control of. (no changes).

Q: Ok, rolling back to a clarifying / confirmation quesiton for a moment if you don't mind. (sure). Well, are we sure our reporting is correct? (yes).

Q: Any changes to pricing strategy such as we now charge people to order online? (no we have always charged if order is less than x $ unless they pick it up at hte store). Ok, so that raises the question: is this cancelation noticed for pick up from store orders or ship to home orders? (I am glad you asked. Ship to home orders). And is this noticed for orders that are below the free shipping threshold $ amount? (no overall). Ok so no pricing strategy relevancy.

Q: Another deeper dive analysis is is the cancelaation happening after certain # of days of order? (yes). Alright, is it happening after order has been shipped? (yes). Ok, is this happening after certain # of days of shipment or as soon as customer is notified of shipment? (certain # of days of shipment). Ok, do we hve any order inquiry logs? (don't understand. Help me better understand and maynbe I can pull up). I would like to understand if people are first calling in to check on orders before canceling. + is this cancelation happening directly over website or via phone calls as well. (upon looking at data some are canceling over website directly and yes there is an increase in order inquiry calls). So is this order inquiry truely or are people calling to maybe eliminate items or add additional items to that same order? (well, there are less of eliminate or add since people understand order is in shipment). And do we know what specific of these orders are being canceled? I have a hunch that our delivery times have increased? (yes it seems that for all those orders that seem to be delayed or held up in transition seems to have higher levels of drops).

Me: Ok, to me seems that this is a key factor. Correct me if I am wrong but I don't think there is a whole lot we could do here except couple suggestions. But first I would like to tell you my analysis. It's likely that due to COIVD restrictions being lifted, more people are now canceling the order to save the shipment cost and going to the store (afterall restrictions were lifted just a week or two ago), also with more and more shipment increasing these days logistics is still backedup from before so that might be causing issues), there are also less employess everywehere since people want to live off of stimulus checks (I know this is the case for sure since it's in the news and my mom works at unemployment office of IL so she tells me the data) so things get backlogged in being shipped due to that from Target itself and also from logistics companies like UPS, USPS, FedEx, etc.

Interviwer: I think you are correct. this is our primary issue. What do you suggest we do?

Me: Yes to that I think here are couple options that we would want to evaluate. 1)Work with our logistics partners to provide estimate time of when the order will leave their facility and we can then dispaly that on our website for providing greater insight ot customer 2)Offer cupon and discount towards their current order if order is delayed 3) if order can be served locally, get it delivered by partnering with Uber, Lyft, Go Puff, etc.

Interviwer: Great! I like your ideas. These things, however, will take some time.

Me: Yes I understand. However, if we had more time I can spend some time thinking through some quick things.

Interviewer: No it's fine. I think you did well. Thank you for your time.
Access expert answers by becoming a member
0 likes   |  
Get unlimited access for $12/month
Get access to 2,346 pm interview questions and answers to give yourself a strong edge against other candidates that are interviewing for the same position
Get access to over 238 hours of video material containing an interview prep course, recorded mock interviews by expert PMs, group practice sessions, and QAs with expert PMs
Boost your confidence in PM interviews by attending peer to peer mock interview practices, group practices, and QA sessions with expert PMs
badge Silver PM
Not sure what Deliveroo is - assuming this is an ecommerce shop providing grocery or physical goods.ss 
 
Understanding Context
 
  1. Assumption - cancellation = customer never received the order
  2. Do we have more info on the definition of order cancellation - is it immediately post order? Or is there a time period after which the order is placed that we see cancellations going up? Is there a specific product or set of products that this is occurring with?
  3. Time period: Was this increase sudden? Or has it been occurring gradually? Was it the same this time last year? 
  4. Do we have an info on which user groups this has impacted? 
  5. Some other info/ metrics I’d want to check: 
    1. Typically there is a question when customer cancels order - so I’d want to review reasons. 
    2. Any new customer reviews 
    3. Average shipping times (any changes) 
    4. Whether the customer ordered something else in place 
    5. Is there any other competitor that has entered this space that is offering this product? 
Do you have any more info on any of these? 
 
Let’s say the order cancellations are due to increased shipping times. 
 
My hypothesis is that customers are losing interest in waiting for the product or finding the product elsewhere that can ship more quickly - which is leading to increased cancellations. 
 
First, I want to understand what’s causing the changes in shipping times. 
  1. Out of stock in a primary warehouse facility 
  2. Increase in orders that has led to back orders 
  3. Change in worker or system processes at the facility that has slowed speed 
  4. Change in process (ex: ship multiple things together) that has led to this change
 
Next Steps: 
  1. Work with the warehouse engineers to reduce shipping times
  2. Work with manufacturer/producer to find new sources or find alternative providers to fulfill orders
  3. Provide customers with incentives for waiting for order 

 
 
Access expert answers by becoming a member
0 likes   |  
Get unlimited access for $12/month
Get access to 2,346 pm interview questions and answers to give yourself a strong edge against other candidates that are interviewing for the same position
Get access to over 238 hours of video material containing an interview prep course, recorded mock interviews by expert PMs, group practice sessions, and QAs with expert PMs
Boost your confidence in PM interviews by attending peer to peer mock interview practices, group practices, and QA sessions with expert PMs
Get unlimited access for $12/month
Get access to 2,346 pm interview questions and answers to give yourself a strong edge against other candidates that are interviewing for the same position
Get access to over 238 hours of video material containing an interview prep course, recorded mock interviews by expert PMs, group practice sessions, and QAs with expert PMs
Boost your confidence in PM interviews by attending peer to peer mock interview practices, group practices, and QA sessions with expert PMs