Amazon is developing a machine learning algorithm to better target shoppers who have saved items in their shopping carts. The algorithm will send an email to users to remind them about their shopping carts. What factors would you consider when designing this algorithm? What variables would you tweak to yield higher shopping cart purchase rates?

Question

Amazon is developing a machine learning algorithm to better target shoppers who have saved items in their shopping carts. The algorithm will send an email to users to remind them about their shopping carts. What factors would you consider when designing this algorithm? What variables would you tweak to yield higher shopping cart purchase rates?

Asked at Amazon over a year ago

2.8k views

Asked at

2.8k views

How to answer Technical questions

Interview Guide

Top Technical interview questions

Imagine you're the product manager for Facebook Marketplace. Since many sellers don't mark items as sold, what existing functionality and metrics could you use to determine whether an item has likely sold?7 answers | 20.9k views
What happens when you enter a URL in your browser?6 answers | 10.8k views
How would you determine how to rank posts in the newsfeed?4 answers | 3.3k views
See Technical PM Interview Questions

Invite members
Invite by email

Add another

hjc · Answer 1 · 2020-09-03T00:17:13+0000

An amazon technical question like this requires clarification before moving forward.

Clarify

Can we define "Shopping Cart Purchase Rate"? [The ratio of shopping cart that resulted in completed purchase since creation]
Is the final goal to improve the revenue? [Yes]
How often can we send email to the same user? Only once or once per fix period? [Once per fixed period]

Users

There are a few personas who might leave items in shopping cart:

[Deciding shopper]: who is still comparing several options, typically they have multiple items in the shopping cart, with similar price band and purpose.
[Price shopper]: who knows what to buy, but would like to check price in other stores before order
[Interrupted shopper]: who shops regularly on Amazon, has multiple different items in the cart, but got interrupted before placing the order.
[New shopper]: who is new to Amazon, probably doesn't have Prime, probably hasn't ordered anything in the past

For optimizing revenue, I probably wanna focus on Deciding shopper, Price shopper and New shoppers.

There are a few potential outcomes after user receives the email:

Complete purchase from the cart
Click on the email link
Ignore the email
Add the email to SPAM list
Stop using Amazon as they hate email

We hope to achieve 1 or 2, but hope to avoid 3, 4 and 5. 4&5 might eventually cause the user to churn.

Solution

we will design a machine learning algorithm to predict their probility to click on the URL.

There are a few factors that will influence this probability, which we can use as feature to the ML model.

Whether the user is prime member
Whether the user has purchase record on Amazon
How many orders has the user placed in history
How long has the user been with Amazon
How long has the shopping cart been non-empty
How many items do they have in the shopping cart
Does the shopping cart contain competing goods
How many times has the user been reminded of the same shopping cart

When preparing training data, if an user clicked on the email link, the sample will be labeled 1; if an user didn't click on the email link, the sample will be labelled 0.

To yield higher shopping cart purchase rate, I can tweak the decision threshold of the machine learning model and tune through A/B testing. In reality, I probably will compare overall revenue, click rate, purchase rate and 14/30 day retention, to make sure we are not only optimizing the purchase rate, but damaging user retention and overall revenue.

KN · Answer 2 · 2020-06-05T01:12:08+0000

As Amazon wants to target users who have abandoned their shopping carts, it is most likely to wanting to increase the Revenue.

So, as far as Users are concerned, they are of course people who have abandoned their shopping carts. I will group them basis the amount of goods in the cart. If it is above a certain threshold( to be decided by Amazon), i wil focus on them and not otherwise.

So, next we have to see the pain points. Why are users abandoning their shopping carts?

1. Prices were too high

2. Prices changed during checkout

3. Ratings were too low

4. Competition products were better priced or better rated

5. The Payment failed

6. The customer forgot

So, my ML algorithm will focus on the below factors

1. Prices of goods in the cart

2. Price fluctuation if any

3. Ratings of goods in the cart

4. Competition prices and goods of same/similar products

5. PG data

First of all, i will see if the prices of goods have changed in the last 24 hours. I will also see what is the average price of the goods in the shopping cart is versus a similar rated/brand good in my inventory in the same category.

I will collect Ratings data for all the goods in the shopping carts of the users.

Next, i will run a web crawler to see what is the average price and average ratings of similar goods in my competiiton websites.

I will also study the PG data to see how many payments failed and what were the reasons behind it.

Combining all this data, i will try to predict what could have caused the user to abandon the cart.

Using Clustering algorithms, i will cluster the user into Reasons bucket.

First of all, i will look at PG data. If there was a tech glitch which i have solved, that will be part of PG bucket. I will email the user of Retrying the payment as before or with a different mode if the problem was with an instrument.

In other cases, i will study the Price data.

If there was a price fluctuation, i will lower the price to see if it is leading to better conversions.

If the product was high priced or competition products were low priced, i will suggest next best alternative.

I will study the ratings data and suggest higher rated products on my platform.

So, i will group the customer on PG, Price, Ratings Bucket and have a custom message for each of the groups.

I will keep tracking my conversions to see how well my ML Algo is working. My north star metric would be No. of Re conversions/ Total No. of abandoned carts

I will also see if the price experimentation is not lowering my conversions as a whole in which i will go back to the earlier price levels.

Ujjwald · Answer 3 · 2024-07-11T14:56:54+0000

Step 1 - Clarifications

I have a few clarifications and assumptions on this.

Can I assume that this algorithm will target all shoppers who have saved items in their cart, and not a subset of that?

Interviewer - That is upto you.

Are we looking only at emails or at in app notifications too

Interviewer - For now, let's look at emails.

I will assume that this algorithm is designed specifically for people adding items to the cart and does not include people adding items to the wishlist.

Interviewer - That is correct.

I assume this across all channels (mobile vs web) and across platforms (Android vs iOS)

Interviewer - That is correct.

Is there a specific geography that is being targeted here. I ask this because some geos are more price sensitive that the others.

Interviewer - This is for a global rollout

So, the goal seems to be to increase purchase rates by reducing cart abandonment rates across the globe.

Interviewer - That sounds about right.

Step 2 - Describe the product

The customer opens Amazon on the web or the app and adds some items to the cart. The customer then closes the app or the order doesn't pass to the next step of completing the payment. We will build an ML algorithm to target these shoppers and ensure that they finish the payment, as that will increase the average order value metrics of Amazon.

Step 3 - Personas

If we look at the customer persona, there are multiple types of customers we can look at

18-35 year olds
36-50 year olds
50+ years old

Let's look at the customer segment of 18-35 years old. If we look at the attributes of this segment, they are very busy people who are quite impatient and sensitive to cost and time. This particular segment also has a lot of impulse purchases (especially when they purchase something in the fashion vertical of Amazon).

Step 4 - Pain Point and Attribute Identification

Journey path to cart	Pain Points	Attributes
I liked something, so added it to the cart, instead of wishlist	I didn't know there was the concept of a wishlist	i. time in cart ii. Availability of item
Added multiple items to the cart but then forgot about it	I forgot I added the item to the cart	i. time in cart ii. Availability
Added items in the cart. I lost the connection but I didn't know about it	I didn't know that the process is stuck at the payment stage	i. order status
Added to the cart but decided against going ahead because of price/time as I want to get back later	The price is too high It takes too long to deliver	i. Price ii. Time
I added it to the cart but found a better deal elsewhere and hence haven't completed the purchase	The price is too high It takes too long to deliver Item not in stock	i. Price ii. Time iii. Availability
Item becomes unavailable by the time I add it to the cart	Item becomes unavailable Item becomes more expensive	i. Availability ii. Price

Therefore, the key attributes that we have identified are :

Price of the item
Time to deliver
Availability of item
Order status of the item
Time items spends in the cart

The algorithm can take the above attributes into account before sending the email about completing the purchase on their abandoned carts.

Some of the tweaks that can be done are :

If there is a drop in the price and the item is still in the cart, you send that to the user, saying that there is a drop in the price and you can complete the purchase now.
If an item in the cart becomes available, you can notify the user via an email to complete the purchase from. the cart directly with one click so that the item can be delivered to them.
If the order status is still in a state apart from ordered, you can send an email asking them to complete the payment so that the item can be shipped immediately.
Sometimes, if the item spends too much time in the cart, and the value of the item is more than the average purchase price of the user, you can point the user to alternate items in the price range they are comfortable in, so that the purchase will be completed.
In the email you can say that there are alternates which can be delivered sooner, so that they can modify the cart and complete the purchase.

If the item is a time sensitive item (like say, a PS5), then you can also tell the user to complete the transaction quickly as there is limited stock available.