What metrics would you use to measure success in designing the estimated time of arrival?

I am making an assumption that we are Uber.

Uber’s mission is to make movement simpler and affordable for everyone. To that point all experience designed to provide transparency and monitoring Is going to further its mission. 

I want to take a moment and understand what the North Star metric for Uber Riders are.  I believe it would be the number of rides completed where all parties provided five star reviews and no refund was requested or no complaint was registered.

now coming to estimated time of arrival, If I were to understand how estimated time of arrival please into that experience we could look at it two ways the customers require requesting a ride can get to see when the ride is going to arrive recording me if there are other airport or at home they could come to the pickup location in time for better yet they could plan to meet the rider meet the driver at the expected time. from a driver's standpoint having a good estimated time of arrival give some cues to look at if the customer is there can I find the house cuz if I'm at the end of my estimated time before I will I know when to start looking for my passenger. while this is particularly very very important at an airport it's also very important at residential or urban area pickups.

next let's define how estimated time before I will fit into the overall Journey map.  from 

1. a customer standpoint the Journey Begins when 

   1. I identify need to go somewhere 

   2. I open up the Uber app 

   3. I provided destination and then ask for a pickup.  

   4. next I got a confirmation once a driver selects what chooses to accept the ride and the fact that you know my ride is 15 minutes away.  

   5. post that I get to see 

      1. on the map itself the driver's coming through 

      2. and the estimated time of arrival of the driver 

   6. next I meet the driver and begin the ride. 

2. From a driver standpoint

   1. The driver clicks go to suggest he is ready to accept rides

   2.  the driver gets a notification off the ride

   3.  the driver accepts the ride

   4.  the system provides the navigational map  and an ETA where to pick up the customer

   5. The driver looks at the ETA choose to accept the system provided map or chooses one of his own

   6.  the driver picks up the customer  and then proceeds to the destination

   7.  the driver and the customer then look at the ETA of arriving at the destination

   8. Along the way if the ETA to the destination is less and the driver gets a request then he choose to accept a back-to-back ride

For the purposes of this exercise I like to scope out this problem into steps 4 through 6 for the customer, and steps .  so let's understand the metrics at each stage

1. Customer

   1. Confirmation

      1. No of rides booked

   2. Tracking the ETA

      1. Usually no one pays attention to the ETA if the driver arrives on time however if the ETA is showing arrived in the driver is not yet there and the customer would usually text the driver or cancel the ride.  so there are two metrics here we can track

         1. number of cancellation after the booking is confirmed

         2. number of contacts to the driver

   3. During or after the ride

      1. If the ETA was wrong, And the customer was really unhappy The review scores might be low.  so the metric here we can check for is 

      2. Number of rides where review is less than five stars and ETA for driver pickup is greater than 20 minutes.

2. Driver

   1. Most of the metrics mentioned above would be still be applicable for the driver,  with a few additions

   2. If that is a mismatch between the plan ETA and the actual time the driver may end up canceling a ride so the metric we can track for here is number of cancellations by the driver assuming the driver can still cancel

Next we want to prioritize these metrics And I want to measure it's correlation with the North Star metric which is the number of rides completed with a five star review.  so let's look at each one of them

1. number of rides booked  - There is a strong correlation, but it has got nothing to do with ETA.  there may be a way to understand if a high ETA has resulted in  a cancellation.  but we do not have a way to measure that yet

2.  number of cancellation after the booking is confirmed -  there is a strong correlation to the North Star,  and it depends on ETA as well

3.  number of contacts to the driver -  there is a strong correlation to the North Star  and it depends on ETA but there are some other factors too.  so maybe we could trim this down to measure number of contacts to the driver that are ETA related.

4. Number of rides for reviews less than 5 stars and ETA for driver pickup is greater than 20 minutes -  this has a very strong correlation to the North Star.

5. Number of driver that cancellations -  while this is important cannot be wholly attributed to ETA. 

 so in summary I would like to prioritize number four which is number of rides with reviews less than five stars and ETA for driver pickup is greater than 20 minutes