How do you measure the success of Airbnb online experiences?

Context

Great, first off let's just spend a moment getting on the same page in terms of what Airbnb Online Experiences is and more importantly why users use it. 

During COVID, domestic and international travel slowed to a halt and in parallel Airbnb bookings tanked as well. As a response to his Airbnb launched Online Experiences which are digital experiences users can book on the platform. For example, a Tokyo based Sushi chef could host an online sushi cooking lesson that users could sign up for. Hosts are able to continue to share their culture and generate revenue while travelers are now still able to experience new cultural experiences while being relegated to the confines of their home.

Before we dive into the nitty gritty, let's first take a step back and make sure we understand the context surrounding the bigger picture. Airbnb's mission statement is to create a world where anyone can belong. Online Experiences helps further this mission statement by both allowing people to experience the world when there are travel restrictions in place and beyond that by enabling those who can't afford travel to still experience other cultures. For example, not e veryone can afford a $3000 trip to Toyko but a $50 online sushi lesson is going to be a lot more accessible.

Users

We touched upon it above but it's worth calling out that there are two distinct user groups here. The hosts who are creating and putting on the Online Events and the "Online Travelers" who are attending these events. Both are critical to the success of Airbnb Online Events.

Metrics

Success and its definition varies from product to product and a big component of that is which stage of the product lifecycle that product is at. 

For a recently launched product, like Airbnb Online Experiences, I would first want to address whether or not we've found product-market fit. Are we able to effectively address the needs of our users? More specifically, are we able to help Hosts make more through putting on digital experiences and are Online Travelers able to enjoyably experience other parts of the world from the comfort of their home.

The best way to measure product-market fit is through a combination of engagement and retention. If users are meaningfully engaging with the product then we're adding value to them. If they come back and engage again then we've added enough value that they think it's worth coming back to. 

We could measure success and our product-market fit for Airbnb Online Experiences by examining the following metrics:

Engagement:

• Total number of event attendees 
• Average number of attendees per event
• Total Number of Events
• Average number of events per Host
• Average ticket price

Retention:

• % of hosts who hosted 2 events last month who also hosted 2 events this month
• % of online travelers who attended an event last month who also attended and event this month
• Average event review rating

• While intentional, all of our above metrics are fairly high level. They don't give us much granularity in terms of how different different segments may behave differently. For example, maybe cooking events do really well or people with kids don't enjoy Online Experiences.
• While we're focused on the success of Online Experiences we want to make sure we aren't negatively impacting other parts of the Airbnb platform For example, if people are enjoying the Online Experiences so much they may not be as compelled to book actual travel and lodging through the platform.

1. Describe Product - AirBnB is a marketplace platform that connects short term rental (vacation home) providers with consumers. 

   1. People (host) can list their property as vacation property and make it available for rent. They can add photos, property description, nearby attractions, convenience factors (close to station, grocery) etc. Sign ups and listing on airbnb for the host

   2. People (guests) who are looking to book a vacation home can go to airbnb, search for the properties that meet their criteria, select the property they like, reserve it and make payment. It also allows guests to change or cancel their bookings

   3. Airbnb charges a fees from the listing party anytime a property gets booked (~3% of total booking)

   4. Airbnb also offers services like cleanup after checkout to make it easy for people to let out their homes 

Online experience for host or guest? Lets say we want to focus on the host. Host can list a property of experience. I will focus on host who is listing an experience

2. Goal of Airbnb - mission is to create a travel platform that will enable every part of your trip. Goal of AirBnB is well known in vacation home rentals, not sure how well they are established in the market of selling experiences. Goal of AirBnB should be to drive growth in this segment which will allow them to diversify their revenue streams. I will focus on that. 

3. User Journey - host listing a property 

   1. Sign up with AirBnB and create a profile (included bank details where money will be transferred for successful stays)

   2. Create a listing - Gather property details including photos, description, nearby attractions, convenience factors, etc. determine price per day, create a listing 

   3. Receive a booking, accept/reject 

   4. Manage checkin - key availability, customer support during stay ,etc

   5. Cleanup property after checkout and get ready for the next booking request

4. Metrics

   1. # of sign ups per month, # of users who signed up and completed profile 

   2. # of active hosts, # of new listings per month, % of hosts who created a listing after signup, avg number of listings per host, % of listings activated vs created

   3. Conversion rate (# of listings clicked vs booked), avg price per booking, avg # of days per stay, avg # of days the listing stayed on the marketplace before getting booked

   4. % of bookings canceled, % of bookings modified, avg price per canceled booking

   5. This is the cost to the host, may not be tracked by AirBnB unless host is leveraging services from AirBnB to maintain the property. AirBnB can track metrics around guest satisfaction like avg customer rating 

5. Prioritize metrics - Host will have a good experience if they are able to successfully list the property, book it fast, not have too many cancelations and receive a high guest rating. So key metrics AirBnB would track to measure  success of online experiences of hosts would be 

   1. % of hosts who created a listing after sign up

   2. % of listings activated vs created 

   3. Avg # of listings per host

   4. Conversion rate per listing (clicked vs booked)