Compare two search algorithms by defining the North Star KPI. Measuring that KPI through 3 metrics and identify success flags.

Clarifying questions:

1. What is this search for i.e., website or mobile? (Answer: Doesn't matter, we have the same search everywhere)

2. Which type of search is it Text, Voice, or Picture? (Answer: It is a Text search)

3. What domain are we talking about? Whether the search is for an E-commerce platform or Google or something else? (Answer: Agoda's search functionality)

4. What is the search journey? (Answer: The below journey looks correct)

Types > View suggestions > Click suggestion > SRP 

5. What are the two search algorithms that we have? How is one different than the other? How do they work? (Answer: We don't want to get into the technicalities of how they work, we want to compare between the two of them.)

6. I'd like to understand what is the purpose of this search, and where is the search functionality present, on the Agoda app we have different searches present for Hotels, Flights, Homes, and Apartments. (Answer: Search functionality for hotel, to keep this discussion short)

7. What is the objective of this search? (Answer: Provide users with relevant suggestions but the main objective is to drive conversion)

Fine, going ahead. 

Agoda Hotels search feature allows users to look for a hotel based on the location (city, locality), landmark, or property name. These fields are system-defined or pre-defined fields from which users have to select one of the suggestions. This allows users to mistake-proof as they look for a property. 

Check with the interviewer

Funnel under consideration:

User types ---x% -> User see suggestions --y%-> User clicks on a suggestion --z%--> Goes to SRP

The important KPI would be traffic going to the search result page

For algorithm A: x%*y%*z% 

For algorithm B: X%*Y%*Z%

Metrics to track relevancy, correctness, and time taken for search results to get displayed

1. For relevancy:

• Users don't hit search often in a single session with similar or the same keywords. This could mean that users are not able to find relevant results with search.
• No. of times the same/similar keywords searched in a session

• I'd want users to engage on SRP, basically doing actions like sorting, filtering, browsing, and checking hotel properties. If users are not doing that after the search, then it could mean that the results displayed are not correct or maybe no results at all (or maybe an instrumentation issue, either way, I need to check this).
• Avg. time spent per user on SRP in a session
• No. of active users on SRP in a session (Active users = Users who do actions like filter, search, etc.,)

• I'd want to present users with relevant search result suggestions when they enter a character, the latency should be low here
• The time is taken to reach SRP once the user clicks on suggestions, we'd want to present results to the users as soon as possible.
• A check against timeouts has to be measured here
• Metric 1.1: Avg. time taken per user per session to load suggestion as character is typed
• Metric 1.2: Avg. time taken per user per session to view SRP once search results are clicked