How would you improve image search?

I would first start with the clarifying questions:

1. Are we talking about a specific product? Since this question was asked at Google, it's a safe bet to assume we are talking about Google Image Search but I think it's always recommended to verbalize this explicitly. We could just as well be talking about image search on a social media platform like Facebook. Let's say it indeed is Google Search that we are talking of. 

2. Next, I would like to know what do we mean by improvement? Do we mean to improve engagement i.e. we want more people to use image search (increasing usage) or do we want to improve the quality of suggestions (so that users find relevant results quickly)? We can also look at improving a specific metric such as the revenue collected due to image search (through the ads it displays on image results). For this question, let's say we wish to increase usage.

3. Next, we can clarify if we are looking at improving the product for a specific platform such as mobile device/desktop or we are open to suggesting improvements that are platform agnostic. A platform-independent solution will have a wider reach. However, some reasons for selecting a specific platform such as mobile could be access to specific hardware (such as the light sensor, accelerometer, etc.). These can act as additional data points and can help us in providing more contextual search results (if the accelerometer reading tells us that the user is traveling then we can include some nearby tourist spot suggestions among the top results if the user searched for a specific monument that is close to his/her current location). If left to me, I would say I wouldn't want to commit to this decision yet as it can vary depending on the user persona and the specific need I am working on.

Let's say after this, our problem statement is narrowed down to improving usage of Google Image Search with no specific constraints on the platform targeted. Next, we will proceed with defining the customer personas. Some user personas that I could identify are:

1. Academicians/University students & Researchers: They would like to search for images with a specific context and containing specific keywords. For ex. as part of my research on the carbon cycle, I would like to see more flowcharts/structural diagrams containing all/some of the keywords such as carbon cycle, grey atmosphere model, biogeochemical cycle, etc. Solutions for this space could be platform-independent.

2. Travellers: These are young tech savvy individuals who are interested in looking for more relevant places to visit, events to attend, and local cuisine to try out. One need here is to be able to know more about specific spots at their location so that the traveling experience is enhanced and dependency on physical agents is reduced. Solutions suggested here could be better targeted just to mobile phones as travelers may not be engaging in image search on laptop/desktop while traveling.

3. Users who buy/sell used products: As buyers, they want to be assured that the product is of decent quality, and sellers which to get the best price that befits the condition of their product. Certain product categories such as paintings, clothes, bags, etc. can be assessed to a fairly accurate level via images. Hence identification of a specific product and subsequent assessment/valuation is an opportunity here. Solutions for this space will also be platform-independent.

Out of the above 3 personas, I would like to proceed with travelers. As a search engine, Google already has a wealth of information to be able to cater to this segment effectively. It also presents an opportunity for cross-selling (we can recommend specific restaurants/provide an option to book taxi service/allow them to snap photos to get more information and simultaneously give them the option to upload to google photos etc.)

Next, I will list some solutions to cater to the pain points of this user group:

1. Image-based itinerary: With this feature, the user clicks the photo of a particular spot/mural let's say the fountains in front of Taj Mahal after which it suggests brief info about the image such as history, information like times during the day when it operates, other places nearby that are known to have fountains, etc. There will also be the option for the user to listen to this content instead of only reading it.

2. Intent-Based Recommendations: Once the user snaps a photo and does a search, we can use ML frameworks to determine the mood/exhaustion level/type of event (birthday/trekking, etc.) from the photo and customize the results returned. For ex. if we can infer from the photo that the users are showing signs of exhaustion on a sunny afternoon, we can show recommendations of nearby beaches/spa along with the estimated time of arrival and the footfall at those locations (with an option to book a cab via Ola/Uber of course).

3. Style Assistant: Based on the image searched, we can leverage the photos that other users have taken at the same spot and make recommendations to the user regarding their style statements. Maybe people with fair colored skin who wore a black dress at a particular spot got more comments/likes and so we can suggest nearby shops where user can buy the black dress. This feature however will work more effectively if we can integrate with the Facebook profile of users (now that Google+ is discontinued).

I will decide priority for the above solutions based on evaluation along 3 dimensions (Time to develop, Customer satisfaction, Driving Usage of other google products):

                                                                   Time to Develop             Customer Satisfaction          Driving                                                                                                                                                                    Usage of other                                                                                                                                                            Google Services

Image Based Itenary                                     Low                                      Medium                              Low

Intent Based Recommendations                Medium                                 High                                   Medium

Style Assistant                                                High                                     Medium                             Medium

The image-Based itinerary could make use of the Google Earth itinerary feature (if the image being searched is cataloged in Google Earth's repository). Intent-Based recommendations will not only make use of google maps to suggest footfall at nearby places but also make use of google business to suggest places to visit. Also, suggestions for ola/uber may contribute to revenue. Style assistant also makes use of Google maps and google business to suggest nearby shops to buy style accessories.

Based on the above prioritization, Intent-Based Recommendations will be priority 1. 

Style Assistant will take more time as it will require integration with Facebook Profiles of users to determine the popularity of a photo. Thus establishing a base from which to build a model for style suggestion will be very time-intensive.

To determine if this is successful, we can look at the clickthrough rate of users on the recommendations shown. I suggest this feature to be rolled out just for mobile phones so that based on mobile sensor readings we can determine if the user is currently traveling and thus recommendations are shown only to such users (so that only relevant users are targeted)

In summary, we looked at how to drive usage of google image search across all platforms. We identified travelers as a target segment due to cross selling opportunities it presents and the lucrative knowledge based we can leverage. We concluded that intent-based recommendations suggesting users places to visit next and providing information such as footfall/means to get there should be the feature we should provide first.