How do you measure the success of Google Search?

Question

How do you measure the success of Google Search?

Asked at Google over a year ago

2.9k views

Asked at

2.9k views

How to answer Metrics questions

Interview Guide

Answers (3)

To answer this question, Log in.

Top Metrics interview questions

What metrics would you use to measure the success of the Save feature at Facebook? 39 answers | 87.2k views
How would you measure the success of Gmail?23 answers | 17k views
What metrics would you look at as a product manager for Instagram ads?19 answers | 23.8k views
See Metrics PM Interview Questions

Invite members
Invite by email

Add another

J_killjoy · Answer 1 · 2024-07-29T16:15:10+0000

Clarifications:

Which google search are we referring to ? web search engine/ gmail google search/ search capability within any of the g-suite products
Understanding of the product - Google search web is a search engine which allows users to search from a wide array of information available on the web. The search results can be texts like articles/news/blogs etc, image, video, websites, products etc. It is one of the most important products to google and is a testament to google’s mission of “organizing world’s data and make it universally accessible”. Google search engages and retains users by providing relevant search results through indexing and crawling, it monetizes users through ads in search result
What is the goal of google search currently? I would say currently google search has more than 90% of the market share and boasts more than 5 bn users worldwide. So acquisition is a non-goal. Ads monetization has been a robust business as well and i am assuming we are showing steady growth on that as well. So monetization is a non-goal. However, with the rise of LLMs and gen AI capabilities and the likes of chatgpt now offering search gpt as a competitor can be a serious threat to google. So, to continue to be relevant in the foreseeable future, I would choose “engagement” as the primary goal for google search where the product goal would be to not only provide relevant search result to users but create meaningful search experiences.
Also i am assuming the experience would be broadly identical over web and mobile so I am not prioritizing any one over the other but as majority of the searches must be happening through mobile, while thinking of the user journey i would like to keep a mobile first experience in mind
I would like to breakdown this problem into below parts:
- Will think about User segments
- Will layout the User journey of google search for a prioritized segment
- List Metrics corresponding to different actions in the user journey
- define the NSM and other sub-metrics

User segments can be based on many segmentation cuts but i would like to categorize them basis their usage frequency : power users (using >10 times a week); active user (using 5-10 times a week); Passive user (using <5 times a week)

- of the 5bn active users of google search, assuming a simple pareto, power users would be around 20% of 5b which is 1bn users. I would like to focus on the power users segment as this segment engages the most with google search and we need to think of continuously engaging them so as to not to lose out to searchgpt or the likes.

Lets see how the power user carries out google search

User journey	Actions	metrics
Realizes he needs to look up for an information online	Awareness and interest	NA
Opens mobile browser/ google search app on the phone	App open	Search app launches (we need to be only looking at organic launches and not launches as part of the default device settings)
Types in search query on search homepage	preparing search query	Average no. of queries per user (too high can indicate poor search relevancy)
Sees search suggestions or autofill suggestions	Selecting relevant search keyword or suggestion	CTR on search suggestions Recommendation efficiency (re-searches post CTR on suggestions) Autofill acceptance rate (% of autofills accepted over total autofills shown)
Clicks on search suggestions or hits enter on the search icon	Choosing what to search
Sees search results page	Viewing search results	Bounce rate % of re-searches
Clicks on search result to view further	Evaluating search result	CTR on top result (result relevancy) Depth of page scroll/ link visits
Or types again upon not finding relevant search results or clicks on did you mean? suggestion	Finetuning search or retrying	CTR on “did you mean?”
Takes follow up actions on the search results like reading a content or viewing a media	Exiting search	CTR on search results CTR on top results
repeats		D7 retention No. of sessions per user in a week

NSM - No. of weekly sessions per user per week; If this is going up then it indicates that users are using google search more often and is the primary search engine for users

Trade off - will be tricky to define session definition

vparam · Answer 2 · 2024-05-30T18:07:58+0000

Product Feature:
- Google search is to enable users to search online information by typing keywords that the users thinks is the appropriate words for their search. The search has now extended to support voice prompts and images based search. More recently Google search support through Duet an AI drive conversational assistant taht not only provides links but through prompts
Goal
- Success measure of Google Search
User journey
- User goes to Goolge search in browser
  - types keywords in search bar
  - types prompts to the conversational interface
- gets links that are associated to the keywords in the case of the search bar keywords
- gets summaries and associated links int he case of conversational prompts
Metrics
- Search bar
  - Engagement
    - # of times the users uses search
    - # of clicks the user clicks on the available links
    - # of ads the users clicks demonstrating the user interest in those relevant ads
- AI conversational search platform
  - Awareness
    - % of users taking advantage of concversational search feature
  - Acquistion
    - % of users who continuesly use the conversational platform post being aware of the feature
  - Engagement
    - frequency and durationt he users spend ont eh conversational platform
  - Monetization
    - % of the users who convert to paid duet feature based on the free conversational platform

SPD_2017 · Answer 3 · 2022-09-07T00:50:32+0000

Google search enables its users to use keywords to search for information online. User types keyword(s) in the search bar at Google.com or the google search app and relevant search results are returned within seconds.

Let me brainstorm a few ways Google search success can be measured in the form of success metrics.

Google's mission: To organize the world's information and make it universally accessible and useful in one click.

User Journey:

The user seeks information
user type keywords and see search result.
User finds relevant information in on the first page of search result

Metrics:

number of clicks per topic user is searching for
% breakdown for the page number where user found relevant information per search result page, per URL by PageRank
Avg time user spent using Google search per topic
% ads user clicked while searching for the relevant information- although no direct monetization, there is a loop that can be activated by Google search
1. secondary metrics: % breakdown for users with Google account and users without Google account
2. Fake URLs, tracking whether fake URLs are landing in the pagerank algorithm's higher ranking or successfully filtered out

To summarize, I would like to track how effective Google Search's page rank algorithm is by measuring #2 since this shows whether the information user is seeking is retrieved quickly.

4.2 is a secondary metric that should always be within a minimal range, slight increase in this value might indicate issue where scammers are gaming the algorithm.