17k views
17k views
17k views
Access expert answers by becoming a member
You'll get access to over 3,000 product manager interview questions and answers
Recommended by over 100k members
Bronze PM
Clarifying Qs:
Q: Do we have a specific business Goal that aligns with any new strategy of the company?
Q: Should we focus on a particular geographical location?
Q: Should we focus on a particular user segment say working professionals using GMail at a workplace?
Framework that we will use:
We will look at the mission of Google and where Gmail stands with respect to it. What purpose it serves for Google as a company, where Gmail is in its Growth stage. We will also look at the various segments of consumers using Gmail. Depending upon these we will zero in on a focused Customer Goal that has real impact in users' life and how they can get value in the best way possible. We will then look at some metrics - both primary and secondary.
Mission of Google and where Gmail stands:
Mission of Google is to organize all the world's information and make it accessible and useful for people. Gmail has been one of its watershed products in meeting google's mission wherein all the data in one's email communication is stored at one platform which can be categorized and searched for easily. It has always been a free product so Monetization has never been a goal; but the scale of people who have been onboarded on Gmail and eventually connected to Google is humungous and instrumental in Google's stragegy to pitch other products in their digital ecosystem. So it has not been a product giving direct revenue to Google, but an important product to acquire users for future products. Gmail is in a matured phase of its growth journey- it may also be seeing decline in its user base/usage (in terms of inactive users) because of prevalence of more direct forms of messaging chipping away the use of email as a whole product. Gmails revenue has been focused on selling Google Workspace subsciption which i think is in its growth phase.
Customer segments:
Gmails user base can be broadly classified into:
- Personal usage
- Professional usage
Refined Customer Goal: Given the prevalence of direct messaging apps, the personal usage of Gmail I belive has gone down. It is in the declining stage. So we will for now choose to focus on Professional usage. That area is where Google wants to monetize given the presence of Google Workspace. We will look at engagement to monetization phases and somewhat focus on Activation as second priority. Although the professional email phase is dominated by Microsoft, majority of these professionals do have a personal Gmail account so they know how to use it. Hence, acquisition (in terms of onboarding the user) we can de-priortize.
User journey for a Professional user:
- Sign up/ login
-Receive work emails
-Reply to them
-Categorize work emails into folders
- Mark an email as important
- Maintain a book of contacts
-Be able to connect with a contact over a quick message
-Be able to connect in a meeting with the team
- Search through old emails.
Metrics:
Activation:
A1: No. of organizations using Gmail as a professional account (with a workplace domain email id intergated with Google Workspaces)
A2: No. of users using Gmail as a professional account.
A3: No. of Gmail prof users who have at least sent a few messages (say 2-3 in a week's time). - Can ignore because in case of professional use, we would need to onboard a whole team/organization itself. If that is done, it is obvious that everyone on the team will use the same platform for emailing.
Engagement / Retention:
R1: DAU/WAU
R2: Total No. of emails received
R3: Total No. of emails sent & Avg no of emails sent per user.
R4: Total No. of Searches done
R5: Average no. of contacts stored per user
R6: Increase in no. of Gmail professional accounts Q-on-Q
R7: Average No. of meetings initiated per user
Monetization:
M1: Increase in the number of organizations Month-on-month who are onboarding for a professional account ecosystem
M2: ARPU/per organization
M3: CLTV
North Star Metric:
R3: Total no of emails sent & Avg no of emails sent per user.
Sending an email is a primary activity that a user does as compared to receiving one and doing a search. In the presence of multiple channels through which data can be sent or communication can be done, a user choosing Gmail for it will be a good win. Even if the organization is using Gmail as its default setup for commuication.
Avg no. emails sent = Total no of emails sent / Total no of users
These two would give us an idea that not just the volume of emails sent is increasing but users are also increasing.
Supporting Metrics: R1, R2, R4, R5, R6, R7, M1, M2, M3
Conclusion:
We looked at Gmails success for professional user as it is an expanding category as opposed to personal usage which is a declining category. We chose our metrics in such a way that measure whether a professional user is able to get maximum value. Hence we focused on engagement. This engagement should also translate into revenue so we looked at monetization also. We chose activation as a second priority goal.
PM
About Gmail
Its a google product where user can send and receive emails. The typical things user can do:
- Send/Receive emails
- Forward/reply emails
- Delete/Archive emails
- Starred/important email
- Categorise emails
- Can start chatting with email contact
- It can be for business or personal users
Clarify questions
Scope
- Are we considering personal users or business ?
- Personal
- Are we considering web or mobile app ?
- Mobile app
- Is it IOS/android?
- Both
- Are we considering chat as a part of success of gmail or just email?
- Only emails
Goal
As Google has penetrated really well in market, its more about engaging users with different products and gmail is also having this goal.
User journey
When user joins it for the first time
- Downloads the app
- Opens the app
- Signup and create an email account with @gmail
- Send an email
- Add attachement
- Receive an email
- Forward email
- Reply to email
- Star email
Recurring journey
- Opens the app via notification or organic
- Read an email
- Send an email
Actors
- User
- To be able to send or receive emails
- Average emails sent/week
- Week: because we don’t send emails everyday and daily data might fluctuate
- Replies
- Forwards
- Average emails read/day
- Average emails received/day
- Average emails sent/week
- To organize emails at one place
- #no. of emails starred/week
- #no. of emails marked important/week
- #no. of emails added to category/week
- To be able to send or receive emails
- Google
- To increase active users using any google app
- Daily active users
- Active: Read/sent any email
- Day 7, Day 14, Day 28 retentions
- (#users came back on day 7/#users used the feature on day 0)*100
- Daily active users
- To increase active users using any google app
North star: Daily active users: Intersection of user and business goal
L1:
- Retention rates (Day 7, 14, 28)
L2:
- Average emails read/day
- Average emails sent/week
Bronze PM
Some tasks that a user over Gmail can do are:
Send/Receive emails.
Delete/Archive emails.
Snooze or set a Reminder for emails.
Categorize, Organize and Prioritize emails using Tags/Folders.
Integrate with other Google products/services.
Send invites for a virtual meetup or meeting via integrating Google Calendar and Google Meet links in the email.
Send files(audio/image/video) attachments in the emails via Google Drive.
Chat with other Google users using text, voice and video via Google Chat.
Gmail has both Personal and Business users; We will focus only on Personal Users (General/Free GSuit Version).
Gmail has messaging chat features; We will focus only on email.
We will focus only on Existing users between Newly Acquired and Existing users.
For simplicity, we will be looking at Indian users and their User journeys on Web and App both (Android & iOS).
3. Goal
Acquisition — Google has reached good user penetration. Gmail being an entry-level product, may have excellent acquisition metrics.
Revenue — As we are looking at the free version of the product, revenue is indirectly through advertising. Also, there is a paid option to get more storage, but that does not apply to large sections of the user base.
Engagement and Retention — The most critical goal for Gmail is to keep Google’s users engaged so that they will stay with Google’s app and services ecosystem and reoccurring engagement shows Retention.
Thus, Engagement and Retention are essential for this product.
4. User JourneyDesktop
Acquisition
Visit mail.google.com and click on “Create account”.
Fill in personal details and set a strong password to sign up.
Send/Receive the first email.
Repeat the previous step multiple times (Loop).
Engagement and Retention
Open Gmail on the browser organically.
Send emails.
Read emails from the Primary Inbox.
Read emails from Social, Promotions, Updates, Forums, and other categories.
Reply to emails.
Other actions — forward, delete, archive, sort, snooze.
Close tab/browser.
Acquisition
Download the Gmail App from Play Store/App Store.
Sign up and create a Gmail account.
Send/Receive the first email.
Repeat the previous step multiple times (Loop).
Engagement and Retention
Open Gmail app Organically or via push notification.
Send emails.
Read emails from the Primary Inbox.
Read emails from Social, Promotions, Updates, Forums, and other categories.
Reply to emails.
Other actions — forward, delete, archive, sort, snooze.
Exit app.
5. How do we measure?Acquisition
Desktop
# of users who visited mail.google.com organically from the browser.
Mobile
# of users downloading Gmail app organically from Play Store/App Store.
# of users who opened the app to sign up / Total number of users who have the app pre-installed on the devices from the OS side.
Desktop
# of users that completed signup and created a Google ID on mail.google.com
% Conversion from Page Visits → Signup.
# of users who read at least one email.
# of users who sent at least one email.
Mobile
# of users that completed signup and created a Google ID on the app.
% Conversion from App open → Signup.
# of users who read at least one email.
# of users who sent at least one email.
# emails sent/read per session, per day, per month.
# of searches in the mailbox per user.
Avg Time Spent/Session Duration per day.
% of users who checked their inbox organically or via notifications on Day 1, Day 7, Day 15, Day 30.
# of days the user was active in 30 days.
# of users who read emails from Primary VS other categories (Social, Promotions, Updates, Forums, etc.)
% Split of total users who read emails from Primary and from different categories (Social, Promotions, Updates, Forums, etc.)
# of users who forward, delete, archive, sort, snooze their mails.
% Split of the total users who forward, delete, archive, sort, snooze their mails.
% of users who used other Google Services integrated inside Gmail.
# of other Google Services also used.
1 Day, 7 Day, 15 Day, 30 Day Retention rate.
DAU, MAU.
Amount spent in purchasing additional storage.
# invites sent.
% of signups from referral codes.
6. Evaluating and Prioritising the metrics:
To measure success, we can focus on the Retention and engagement metrics. Let’s evaluate the metrics based on impact, reach, relevance to goal and measurable metrics.
North Star:Retention Metrics.
Primary:# emails sent/read per session, per day, per month.
% of users who checked their inbox organically or via notifications on Day 1, Day 7, Day 15, Day 30.
Avg Time Spent/Session Duration per day.
# of users who read emails from Primary VS other categories (Social, Promotions, Updates, Forums, etc.)
# of users who forward, delete, archive, sort, snooze their mails.
# of days the user was active in 30 days.
7. Summary
We looked at measuring the success of Gmail (Desktop & mobile app) for users in India. In the process, we looked at the major sections in the user journey to understand that Retention and engagement metrics will eventually have a higher impact. Thus we narrowed down the North Start, Primary and Secondary metrics, which we will help track the success of Gmail.
Clarification
- Gmail has both consumer users and business users. Should I focus on consumer users here? [Sure]
- Users visit mail.google.com, or alternatively, powered up an Android phone, or downloaded the Gmail App.
- Users create a Google ID
- Users login into Gmail
- Users use various Gmail features; or move on to use other Google services, such as drive, calendar, hangout.
- Users generate purchases at services such as drive, youtube, Play, etc.
- Acquisition
- # PV to mail.google.com
- # downloads of Gmail
- Activation
- # users that creates ID at mail.google.com/Gmail App
- % Conversion rate from PV to Signup
- # users who log into Gmail
- % Conversion rate from signup to login
- # users who read at least 1 emails in Gmail
- # user who send at least 1 emails in Gmail
- Engagement and Retention
- DAU, MAU, 6-MAU
- Avg Face time or avg session time per day
- Avg App Swaps to front screen or avg sessions per day
- # emails sent/read per session, per day, per month
- # of Active days in 30 days
- 7 Day, 30 Day, 180 Day Retention rate
- # of other Google Services also used
- % Gmail user who also use other Google Services
- Average Account ages
- Churn rate, users who didn't log into gmail for over 30 days or 60 days.
- Monetization
- ARPU
- LTV
Metrics has similar limitations as metrics 1.
PM
1. Describe the feature
Gmail allows users to use their Google credentials to access a mail account with a good storage limit, free of visual ads and with a lot of convenient features. A user can visualize the received emails, answer them and send messages to friends through Gmail. Also, a user can send invitations integrated with google calendar (or other calendar services) and send files integrated with Google Drive. Some emails from flight companies and hotel reservations are seamlessly turned into actionable items inside Gmail to allow users to easily remember important events and quickly check in their reservations.
2. Determine the goal of the feature
An e-mail is a form of communication that is used by virtually every Internet user. Even for users that are more likely to communicate frequently via instant messages, they will eventually use their email account as a means of identification for a myriad of services.
The goal of Gmail is to retain Internet users as Google users by offering them a great email tool and integrating them into the Google Portfolio of products (Search, Ads, Docs, Drive, etc.)
3. Customer Journey
1) a user goes to the Gmail landing page and sing up for the email service
1.1) he creates an account informing his name, surname, phone number, address and, a strong password;
1.2) he validates his account utilizing a code that is sent to his phone
2) a user signs into his account to check for new messages in his inbox
3) a user opens new messages and eventually reply
4) he searches for old messages through the search box in the upper field
5) he deletes a useless email or reports it as spam if he thinks it wasn't solicited.
6) a user uses his Gmail address to signup into other internet services like Twitter, Facebook, news portals, etc.
4. Map and quantify user behavior
a) Awareness
Is every google user aware of Gmail?
Since other services like Whatsapp use a phone number as a means for signing up instead of email, is Gmail losing people's awareness?
Metric: Google could measure such awareness by comparing how many devices (throurgh IP identification) have loaded the google search page but haven't loaded the Gmail internal page in the last month. Measuring awareness in the universe of Google users would be pretty significant. Then, we will assume 70% of all Internet searches are done through Google.
For example, let us say we found 70% of the devices which loaded the google search page also loaded the Gmail page, then we have 70% awareness among google users, but 49% awareness among Internet users in general (not taking Cinese market into account).
b) Aquisition
Gmail acquisition has grown organically since its launch. Furthermore, over the years its usage has been promoted by the native install in most of the android distributions as a basic software. Since one needs an email account to use Google Play Store, users commonly use or create a Gmail account for that, even though they could use any email.
We can measure acquisition by
- # of users
- AOT (Acquisitio Over Time)
- # of total users who signed up in the last period of x days versus the total number of users
- x could represent a week, a month, a quarter, a 6 months period or a year.
- We would know the acquisition growth percentage by period.
c) Activation
Since users complete the signup process, they are ready to go and send some emails.
d) Engagement
We can measure engagement by the following metrics
- # of emails read per day per user
# of emails sent per day per user
# of searches in the mailbox per user
e) Retention
We can measure retention by
% of users who checked their inbox in the last 12 months, 6 months, 30 days, and 7 days, 24 hours. ]
f) Monetization
As Gmail is a free service, it doesn't drive direct revenue for google. Instead, it is part of the Gsuit for business and it consumes user space along with services like google drive. Users pay for storage in drive and, as tons of emails are sent each day, some users will eventually need more space. I would assume 15% of users who purchase additional space for google drive make it because of Gmail.
Then, a good metric for estimating revenue would be
- $ in purchases for additional google drive storage x 15%
g) Referral
As people may send invites to friends for gmail, we could monitor the number of invites sent and the percentage of signups from referral codes.
- # invites
- % of signups from referral codes.
CONCLUSION
One can say the role of Gmail as a service in Google's portfolio is keeping users into the Google environment, and also consider it as a mature product (being in the market for circa 20 years). Based on that, we can say it has been successful if:
- It is keeping its awareness stable along time;
- 50% of internet users.
- It has a stable or growing acquisition along the last year at least;
- AOT(2019) >= AOT(2018)
- It has moderate engagement (since people are more keener to instant messages these days);
- Engagement Metrics 2019 >= Engagement Metrics 2018
- Its retention metrics show stable or growing usage along the last year at least;
- Retention(2019) >= Retention(2018)
In case the conditions above are not satisfied, we can consider Gmail to be declining.
Platinum PM
Feature Description :
Gmail is the free email application used for people to connect via mails.It was launched in 2004.Since then it has made better day by day for users with features like multiple gmail accounts sign in a single tab,organizing emails more conveniently.
Goal
Gmail success goal is to drive more adoption,engagement and retention for the users.Assuming its to have more people on board and retain existing users .
Customer Journey :
- User sign up and create a gmail account.
- Sends/receives his first mail.
- Repeating step 2 frequently.
- Using features like priortising messages , deleting unwanted mails etc.
- No of new users creating gmail account WOW, MOM.
- No of users dropping off during sign up process.
- No of newly resgistred users able to send their first mail.
- No of newly resgistred users able to receive their first mail.
- Average no of mails sent by a user daily and weekly basis.
- No of times users logs in to check mails in a day (following notifications or on its own)
- Average no of mails read by a user daily and weekly basis.
- Average time spent in a session
- No of users coming back on daily and weekly basis.
- No of users who went inactive after a day ,week ,month or 45 days usage.
- Average NPS score for the app
- App ratings /reviews.
Silver PM
1. Describe Gmail product. The main function of gmail is to 1) send and 2) receive emails, with some additional features to support these 2 base functions as 1) delete email, 2) archive email, 3) set snooze/reminder of an email, 4) category emails into tags/folders, 5) integration with other G products (calendar, drive, etc.)
2. Product proposition. develop a tool for people to communicate (personal and business) via Internet.
3. user group: both business and personal consumer users
4. define success: users use gmail as a tool to communicate 1) they create account, 2) compose and send emails, 3) use email as their contact and receive emails, 4) they use it very often
5. user journey:
1) register account/sign up
2) send out first email
3) receive first email
4) send/receive more emails
5) use it everyday?evey couple of days of a week? couple days of months
6. Metrics:
1) # of sign ups in a week/month/year (depends how long the product is launched
2) # of emails sent/received in first week/month/year (if just launched)
3) # of users send out at least 1 email per week (WAU)
4) # of users receive at least 1 email per week (WAU)
5) opt out rate - how many active users stopped use it after how long (in 1 month, 3 month, 6 month, 12 months)
6) total space used per user account
7) could consider other side features as delete, category, or other G products widget but not main measurement here
I'm assuming we are talking about Gmail for Consumers. And that we define success based on engagement, not revenue or growth of users. Let's also assume this is platform agnostic.
When I look at the funnel for Gmail, the parts specific to engagement and retention are:
- Opening Gmail on web / mobile
- Organic open
- Open via push notification
- Sending emails
- Reading emails
- Primary Inbox
- Other categories (Promotions, Updates, Forums, etc.)
- Replying to emails
- Marking them as spam
- Other actions - archive, snooze, delete
- # of DAUs
- Sessions per user per time unit
- Time spent on Gmail
- CTRs by inbox category (primary should havev super high)
- # of emails sent per day per person
- % of emails read marked as spam
Gold PM
Describe Gmail - Platform to send/receive gmail, send chats, segment mails, create folders etc.
Users:
Students
Working Professionals
Elderly
User Journey:
Ops URL
Signs Up
Activates the email id
Sends/receives email
Metrics:
Acquisition:
#Sign Ups and trend
Funnel drop
Activation:
% signed up users who activate, trend
% users who activate within 24hrs, 3 days, >3 days
Engagement
No of emails sent/received per day/user
No of emails read per day/user
% emails unread/user
% users who are within 90% of allotted free memory or have exhausted
Retention
% users who return every day, every 3 day, 5 day to read emails
Monetization
% users who have bought additional memory
# users who subscribe to additional memory, trend over time
Bronze PM
Clarify: Are we talking about desktop or mobile? Overall. What do we want to focus on when meaning success? User acquisition, engagement, revenue? Engagement?
Gmail is a product that wants to make email experience as smooth and fast as possible and available to anyone in the world.
We can mainly put users into two categories:
Casual Users: Email is not the primary way of communication. Checks a couple of time a week. Once daily maybe
Power Users: Email is the primary way of communication. Check their inbox a couple of times a day.
What are some key flow
- Open Inbox
- Read emails
- Reply to emails
- Compose and send new emails
- Create draft emails
- Delete emails
- Search for emails
What are some key metrics we can track? I would say that Natural Usage Frequency is daily.
- Average number of emails a user read a day.
- Average number of emails a user send.
- Average number of replies by a user.
- Number of daily active users.
- Number of emails read.
- Number of emails composed.
- Number of emails replied.
- Average duration of composing an email.
- Average duration of loading the inbox.
- Average duration of loading an email.
- Average duration of loading an attachment.
- Number of users who are using shortcuts daily.
- Number of unread emails in the Primary Inbox.
- Number of searches in a day, and clicking an email following search.
- Average duration for an email to be sent.
Since we are focusing on the engagement and defining the goal of the product as making the email experience smooth and fast, I would focus more on the metrics that would be more relevant.
So the Average duration metrics are important.
Number of searches and short cuts would be important as well and I would keep an eye on Shortcuts.
The number of unread emails in the Inbox is important as well, helping us how well we are doing categorizing important emails.
PM
Describe Gmail - make sure we’re on the same page
Web or app? - as a whole
Any recent feature change we want to measure? No
User journey :
What can a user do on gmail?
Login
Create an email
Read an email (unopened or old or snoozed)
Send an email
Save draft
Archive / delete
Organize / add to calendar
Prioritize (Star, mark)
Group
Searches within emails
Describe google’s mission - To organize the world's data and make it accessible and useful.
Goal :
Key part of the ecosystem
Acquisition
Data (improve the product, ad)
Revenue
Productivity (Prioritization and filtering)
Quick recap at this point.
Metrics :
# of active gmail accounts (Number of sent/received)
# of signups (we can check this number directly and go further to check amount of space used v/s free)
classification - Auto-starring and mark as important - how many emails received v/s how many marked as important/categorized automatically
Filtering (spam and blockers - # of emails marked as spam by the user)
Evaluation :
We defined 4 key metrics that lie along our goal of improving productivity and user experience.
# of signups would not be useful standalone. (fake sign ups can lead to false positives)
# of active gmail accounts would be measured by emails read per day (Daily active users) - some folks may not send emails much so we can’t rely on that metric
# of starred emails would not necessarily be a strong metric. It hugely depends on the type of emails the user receives
# of spams are good to check since every user group receives some spam. Further along this line, good to check health of spam checker by seeing # of spams or archives by the user.
Bronze PM
Product description
Objective
User journey
AwarenessEvaluation
- Engagement - Percentage of emails not classified as Promotional read, Number of emails sent per week, Time spent using associated service, Time spent in Hangout calls, Time spent using associated Google services, Number of chat messages sent per week
- Retention - MAU, DAU, Session duration
- Monetization - Additional storage purchased per paying customer
Silver PM
Gmail provides an easy-to-use, efficient email client for users to manage their inbox. Gmail is a freemium model. The way Google captures value is from the enterprise accounts where companies use the Gmail app as their corporate email client. And Gmail app is also part of the overall Google apps package which includes other products like meet, calendar, office tool like Sheets, Slides. So basically Google’s revenue model for Gmail and more generally Google apps is to upgrade free accounts to paid accounts.
The way I would want to think about the metrics is around the customer journey the user takes around gmail.
Acquisition:
This is where a user first creates a free user account. Now, Gmail already is the most used email client. However, it would want to track the new users especially in the teens that are subscribing to gmail once they reach the email-allowed age.
How many new gmail accounts are getting created?
Engagement:
One aspect of engagement is the volume of emails sent/received. If this drops then maybe the user is shifting to some other email clients. So, the metric here would be the number of emails sent by users.
The other aspect is the efficiency of engagement. There are a lot of focos from Google on this for example Spam filters targets tries to hide noise from user’s attention. Similarly, having the Primary, Promotion and Social tabs enables users to focus on the most important emails in the Primary section. Also, the auto text suggestion while composing emails. The way we can think about efficiency is how quickly and easily users are getting the job done. For that we have to define what is the job at hand. Job is reading emails and writing emails.
On the email writing side, the important thing is how easily/quickly the user can write the emails. So, the time taken to write emails would be a metric.
On the reading side, the value would be to surface important emails while hiding unimportant/spam emails. So, if the prioritising was working perfectly then 100%v of the emails will be opened by the user. So, the metric could be the number of inbox emails opened by the user.
Conversion:
This is where Google captures the value when the user upgrades to a paid account.
Free accounts upgraded to paid accounts
I don’t remember if the pricing model is per seat based, if that is the case, then
Number of seats per paid accounts
Retention:
In any SaaS model, Retention is key hence we need to check for.
Retention Rate of paid accounts
Platinum PM
Approach:
1. Ask questions to understand the business goal
2. Identify key user actions
3. Define user metrics and financial metrics
Questions
Candidate: Gmail is created to offer a mailing service with sufficient storate capacity for the users; therefore success of gmail will be measured in terms of user adoption, user engagement.
Second, in order to run a sustainable business, we need to monetize the service; therefore the success of gmail will also be measure in terms of its financials.
Interviewer: Sounds good
Candidate: Do we want to focus gmail for personal uses or gmail for business uses?
Interviewer: Personal use
Candidate: Do we plan to track macro metrics (internal/external), or micro metrics ( only internal metics)?
Interviewer: Both
Key actions:
A. Users sign up for mail service through web/app
B. Users launch the app
C. Users receive the messages/ read the messages/ create a draft message/ send the messages
D. Sign out of the app
Metrics:
I would like to track user metrics, financials, macro metrics
User metrics
A. #Users that have signup for the app; breakdown by platform/device (discovery)
B. #Users that logged in to the service in the last month; breakdown by platform/device (activation)
C. #Users that opened the inbox message; #Users that composed the draft messsage; #Users that sent the message (engagement)
D. Average number of messages opened in the last month; average number of draft messsage composed; average number of draft messages sent (engagement)
E. Average number of months in a year a typical users open atleast one message/compose one draft/send one message (retention)
Financials:
A. Overall ad revenue generated from gmail; its % of over overall google revenue; breakdown by platform, geo
B. Overall revenue from cloud storage; its % of over overall google revenue; breakdown by platform, geo
Macro metrics:
A. Gmail's market share in the personal mail service
Clarifications
- What is the objective/goal for the product that defines success?
Assuming Increase in customer engagement as more number of people engaged will end up creating clicks on the advertisements
Describe the product
Gmail is used to exchange communication in form of text/emails attachment between the two or more than two parties. People can receive and the send the emails to each other using gmail. The information can be stored in a structured manner.
User group
Personal use Individuals to individuals
Individuals to Business and vice versa
Business to Business use (small or medium scale business)
Value Proposition
During the old days we use to have physical post office to send the communications via letter and telegrams that use to take lot of time in sending the message to the other parties. The urgency use to lost by the time it reaches to the receipient. It involves lot of efforts and time to collect, compiled and send the letter from the post office side as well as the senders side who need to buy the letter, write it paste the ticket, weighting it in the post office and send it out. At times letters use to get lost in transit. There were no other means of communication for a common man, Indeed, very few selected segment were using telephones but that use to be an expensive affair. Rise of gmail in the era of internet and cell phone at a very cheaper cost is like a boon in the business model.
- Emails can be sent in just few seconds to the other party and receives by them in few minutes
- No extra cost is needed unless the user wants to buy storage in drive but even the free limit is sufficient enough for a normal user
- Spell check and grammar helps to write email with ease and takes less time in writing
- Videos and pics can be sent in form of attachment
- No extra effort is needed for a very good handwriting. All the formatting options are available to make email looks standard and formal
- Receiver and sender can keep the record of communication in a user friendly structured manner with ease that was difficult to manage in the era of telegrams and physical letters.
User Journey- I would consider quick transmission of emails without data loss and proper record keeping of emails as the key pain points served by the aforesaid value proposition.
- User sign up for gmail account
- User create an email enrich it with required text, and the required files in the attachment
- User checks the grammar and formatting of the text content
- User add the receipients
- User sends the email. Receiver receives the emails
- User can search the past emails in the inbox, sent items and PSTs if received or sent out
- User log out
Metrics
- Average number of emails sent out by user just after sign up
- Average time taken to draft with the required content send out an email
- Average time taken to send an email and the taken to receive the same email. The size of the email should be maximum 25MB. The receiver and sender should be two distinct gmail IDs.
- Average time taken to send an email and the taken to receive the same email. The size of the email should be maximum 25MB. The receiver and sender should be two distinct IDs where sender is from gmail and receiver is from other domain like yahoo, rediffmail.
- Average amount of time required to search an email in the inbox and sent items when the available limit of Gmail is full
- Average number of gmail users sends email to the unique gmail recipeints periodically
- % increases in number of gmail users sends email to the unique gmail recipeints periodically
- Frequency of sending emails by one user through gmail in one session
- Average number of emails miss placed/lost when the capacity is full in a year
Evaluation
The success of gmail can be measured based on the aforesaid metrics. There are few trade offs while using gmail as a means of communication. The confidential data can be lost if your email ID is hacked. Indeed, the user can maintain strong passwords that needs to be updated periodically but users might not do that activity due to laziness and it is also not mandatory to do so. There is a limit of 25MB to send the data in one email which might force the user to opt other features like sharing in google drive or stop using gmail for sending large content. Users might suffer if the password is lost and even after using all the alternatives to reset password they end up losing time and efforts that will negatively impact the usage. Unavailability of internet is yet another threat. If a communication needs to be sent on urgent basis and internet connectivity is not available then usage of the feature will steeply fall down.
Impact, Reach and Revenue are the three key areas under which I would recommend 1,2,3, 7, and 8 as it will help to understand how many users are actively using the feature to communicate and how much is the frequency. The impact will be high as these features and metrics will help in assessing the if the communication has been reached on time or not to make the feature efficient. Revenue generation will happen through ads which is an indirect method. I am assuming regular user who uses it for personal use might switch to another email account the moment their storage is full or will deleted the unwanted emails. In some cases which involves business interaction they prefer to buy the space.
Gmail: is used for sending and receivings emails:
Users of gmail:
- Businesses promoting through email marketing
- sending emails to friends, family or to any organization
- Sites giving access using gmail accounts: helps in autentication as well fecthing basic details
- Gmail gets mapped with other apps and can be used for tracking purpose and for the validation
- Used by businesses to have their domain and use this as a medium for communication
Success of gmail:
By tracking following : for non enterprise and enterpeise accounts
- How many unique users are creating accounts-daily, monthy and yearly
- how many users that are creating an account , how mnay are actually sending atleast 1 email and receiving 1 email
- How many emails are being passed(in and out both) from an account ( sent)
- How many msges are getting ( delivered) and (opened)
- How many emails are getting read from the notifications title ( sent and read from notifications landed to tray and are never opened)
- how many emails are being (sent to/ received from) the repeated users
- tracking Otp based, social media, other sites promotional emails differently will give a broader picture
- How many emails on daily basis are marked spam
- What is the average session time of users
- How many users are switching between the accounts and then its reading and opening funnel will be tracked
- how many enterprise users are there in the system for more than 12 months, 5yrs
Platinum PM
Step 1: Explain your understanding of the product
Gmail is a free software mail service by google that is essentially used for communication via an e-mail. it's the largest mailing service in the world with over a billion users.
Main mission of gmail is to connect people across the world and bring them closer to each other thus instilling a sense of community. Even though it's mainly used for official/formal conversations can be used for informal conversations as well.
The different kinds of user groups using gmail are as follows:
A) Business users B) Casual users
We can further categorise these users also based on their usage as follows:
1)Power users: These are the kind of people that use gmail very actively. Use it multiple times a day and very promptly read, reply and compose emails
2)Casual users: These are the kind of users that use gmail multiple times a week
3)Passive users: These are the kind of people that rarely use gmail.
One can write, send, receive, delete, save emails using gmail.
Step 2: Ask clarifying questions to narrow down the scope of the product/question
1) Are we considering gmail as a whole or part of gmail? (Part of gmail)
2) Mobile or web based app? (mobile)
3)IOS or Android? (Android)
4) Measuring the success of gmail in a particular region or not? (Yes only for users in US)
5) What do you mean by measuring success? (Is there any particular thing that you have in mind and want me to track or should i decide what to track and measure)
Step 3: Define a goal to further narrow down the scope of the question and state your reason for the same.
Goal: The main goal of gmail is to increase number of emails sent and received on a daily basis, thus this will only be possible if users engage and spend more time on the product for that reason we will be using User Engagement as the goal of the question.
Our main aim will be to increase/drive user engagement of gmail and we will be measuring success of gmail regarding the user engagement metrics.
Step 4: List down the customer journey
-User goes to gmail app
-Signs up to create an account
-Logins in to the account
-Starts composing an email
-Sends an email
-Receives an email
-Opens to read the email
-Decides to reply to it
-Deletes the email after replying
-Logs out of the app.
Step 5: List down the metrics for each of different phases of the customer journey
Awarness, Acquistion/Adoption, Activation, Engagement, Retention, Referral, Revenue are the different phases of the customer journey.
Since we will measuring Engagement. Hence let's down the metrics for user engagement.
1) avg # of mails sent by a user on a daily, weekly, monthly basis
2) avg # of mails received per user on a daily, weekly and monthly basis
3) Avg # of a times a user logins to the gmail app in a day
4) Avg time spent by a user on gmail.
Step 6: Evaluate and then prioritise the metrics based on RICE model
| # | Relevance to company's mission | Impact to end user | Confidence in the metrics collected | Implementation effort |
| 1 | H | H | H | L |
| 2 | H | H | H | L |
| 3 | M | M | M | M |
| 4 | M | L/M | M | M |
avg # of mails sent and received by a user on a daily, weekly, monthly basis as the primary metric using which we will track the success of gmail across user engagement.
Platinum PM
Question: How would you measure the success of Gmail
Clarifying
“Gmail” is the whole product or specific feature of Gmail?
Time is limited so I will focus on the the core value of Gmail which is to receive and send an email
Product
Problems it is solving
Formal communication to each other
Goals
Get more insight from users
User = existing users
Journey map
Users Write and Read email
System detects the keyword, predict the demand of users
System suggest the targeted ads
Users See targeted ads
Users Click to ads
Users Engage / Purchase contents, products of ads owners
Metrics
Write and Read email
# of email write
# of email read
System detects the keyword, predict the demand of users
# of keywords per email
# of keywords per users
% of prediction confidence
System suggest the targeted ads
Funnel of targeted ads
Reach
Engage
Activate
Purchase
...
Evaluation
Impact | Complexity | Risk | |
# of email write | Medium | Low | Medium |
# of email read | Medium | Low | Medium |
# of keywords per email | High | Medium | Medium |
# of keywords per users | High | Medium | Medium |
% of prediction confidence | High | High | Medium |
Funnel of targeted ads | Medium | Low | Low |
Recommendation
# of keywords per email
# of keywords per users
% of prediction confidence
Trade-off
Technical challenge: Hard to measure
Unstable preciseness
Not so sure the keywords are relevant to user insight (easy to be skewed)
Platinum PM
How would you measure the success of gmail?
Describe the product
Gmail is the cornerstone of the google suite of products. It is the entry way for users to access other google services. Gmail itself is a web and phone application that allow users to communicate through the sending and receiving of emails. Users can also edit, delete, organize emails but these are secondary to the send and receipt function.
Gmail address also serve as a primary authentication for many users when logging into other platform and apps. When you sign up for a service you are asked to enter a valid email and then authenticate yourself as that user by responding or collecting a code from a sent email.
Gmail is part of a free suite of offerings and part of the g-suite paid functionality.
Assumptions
I’d like to focus on the free gmail.
I’d like to set a goal of engagement / retention. While we can look at other goals such as acquisition, revenue, or market share I think engagement / retention is the most important.
Acquisition is very important but because gmail serves as an entry product it is possible to have great acquisition metrics and not actually be able to measure success.
Revenue – for the free version of the product is indirect through advertising. There might be a paid option to get more storage – but this does not apply to large sections of the user base.
Market share – gmail is the leading email service but like acquisition this number could be very high as gmail is a gateway but still not show success.
I think engagement / retention is really important as it shows that users are actually using the product and reoccurring use is success. Not just signing up and then never using it (my Yahoo email) I’ve lumped these two together as reoccurring engagement shows retention.
Customer Journey
1. Sign up for an account
2. Send email
3. Receive email
4. Read received email
5. Respond to received email
6. edit, delete, organize emails (secondary)
7. Use Gmail address to sign up for other services
Metrics
Journey | Metric | Significance | Primary / Secondary |
Sign up | d/w/m active | Just signing up doesn’t show usage |
|
Send email | Number of users that send high medium low number of emails | Tracking the number of users that send a high / medium or low number of emails is great way to show engagement – if you track this number over time it shows retention. However some or many users may get a lot of value out of gmail and be in that low usage group – they might receive emails but not send many. | Primary |
Receive email | Number of users that Receive high medium low number of emails | Tracking the number of users that receive a high / medium or low number of emails is great way to show engagement – if you track this number over time it shows retention. However some or many users may get a lot of value out of Gmail and be in that low receive group – they might receive emails but not read many. | Primary |
Read emails | Number of users that read high medium low number of emails | Tracking the number of users that read high / medium / low numbers of emails is likely the best measure of engagement. As a user I may not send emails, I may only receive a low number of emails, but If a user is reading their emails or a portion of their emails – they are most clearly using gmail. – if you track this number over time it shows retention. | NorthStar |
Read emails | Number of users in bands of Percentage of emails read | This is different than the number of reads and is an interesting secondary metric. It’s possible to be in the low receive band but to read them at a high percentage. I would want to know this because if the banding differed significantly from the number of users that read in bands in might show something is off. It also shows us junk mail rate. | Secondary |
Respond | Number of users that respond high medium low number of emails | Like send and receive responding is a great way to show engagement – but like those metrics I could be very successful user and not respond that often. - If tracked over time there this also shows retention. | primary |
edit, delete, organize emails | Number of users that edit… high medium low number of emails | I think this is secondary when compared to prime factors like send, receive, read | secondary |
Use Gmail address to sign up for other services
| Number of users in high medium low bands that receive a verification / password reset / code email | If users are using their Gmail address to maintain other accounts this is a huge sign that they are actively using gmail. If we time sequence this it shows retention. While this is a very cool metric – I think it is subset of the receive and read metrics | Secondary |
Summary
I focused a lot on users in bands high /medium / low. There are other ways to measure such as avg number of sends/ receive / read per period but by focusing on the users it ties directly into the retention goal that we stated above.
I imagine as the PM I come in every morning and open a report that shows me a four line graphs one for send ,one for receive, one for read, one for respond. Each graph has three lines for bands of users high medium and low tracked over time. I want to see those lines showing an upward trend. Next to each graph would be a percent change for high medium and low users that cold be toggled for time period vs yesterday, vs same day last w/m/y.
These graphs and metrics show engagement which is tightly wound to retention. However it mixes retention and acquisition so having the ability to toggle out new users would get the core of the retention.
Q: How would you measure the success of Gmail?
Product description: Gmail is an email product with near unlimited amounts of storage.
What is the goal of the company?: Activation,Activation, Retention, Referral, Monetization
Given that gmail is a mature product, I think the goals would be focused on retention and monetization
Actions:
User’s journey:
Mobile (android/iOS)
Receive notification when there’s mail
Click on notification
Open app
Read mail
reply/forward/delete/archive/sort mail
Check social/junk/promotional folders
reply/forward/delete/archive/sort mail
Exit app
Desktop
Open mail page/tab
Read mail
reply/forward/delete/archive/sort mail
Check social/junk/promotional folders
reply/forward/delete/archive/sort mail
Close page/go to another tab
Check/reply to hangout messages [out of scope]
Metrics
# of notifications received - # of notifications clicked vs # of notifications wiped
Number of emails read
# of emails sent
Length of messages
# of messages archived
# of labels / # of emails labelled
Time spent on app
# of impressions on ads - click thru rate
# of spam messages received - % of total mail received
# of drafts stored
# of starred messages
# of promotional emails opened -% of total emails received
Average number of recipients
Usage in 1,7,30 days
# of contacts
Amount of data stored
Evaluation
Metric | Pro | Con |
Usage in 1,7,30 days [metric for retention] |
|
|
# of spam messages received - % of total mail received [metric for retention] |
|
|
# of impressions on ads - click thru rate [metric for monetization] |
|
|
Describe product and Users | Gmail is an email service, used by more than 1 billion users. Please can send and receive email and store the year of email without the need to delete. Users: Business users and Personal user |
Clarify | Email also has messaging chat features, do you want me to focus only on email or also on messaging. Only the email part. There is business gsuite version and general version , do you want me to focus on specific - Yes focus on general version |
Goal | The goal of the email is to organize your emails and keep you up to date with your social and professional networks. Product lifecycle: Gmail currently has a significant user base so I believe the current focus of the product is around retention and engagement is goal on |
Key metrics | Product metrics
skipped acquisitions activation referral metrics as not relevant to my product goals.
Engagement
Retention
|
Success metrics | % CTR per impression key metric to understand email are legit and user is not spammed |
Gold PM
first, I will give a quick brief about the product, and the business goal, map through the customer journey and evaluate the success metrics to achieve the defined goal and prioritize them and finally summarize the overall answer.
Let's start with the product
About Gmail: Gmail is a free email service developed by Google that has 1.5 billion active users worldwide. Gmail allows users to connect and communicate with each other over the mail personally and professionally.
Gmail first launched in 2004 and has 1.5 billion active users, assuming the product is in a mature stage.
Goal: To connect with peoples personally and professionally.
Since Gmail was launched on April 1, 2004. Gmail has slowly and steadily become the number one email service provider and has over a billion users across the globe. Gmail came in late to the email party but soon displaced Yahoo and Hotmail from the top spots.
Meanwhile, with the advent of the social media age, the use of email has declined. People have stopped writing emails to just 'connect' and that role has been passed on to companies like Facebook and Twitter although they are not direct competitors of Gmail. The primary goal should be.
- Engagement
- Retention
- Monetization/Increase profitability (Major or High-Level Goal)
User Group: Both Personal and business consumers.
User Journey
- User Create a Gmail account.
- User Login Gmail
- User Compose and sent mail, receive, open, read and reply emails, search mails.
Now I would like to evaluate the key metrics keeping the goals in mind.
Acquisition
- # Account created per day/per week/ per month
- The % growth rate of account created over a period of time. Growth should be MOM, YOY.
Activation
- % of users sent first mail after creating an account in a week, month.
- % of user opens first mail after creating an account in a week, month.
- % of users drop off after the first time using the email services.
- # of mails user send per day/per week/per month
- # of mails user delete, mark as spam per day/per week/per month
Engagement
- # of mails user opens per day/per week/per month (Primary, social and promotional)
- # of mails user reply of the read mails in per day/per week/per month
- # of times user search mail using the search option in per day/per week/per month
- Average session duration
Retention
- No of times user returns on Gmail per day/per week/ per month
- % of users drop off over a period of time.
- Retention rate over daily, weekly and monthly
Now let's prioritize the primary and secondary metrics from the above metrics on the basis of keeping High-level goals in mind.
The High-level goal is monetization as state above the product is in a mature stage with approx 1.5 billion active users monthly, I prioritize success metrics on the basis of revenue Impact.
The business model of Gmail is an advertisement based revenue model so engagement and retention would be the primary metrics to be focused on and acquisition and activation would be considered as secondary metrics.
Silver PM
1) the goal of success? user aquisition? revenue (monetization)? market share growth?
2) the market of success? in US, or global?
3) customer group of success? Gmail to consumers or Gmail to enterprise custiomers?
4) in what time period of success? in a year? a month?
say if the goal is to measure if Gmail is success in US marketing on market share with consumers within a year. then the metrics:
# of new user registration to gmail
# of daily active users (users who registered also use gmail at least have 1 email per day)
# of gmail space have been used per user
# of users who use gmail over the past 12 month
# of users who stop use gmail after 1 months, 3 months, 6 months, 12 months
if this is for enterprise customers:
# of enterprise account registered
# of enterprise account last over 12 months
# of gmail space have been used in 12 months per account
# of enterprise accounts stop use gmail in 1 months, 3 months, 6 months, 12 months
Top Google interview questions
- What is your favorite product? Why?89 answers | 263k views
- Build a product to buy and sell antiques.54 answers | 66.8k views
- How would you improve Google Maps?53 answers | 228k views
- See Google PM Interview Questions
Top Metrics interview questions
- How would you measure the success of Facebook Likes?40 answers | 61.3k views
- What metrics would you use to measure the success of the Save feature at Facebook? 39 answers | 87.2k views
- What metrics would you look at as a product manager for Instagram ads?19 answers | 23.8k views
- See Metrics PM Interview Questions
Top Google interview questions
- A metric for a video streaming service dropped by 80%. What do you do?50 answers | 135k views
- Calculate the number of queries answered by Google per second.45 answers | 78.5k views
- How many dentists are there in New York?33 answers | 36.7k views
- See Google PM Interview Questions
Top Metrics interview questions
- What is the most important metric for Google Docs and why?19 answers | 15.3k views
- What goals would you set for a Facebook Dating feature?14 answers | 9.4k views
- How do you measure success of the Facebook Fundraising feature?13 answers | 5.9k views
- See Metrics PM Interview Questions
