How would you improve Outlook for the use case when people get overwhelmed by number of emails received after returning from a vacation?

Here is the structure I would follow.

Clarifying Questions- 

1. When we talk about improving Outlook, I'm assuming that we want to help the user be more productive and spend less time scrolling through all the emails. I'm making this assumption because the vision of Microsoft is to help people be more productive and the tools they build are centered around that purpose. I will check with the interviewer if this is the main goal for solving this problem. 

2. What is the scale of the problem and how does it manifest? Are users submitting queries that they find it hard to go through their inbox after a vacation? Is it regular or Business users? Any insights about the scope of the problem?

3. Finally, I'm assuming that Microsoft eventually wants to solve this problem across platforms, mobile and Web but it seems like this is a bigger problem on mobile because that we have UI constraints on mobile. 

User Groups- Here are some different user groups who might experience this issue.

1. Enterprise customers- Employees at companies. Their main concern is that they have critical emails that are waiting for their response and they want to respond to the most important email first. 

2. Students- A lot of students get .edu emails from Microsft and in the case of Students they are concerned about missing deadlines and maybe job emails etc. They are not necessarily looking at answering 

3. Free users- Unpaid users who use this as their personal email [email protected] and get a ton of promotional email, newsletters, advertisements, among these they have few important emails. 

Among these user groups I want to look at the rationale for solving the problem. I will analyze the rationale based on the following metrics

a) Impact b) Alignment with mission c) Data suggesting that this problem is worth solving.

Enterprise users- Biggest chunk of paying users for Outlook, so biggest impact, perfectly aligns with microsofts mission of making professionals more productive with tools. If enough enterprise users in a company complain that its a big problem, the company might cancel their Microsoft account and move to Google Enterprise. 

Students- Smaller scope. Students are not often overwhelmed with ton of Email and have a lot more time on hand than professionals to check emails. Also they don't have a lot of leverage because its a free product. While its good to solve, might not be the best use of time. Not perfect alignment with mission.

Free Users- Highest number of users for Microsoft but least impact because it sounds like a paid feature and possibly something to add for paid subscribers.

Based on these factors, solving the problem for Enterprise users seems best use of time and I want to go with that. Check with interviewer what they think and adapt based on response.

Solution Set-

Hypothesis-

How might I highlight the most important Emails that a professional needs to respond after back from vacation because they have limited time for checking email everyday and have a ton of email to check immediately after vacation.

Possible Solutions

1. Create a Blocker Email category when creating email to indicate that the person's input is needed. Automatically filter Emails with this category after Out of Office ends so that user see that first. Can also automatically sort Email based on conent for words like Inform, waiting, need your feedback and use those to create a Blocker category if we think users input is hard to get. 

2. Automatically categorize emails based on meeting invites, Emails with deadlines and FYI Emails. This can give the user a snapshot of action items so that they don't have to read through all the emails. 

3. Out of Office Rule Set- Create new set of rules as part of the Out of Office rule set which specifies 1. Catch up time where user specifies they need x amount of time to catch up over Email. 2. Auto forward high priority emails to contact specified and 3. Auto create responses and save them as ready to send drafts for most important emails. 

4. AI agent- Leverage AI agent to scan all incoming email over the holiday period and provide a list of action items for the user when they open the Inbox first time after vacation. The AI agent can also have draft emails ready which the user can review and send. 

I will evaluate these on the criteria- usefulness and Alignment with Vision. 

Based on the above factors, I would like to choose 2 solutions, one for the M.V.P and the other as a V2 solution.

For the M.V.P I will choose Recategorize Email because this is the easiest to implement and can help the user by doing work in the background and showing a curated list of action items that the user can tackle immediately. This way, the user can feel better about the work needed to be done right after they get back to work after vacation. The Blocker and out of office rule set are harder to implement because they require more work from the user. We want to limit the amount of additional work needed to be done by the user. 

For a V2 solution, i would explore the AI agent solution which utilizes existing AI agent capabilities that Microsoft has but is complex to integrate and needs thorough testing and validation befoe launching. 

Potential Risks- 

1. Hallucination from AI agent. Since LLM's are prone to coming up with information that is not factual, the solution needs to trained well and thoroughly tested.

Metrics

1. High priority Emails caputured in Recategorization.

2. Emails read /Unread ration in test vs control group.

1.       Let me make sure I understood the problem here. We want to improve Outlook for the situation when people are returning from vacation, so they feel less overwhelmed with the number of emails.

2.       Clarifying questions:

a.       When we say improve Outlook, are we looking into any specific device access: mobile, web, or computer?

                                                               i.      Let’s assume the answer is that I should decide.

b.       Are we talking about Outlook for businesses or Outlook for education?

                                                               i.      Let’s say it is outlook for business.

3.       Our problem here is that we are looking to improve Outlook for Businesses, in any platform, where people returning from vacation feel less overwhelmed with the number of e-mails they received.

4.       I will break down this problem into 4 blocks:

a.       Look at the company and market.

b.       Understand the user persona.

c.       Identify current pain points.

d.       Provide some solutions to overcome these pain points.

5.       About the company, Outlook is provided by Microsoft and is part of the Office Suite. Outlook is used as a means of communication within corporations and is vastly used by all company levels. Microsoft makes money by selling the Office Suite and charging access to all systems included within the Suite.

6.       The question talks about employees feeling overwhelmed by the number of emails; therefore, it makes sense that our North Star metric should be user satisfaction. Do you agree? Any issue so far?

7.       Let’s understand our user’s persona:

a.       Employees who took vacation.

b.       Employees who did not take vacation and are waiting for an answer from the ones who left.

c.       Internal IT system who needs to setup the infrastructure and configuration of these systems.

The question here clearly defines that our user persona should be the first group, however, I just wanted to have a quick look at other possible groups involved as well.

8.       Let me take a couple of minutes to understand this group’s pain points.

9.       There are three pain points I judge being the most important:

a.       Relevancy: you don’t know which e-mails still relevant and which ones were solved when you were on vacation.

b.       Priority: you don’t know what e-mails have the highest priority.

c.       Time: you need another day or two to catch up on all the readings, but your calendar is already booked.

10.   I would love to stop here and make sure if you have any point of concern so far. If not, I will move forward with identifying some solutions for these pain points.

11.   If we look for solutions for these pain points, we could take the following approach:

a.       Relevancy: After returning from vacation, many of the questions asked on your e-mails have been solved. We could develop a function that notifies the sender that the person is currently under vacation until an X date. A day before this X date, this function would send an alert to the sender detailed that he or she sent a certain number of e-mails to the person on vacation and ask which e-mails are still pending an answer. He could then select which ones are still relevant. The received, i.e., the person on vacation would then have those e-mails updated to a different color or flag to signalize that the issue has been already solved and that he can ignore that e-mail for now.

b.       Priority: In a similar way, we could create a review function that would notify the sender about all the e-mails he or she has sent to the person current on vacation. A few days before the person returns from vacation this function would notify each sender about all the e-mails he or she has sent and ask to prioritize what is the order of priority for each e-mail. This way if someone send you 5 different e-mails you know which ones to prioritize first.

c.       Time: For the last pain point, Outlook could automatically block the person returning from vacation calendars to avoid everyone from stacking up meetings after his or her return. There could be a setting where you specify how many days you would like to automatically block after returning from vacation. With this solution you reduce the number of meetings you need to attend right after returning from vacation and have some time to catch up on your e-mails.

12.   Looking back at our North Star metric, we defined that we are interested in user satisfaction. These three solutions seem to fit well into the metric. However, when I look back at my previous experiences and remember some of the conversations I had with colleagues, I believe the first option, the relevancy functionality, would bring the most results to increase satisfaction because it avoids spending time on something that has been already solved. Our time is already limited, therefore, removing points already solved from our list seems to be the best approach here.

13.   If I put myself as someone returning from vacation, I would love to see my mailbox and observe some e-mails flagged or colored indicating that the point has been taken care of while I was off. This would demonstrate that my team can get work done without my presence and that the team or company is not 100% dependent on me.

14.   There are some downsides from this solution that we need to take into consideration. The main one that comes to mind is that when someone sees a flag indicating that a problem has been solved, the person may just ignore the subject and never look back at it. This could cause future problems because someone could possibly not be in the loop of a major issue and, therefore, not been aware that something bad happened and needs to be avoided. One possible solution could be to outlook send an update message at a certain time after the person returns from vacation, say, one week, and remind the person that he or she should catch up on these X number of topics.

15.   In summary, we wanted to improve Outlook for business so when someone is returning from vacation, he or she doesn’t feel overwhelmed with the number of e-mails. Our North Star metric is increasing user satisfaction and the user persona here is the person who took vacations. We will focus on the relevancy of the emails and the solution proposed is to have a flag indicating the question or issue in an e-mail has been already solved. There may be downsides to this proposal where the user may just ignore these flagged e-mails, but a way to overcome this is having Outlook send a recap of what has been fixed while he was on vacation after an specific time of his return.

To develop a better understanding of the use case, and define the scope of the problem, I would ask the following clarifying questions:

• How was this problem statement discovered, was it user feedback, user research, competitor study or from our own metrics. Do we have access to that literature/summary?
• Outlook is available across platforms, is this request platform-specific ie desktop app, web app, mobile? Is there a priority order?
• The group of people who use outlook is diverse and can be segmented in many ways, but for this question, I’d like to understand if we are looking at home users or corporate users.
• How much time away do we define a solution for 3 days? 2 weeks? One month?
• The next consideration I’d want to make to define the scope of this problem is concerning hardware/system limitations?
•  Is overwhelming here defined as people having too much email to deal with or the associated system issues such as
•     Mailbox full
•       Rules running too long on restart
•     Mailbox Load time

1. Email statistics report: A dynamic report that takes as input date range and shows a summary of emails recieved in the intervening period. This data can slice data by senders, to and cc, promotional etc. The report is a single snapshot of the volume of emails the user has to deal with.
2. Outlook vacation mode

• Easy OOO setup: This feature reads from calendar or takes user input for days that one is going to be out of office, infers from corporate directory - out of office contacts and creates an OOO message with relevant details. While the feilds are customizeable, they are non optional. Also sets OOO on the calendar
• OOO Rules: These are a set of preconfigured (but customizeable rules) to help prioritize email upon return. Default rule set can contain rules like "To and CC", "Marked as important", "from manager, skip level", "Expired meetings". The default rule rule serves the dual purpose of having a generalized prioirtization algorithm and also guide people on how they might create their own rules. OOO Rules are necessarily simple rules that can run quickly on the large dataset when the user returns. 
• Vacation mode toggle: A toggle button to switch in and out of vacation mode quickly. This switch should display vacation mode settings - ie OOO setup and OOO rules, and allow the user to make changes if desired. This can be extended to work on handheld devices and web versions with default settings.
• Vacation mode auto start: Based on user preference this mode should automatically kick in post 3 working days of inactivity

Approach:

1. Understand the prompt and outline the business goal

2. Establish the user segments and identify the painpoints

3. Brainstorm the solutions

4. Prioritize

Understand the prompt and outline the business goal:

Candidate: 

While on vacation, some people receive large number of e-mails which is overwhleming and some would receive fewer number of e-mails which is easy to manager. 

Should we use the build new experience when the number of inbox messages is above a certain limit when the user is OOTO.

Interviewer:

Yes

Candidate: 

Does the new experience focus on just e-mails or both e-mails and calendar invites

Interviewer:

Both e-mails and calender invites

Candidate: 

I assume the goal here is to improve the productivity of the users which is measured in terms of the time it takes to action e-mails received when the user was in vacation. You agree?

Interviewer: 

Yes

User segments:

A) Power users that the create rules to organize the e-mails

B) Users that receive small number of messages and that manage the inbox manually

C) Users that receive large number of messages and that manage the inbox manually

To deliver most impact, I would prioritze that segment that require the most help and largest in size- namely large users that manage the outlook manually

User Journey:

A) User decides the vacation time and sends a calendar invite about the same

B) User creates a automated OOTO notice 

C) Users goes on an vacation

D) Users contine to receive the message while OOTO

E) Users come back and responds to all the message

Identify the pain points:

A) Users does not set up a OOTO notice, thus co-workers and stakeholders keep sending messages

B) Users forgets to provide an alternative poc while OOTO

C) Users does not a feature to assign/forward the mail while OOTO

D) Users does not have a sophisticated tools to separete e-mails where the user action is required from the ones where the user action is not required

E) Users does not have easy tool to unsubscribe from the e-mail distributions lists that are not relevant

Brainstorm solution: 

A) No OOTO notice set up: When the users send their OOTO calendar invite, we will remind the users to create OOTO note.

Effort: Low, simple UI pop-up

Impact: Moderate, power users tend to set up a OOTO note without any prompt, this feature will help new users more

B) No alternative POC: Provide a template for the OOTO that includes that POC to receive out to, when the user is OOTO

Effort: Low, easy to build a template

Impact: High, 

C) Autoforward to POC: When the user receives mail regarding regarding a project, outlook will use the natural language understanding and will provide a capability to automatically forward the message to the relevant POC.

Effort: High, need to develop new features to auto-forward the mails

Impact: High, alternative POC would get the visibility and reduce the number of unactioned e-mails for the user 

D) Lacking tools to prioritze mails: User wants to first identify the e-mail where the action items is pending from the user vs. ones where users does not have any action items. 

Simple solution is outlook can build natural language capability and find the mails that are addressed directly to the users vs. the ones where the user is just kept in the loop. Thus user can easily prioritize the e-mails that they need to action on and can read other e-mails at their own pace.

Effort: High, need to invest in cababilities to understand the natural languages

Impact: High, users woud benefit a lot from this capability to separate that needs their immediate attention

E) Unsubscribe tools: One method to help the user quickly get through their inbox is reduce the clutter and filter the spam message. We can build a tool unsubscibe tool that prompt the user to unsubscribe from the distribution lists whose email the user did not read in last month.

Effort: moderate, need a build a feature that let users unscripbe from within the outlook

Impact: high, will help a great deal to unclutter the inbox

Prioritization:

If I were to prioritize one feature, I would build capabiilty that separate the mails where the user action is required with just a click of a button.

A simple to use structure for this Microsoft product design question:
Clarifying Questions
Users
Pain Points
Solutions
Metrics
 

Clarifying Questions:
1. Are we looking at a front-end UI based solution or more back end application feature solution. - You choose (could be both)
2. Are we focussing on only company internal emails or even external? - All
3. Is this feature going to work only when we set an Out of Office automated message? - In other words, how would the system know that we have been on vacation and returning? - Assume yes.
4. Web only or App? - Both
5. What is the goal here? - To increase productivity/efficiency in email organization and responses
6. Any constraints to worry about? - No

Users:
Email senders
Email receivers
Microsoft
IT department of the company

We want to concentrate on the email receivers as the primary user group-  but we also want to keep an eye out for the senders group as we think of the solutions.

Pain Points:
1. Too many emails to read - not sure where to start and how to go about getting caught up.
2. Depending on the importance of the role, backlogged emails can cause outlook to be out of space preventing potentially more important emails being delivered to the receivers email inbox.
3. Clogged inboxes could also mean that the receiver wont be able to send out important replies/emails until outlook has space.
4. Difficult to understand what emails need immediate attention and which ones can wait -  Some emails need quicker replies as there might be dependency.
5. Some users have rules for certain emails/senders based on rules - attention does not go to those rules folders when we see too many emails and things can get lost
6. Some emails may even be meeting invites that may be scheduled based on emails where user might need to have read the emails to attend and ensure usefulness 

Pain Point Prioritization: Based on impact and reach, the following is my prioritization:
1,4,2,3,6,5

Solutions:
1. Build a feature that flags emails that have the receivers name in the subject line or body of the email - this will be more important and might require user's attention.
2. Based on org hierarchy, group manager's/skip levels or important stakeholders emails on top for immediate attention. (important stakeholder could be determined by understanding the most frequent person who the receiver interacts with)
3. Use some sort of ML/Text reader to understand emails with deadlines or action items called out for the receiver.
4. Use AI to understand if there are meeting requests in the near future that bear a title similar to an email chain in the user's inbox and add a flag for the user's attention - this allows them to review quickly and complete action items as needed to be best prepared for the meeting.
5. Once the user opens their inbox post their return from vacation - send a pop up notification that shows them a summary of the email count that was received during their vacation time- This can be categorized based on some of the filters/flags above -Psychologically, it is established that people are better prepared if they know what to expect rather than being thrown into the mess without a 'heads-up'.
6. To reduce the clutter in a person's inbox - build an intelligence system to intimate the sender that the receiver is OOO, as they enter their names in the to/cc/bcc lines (this happens today) - This system can share a popup saying that the receiver is OOO and check if the email can wait till they return. - Store this information and remind the sender once the receiver is back asking if they still want to send the email now (if they did not send before) and then send the latest email in the chain.

7. Monitor outlook space levels for employees with an OOO and as their outlook approached full capacity, send an automated notification to IT to temporarily provide additional space for their outlook.

Based on Reach, Impact and Effort, I would prioritize the solutions in the following ways:

Metrics:
# Notifications sent
# Prioritized emails opened first
# NPS Scores
# Notifications sent to IT
# Number of clogged emails ( as a percent of total emails)