A drop in engagement is observed in Microsoft OneDrive. How would you investigate the issue?

What kind of drop are we talking about? Overall engagement has come down

Is it specific to any users - personal or business? No overall

How much is the drop? 10%

Was it sudden or gradual? Since last one week

Is it specific to any geography? No

Is it specific to any platform? No across all platforms

Metrics to check -

% users login -> No major change

Avg time spent per user per session -> Reduced

% users uploading a doc -> Reduced by 10%

% users sharing a doc -> No major change

% users commenting on a doc -> No major change

So, it seems like the drop in engagement is contributed majorly by the drop in % of users uploading docs in one drive

To investigate further -

Internal factors -

Was there any bug fix done recently? No

Did we do any kind of UI changes in doc upload flow recently? Yes, few changes done

We need to go back and check all the apis are working properly and there are no broken connections.

Did we launch any new feature in one drive recently? No

External factors -

Was there any new competitor product launched? Not aware

Was there any change in the pricing policy? Yes, tiered pricing was introduced based on usage

Do we know about the users whose activity dropped, were they heavy users or less frequent users? Majority had come from heavy users, but there was some drop in less frequent users as well

Have we observed similar issue before which came back to normal after few days? Not aware

Was there any negative PR about the company or the product related to data security? No

Hypothesis -

Users are not happy with the new pricing policy.

Need to interview few users to validate the hypothesis.

Also, can conduct a A/B test, where we take few heavy users and divide them into 2 cateogries - one seeing new policy and the other one see old policy and observe the change in user engagement. If the user engagement increases, then we can conclude that the policy changes has triggered this drop. We need to go back and rework on the pricing policy and include inputs from users as well in terms of their willingness to pay.

Short term solution -

If the UI issue exists due to broken api, need to rever changes immediately.

Revert the pricing policy immediately and inform users

Long term solution -

Fix the UI issues, test them properly and then release it again. Also, add a summary of the changes done along with a tutorial video about the changes done for users to un derstand the new flow

In terms of pricing policy changes, rework on the strategy based on competitor pricing and user inputs and aligning with organisations' goals, make the users under the value proposition for each tier of pricing and then launch the new pricing policy.

The interviewer should begin by paraphrasing the problem statement and then move onto clarifying questions:

• What was the duration over which the engagement rate dropped?
• By how much did the engagement rate drop?
• Did a similar drop occur across other platforms?
• Was there a sudden/recent change that occured in the product?
• Was the drop observed in any specific geography/region?

• Drop in performance
• Drop in reliability 
• Drop in security features on the product

• User moving over to competitor product
• Nework issues 
• Device specific issues 

• Roll back recent launches/features if any
• Address security breaches that took place in the recent past

• Put in place a roadmap for a superior product vis-a-vis competitor and ensure all NFRs & FRs are met.