You’re the PM for an e-commerce marketplace and you’re responsible for the recommendations feature. In the past quarter, you ran an A/B test for your new feature and you’re supposed to present the results to the execs. What metrics would you present to them?

That's a pretty vague question and thus, I would like to improve my understanding through assumptions and clarifications before sharing what I understand and howI would like to move ahead.

Assumptions:

I would like to think this E-Commerce marketplace as a place where user can buy and sell pretty much anything and not a particular type. That is I would like the E-Commerce to be a platform like  Amazon, Alibaba and not like Nykaa(Sells only cosmetics).

I would like to assume that the Product is at between Introduction and Growth stage. (P.S: Most of my answers revolved around pretty established firms, So just for a change I'm assuming this and no other particular reason)

I would like to assume that the A/B test was done for a smartphone user and not Desktop/Mobile web application as the % of smartphone users are increasing rapidly

I would like the market to be a developing country like India because, that is the place where Growth rate of Mobiles are higher. Moreover, Coming from a developing country I can relate to this more than markets like US without any data. By the way, I'm not sure how important this factor is gonna be at this stage. However, I would like to mention this upfront.

Clarification:

What was the goal behind the A/B test? I would like you to decide on that. However, you can be sure that the test was related to improving our Recommendation Engine.

By Recommendation system Do we mean the Recommendation for the users in the App or the Recommendation on how much product to be stored in which warehouse? It is the former one

With the above assumptions and clarification, I would next like to understand what would be the best goal for a firm suiting the above situation before listing any metric.

E-Commerce is a space with huge amount of competition in general and thus, a start-up transitioning from Introduction to Growth phase should focus more on creating awareness about their arrival and acquiring new users. This will help the firm increase it's market share and have a steady foot in the increasingly competitive space. However, since the A/B test is on the Recommendation, it makes more sense to look at how users are engaging around this feature and how this helps the firm in transacting more businesses. So the best goal to choose for our A/B test will be Engagement and User Acquisition

Metrics related to Recommendation system:

1. % of increase in organic growth in users over the quarter (users through Word of mouth)- The increase in the growth rate can be attributed to the change in Recommendation system
2. % of increase in in-organic growth in users over the quarter.
3. Customer Acquisiton Cost
4. # of sign-ups over quarter
5. # of users adding their billing addresses
6. # of searches per quarter(the lower it is  the better our recommendation is)
7. Average time spent before clicking on a product from recommendatioon
8.  CTR of Home Screen( the higher it is the better our recommendation system is)
9. CTR of Different Sections (like books homepage, fashion homepage)
10. # of things added to wishlist from recommendation
11. #of Successful transactions
12. Ratio of Products bought from recommendation: Ratio of Products brought through searches

With the above metrics listed I would like to choose one from User Acquisiton and Engagement using factors like 

• Relavance to Goal n situation
• Impact to end user
• Easiness in measuring