How would you determine the optimal number of ads to display on a Yelp page?

Assumption: Assuming we are optimizing ads on the Yelp search results page.

Goal: Optimize ad revenue for Yelp/Advertisers, while keeping the user engagement.

In short: Build different ad experiences and A/B test them as treatments.

Experiment with the parameters:

1. Ad format: The creative's look and feel - e.g. work with the design team to come up with different design requirements that aim at maximizing engagement, and test them out.

2. Ad density: The number of ads on the page - e.g. try 1 ad on the page, vs multiple ads on the page with a certain frequency (1 every 5 organic results), try both a native ad in-between the results, and a display ad at the very top (header of the site) or on the sides.

3. Ad placements: positioning of ads on the page - e.g. try 1 ad at the top, or in the middle, on sticky ad on the right side of the results.

In the experiment, compare these metrics for control vs treatments, to decide the winning version:

a) Ad revenue

b) Ad CTR

c) Advertisers ROAS (effectiveness of the campaigns)

Guardrails:

• User time spent, scroll rate
• Organic results CTR
• DAU (active users - a must)
• Churn rate (to gauge retention)