Recruitment at Google costs a billion dollars a year. In our search for false positives, we have had a lot of false negatives. How would you build a product to solve this problem?

Question

Recruitment at Google costs a billion dollars a year. In our search for false positives, we have had a lot of false negatives. How would you build a product to solve this problem?

Asked at Google over a year ago

1.3k views

Product Design category

Asked at

1.3k views

How to answer Product Design questions

Interview Guide

Top Product Design interview questions

How would you design a web search engine for children below 14 years old?36 answers | 42.9k views
Design a library for the future.25 answers | 23.5k views
How would you design a consumer application for a scooter sharing business?21 answers | 18.6k views
See Product Design PM Interview Questions

Invite members
Invite by email

Add another

productmanager1644 · Answer 1 · 2021-02-26T17:02:36+0000

CLARIFY:
1. Should this product be global? Yes.
2. Is there a specific channel: desktop v. mobile v. tablet? You choose.
3. Should we focus on a specific type of recruitment candidate (ex. Engineering, Product, Marketing, etc.)? You choose.
4. Should this product be focused on helping recruiters or helping source candidates automatically? You choose.
5. Is there a specific aspect of recruitment spend you'd like to focus on (recruiter head count, travel to recruitment fairs, etc.)? You choose.
6. Is there a specific goal? Reducing spend or better filtering positives / negatives? Focus on the latter.
GOAL: Build a product to filter candidates more accurately in the recruitment process - i.e. make sure positives are true positives and negatives are true negatives.
USER GROUPS: There are two main groups of users we can focus on. For this product, I'd like to focus on the Recruiter, as we can help them directly with the candidate filtering process.
1. Recruiter: Google employee sourcing candidates.
2. Candidate: Candidates that Google is actively recruiting. Could be across any department: Engineering, Product, Marketing, etc.
USER JOURNEY: Below is a high level journey of a recruiter.
1. Recruiter is provided a job description to find candidates for.
2. Recruiter posts job on Google job website / opens it to candidates.
3. Recruiter post job on social media sites, like LinkedIn.
4. Recruiter surfaces job sites, like LinkedIn, to find candidates that fit job specifications using key words. For example, candidates can add tags to their LinkedIn profile with areas they specialize in, like "Product Management" and "Digital Payments", etc.
5. Recruiter finds a pool of candidates that meet the job description.
6. Recruiter contacts all candidate.
7. Recruiter sets up time to speak with candidates who respond.
8. Recruiter filters out any candidates after initial call that may not meet qualifications or Googliness.
USER PAIN POINTS: In regards to the user pain points, I'd like to focus on the finding candidates aspect of the user journey, as this portion of the journey is most relevant for the goal.
1. Specific Key Words: Key words for Google's job description are not specific enough to role. For example,
2. Filtering: Google's key words filter out candidates unnecessarily because people use different terms for the same thing. For example, some companies may use "Design" v. "UX".
3. Job Title Misnomers: Different companies use different titles for the same type of role. For example, a "Product Owner", "Product Manager" and "Product Strategy Manager" may actually have similar roles but the titles can be misleading.
4. Education Background: Recruiter filters for certain educational requirements but filters out candidates that have non-traditional educational backgrounds that meet job description. For example, people who did not study engineering in college may have done an engineering bootcamp or self studied / taught themselves how to code.
5. LinkedIn Presence: Candidate does not have that much description on LinkedIn, so recruiter has passed them over. (Certain companies may prevent employees from describing role too much.)
6. Unknown Company: Candidate works for company that recruiter has not heard of (ex. start up). Recruiter is filtering for certain companies and overlooks others.
7. Company Filters: Recruiter is specifically targetting other companies in that area for the job description but misses candidates that work at a company that are not on their radar. For example, candidate may work at Nike, which recruiters associated with sneakers / shoes, but they may be focused on mobile applications. Recruiter is filtering for Big Tech companies only and misses Nike candidate.
8. Poor Interviewing Skills / Lacks Googliness: Recruiter realizes on call that the candidate they sourced while on paper meets qualifications does not have good interview skills (cannot explain their background, no examples, etc.)
9. Deceitful Candidates: Candidates have lied on LinkedIn / listed experience that they have not done or is drastically overinflated, etc.

RANK PAIN POINTS:

Pain Point	Impact to Recruiter
Specific Key Words	High
Filtering	High
Job Title Misnomers	High
Educational Background	Medium
LinkedIn Presence	Medium
Unknown Company	High
Company Filter	High
Poor Interviewing Skills / Lacks Googliness	Medium
Deceitful Candidates	Low

POTENTIAL SOLUTIONS:
1. Historic Google Recruitment Review: Google creates an application that searches through all of its recruitment history by type of role. Reviews trends on false positives / negatives and comes up with additional data that help dictate whether a candidate should have been filtered out / in.
2. Filtering App: Google creates an app that allows recruiters to automatically filter candidates on social media. Allows recruiters to add certain tags and filter accordingly.
3. Key Word Match: Google creates an app that reviews all job posting data and automatically identifies where there are key words that match one another. For example, Company A may use the term "Design" but Company B uses the term "UX". Google creates a list of matching key words to help recruiters filter more holistically.
4. Background Review: Google creates an app that looks at LinkedIn data. Identifies candidates in certain types of roles and what their background was before they started that role. Comes up with holistic overview of what skills / edu background a candidate needed to get their job. For example, Google looks at LinkedIn data for all people listing "Product Manager" as their job title. Google identifies what they were doing prior and builds a list of cross over skill sets / edu background.
5. Recruiter Testing: Google builds an app that runs testing scenarios on their recruiters. Asks one group of recruiters to widen their search on candidates they recruit for with suggestions as how to do so. Google compares the number of false positives / negatives to standard recruiters.
RANK SOLUTIONS:
1. Solution Impact to Recruiter / Goal Cost to Google
  Historic Google Recruitment Review Medium Low
  Filtering App Low Low
  Key Word Match High Low
  Background Review Medium High
  Recruiter Testing Medium Low
CHOOSE SOLUTION: Based on the ranking, I would create a Key Word Match application that scans LinkedIn and other job posting sites to identify key words that each company uses to describe a role. Could start with LinkedIn, as it is a common site for many companies to post their roles on. App comes up with key words that each company uses to describe certain roles and tie them to specific roles. Recruiter looking for a specific type of role could search this database and see key words they should use in their filtering process. For example, the recruiter sees they should filter for "Design", "UX" and "User Experience" when looking for Designers. This app would help recruiters filter their candidates better because they are not filtering out qualified candidates. In order to mitigate the chance of getting false positives, app could suggest combinations of key words. Later, post MVP, app could even come up with better data on specific companies to help recruiter identify qualified candidates by company. For example, we know that Amazon designers in this area likely use "term A" and "term B" most often in their profile based on role postings from Amazon's site.
METRICS:
1. Rate of false positives and negatives when using key word app v. standard recruiter not using app
2. Number of candidates recruiter has to review using app v. standard recruiter not using app
3. Time saved during recruitment process on average / recruiter v. standard recruiter not using app
LIMITATIONS:
1. May be difficult to scan multiple job descriptions automatically if there are formatting differences.
2. Filters may not be comprehensive. There is still the possibility of missing out on candidates and also filtering in false positives.

Nicole2021 · Answer 2 · 2021-02-26T15:16:11+0000

Clarify:

- To clarify, false positives in this case means hiring a person who is not a good fit, and false negative means failing to hire someone who is a good fit?

- Is this issue specific to a particular team or geography?

Metrics:

- I'd recommend we focus on reducing the number of false negatives (people who are a good fit who we are passing over today) while not increasing the number of false positives (hiring people who are not a good fit). Does that sound good?

- We could measure progress on a more frequent basis but overall it may be wise to assess progress over the course of a year. Would it be reasonable to assume that a goal of reducion in false negatives by 20% in one year?

Users:

There are three types of potential user groups or stakeholders that I have in mind:

1. Google recruiters who are tasked with sourcing and managing the evaluation process for candidates from start to finish, and who would be the likely users of this product

2. Pospective candidates who have not applied but may be being passed over for outreach by recruiters

3. Candidates who have applied and who may be rejected during the process

Because recruiters are the ones whose KPI's may rely on number of hires or successful hiring, I would like to focus on them, as they have the largest pain points with the concern.

Does that sound good?

User journey and pain points

I'd like to consider what the journey may be for google recruiters today:

- post a role - PP: may be concerned about hiring someone quickly, or finding someone to fit specific criteria (M)

- proactively search out candidates who fit a certain profile - PP: number of people needing to outreach to with low response rate, hard to identify good people based on linkedin profile (H)

- review resumes from candidates who apply on their own - PP: resume often isn't a good proxy for the candidates fit, hard to discern good candidates from bad as most resumes look alike (H)

- manage the interview process for candiates - PP: hard to schedule candidates, many candidates drop out or are rejected in the early stages (H)

- check references for final stage candidates - PP: references often don't reveal true concerns about candiates, few get rejected here (M)

- accept/reject candidates along the way based on feedback and guidelines

Solutions:

- Train algorithim to identifiy characteristics of someone who may be a good fit for the role (need to be mindful of potential bias), to increase volume of potential candidates (Impact - H, Effort - H, Rating - Must have)

- Partner with businessess who are helping candidates prep to identify candidates who may be contributing content which can be used to evaluate their candidacy (Impact - M, Effort - M, Rating - Nice to have)

- Train algorithim to conduct the interview to remove potential bias in the interivew itself (Impact - H, Effort -H, Rating- Must have)

- Proactively advertise with universities / schools in lower tiers (Impact - M, Effort - M, Rating - Should have)

Recommendation

- Start with algorithim to identify to identify good fit, focusing on top of funnel, in a next release could have algorithim to conduct the interview in an unbiased way

- Success metrics:

Primary: % reduction in false negatives/baseline

Secondary: # of false negatives / stage of process, # of false postitives / stage of process, # of candidates / stage of process. Would like to ensure algorithim is not biased by a third party review of code or of candidates hired .

Summarize:

- Goal is to reduce false negatives

- focusing on recruiters as the primary user

- identified user journey with pain points spanning the hiring process

- identified and prioritized 4 potential solutions

- recommended one for the first release, and identified success metrics to evaluate the solution

Solution	Impact to Recruiter / Goal	Cost to Google
Historic Google Recruitment Review	Medium	Low
Filtering App	Low	Low
Key Word Match	High	Low
Background Review	Medium	High
Recruiter Testing	Medium	Low