In the legal or medical vertical, which one would you choose first for an AI/ML project, and why?

Question

In the legal or medical vertical, which one would you choose first for an AI/ML project, and why?

Asked at Google in the past year

195 views

Asked at

195 views

How to answer Technical questions

Interview Guide

Top Technical interview questions

Imagine you're the product manager for Facebook Marketplace. Since many sellers don't mark items as sold, what existing functionality and metrics could you use to determine whether an item has likely sold?7 answers | 20.9k views
What happens when you enter a URL in your browser?6 answers | 10.8k views
How does TinyURL work?5 answers | 317k views
See Technical PM Interview Questions

Invite members
Invite by email

Add another

vparam · Answer 1 · 2024-05-29T22:46:22+0000

Assumptions
- We are technology company - yes
- We have the AI expertise and funded for creating a vertical solution - yes
- We have access to datasets across both the verticals and that would not be a limiting factor - Yes, that could be assumed.
Product description:
- Develop a model that would have high accuracy on supporting the selected vertical to increase the productivity of the folks in that field
Problem statement
- Determine which vertical to invest first and why that would be the right choice.
Product Attributes
- Medical
  - Data collection and processing
    - Data collection would be challenging as it requires patient consent on using the data to train/ model.
    - processing the data would need to be secure that sometimes would also restrict the access to someor most of the developers of the solution leaving them with test / valdiation limitations
  - Model configuration
    - Configuration of the model will chellenging as human level reasoning in medical field there are many common and specific elements. The number of transformer heads and parameters are extensive
    - Human feedback would need to be extensively used.
  - Bias and Hallucinations
    - The dataset could be biased that it could be a reflective of the most complex cases vs normal vs worngly diagnosed cases. This could introduce bias on the response that could have life / death consequences to the users of the system
  - Quality control and monitoring
    - it has been noted the meeting the quality metrics for medical related LLm has been difficult with med-plam2 model at 67.2% is one of the best. So to get to high 70-80% is a very diffcult task to achieve especially to be considered a vertical leader.
- Legal
  - Data Collection and Processing
    - Legal research material are publically available as they are needed for other cases. It can procured free from public domain or from specialized legal research documents
  - Model configuration
    - Configuration of the model is easy as they can be weighted easily against present judgement/ outcome. Also there are many outsoruced legal supporting organization who do these types of manual work for a fee that is readily available
  - Bias and Hallucinations
    - Bias is very limited as the judgemtns of existing cases cannot be biased for folks to have trust.
    - Hallucinations are big concerns in legal feel as the models could produce outputs that is entirely made up based on the data available. Extreme precaustion and validation is needed
  - Quality control and monitoring
    - We could apply automated and manual monitoring through a human validation process that could be available throught he outsourced model. This could be backcheck/ backwalk mechnisim that is initially offered to ensure that outcomes are human verified and model subsequently be reinforced learning is extensive
Recommendation
- I would recommend going with the legal vertical as the first project as the success of model outcome could be higher and the learning from the customers who are very vocal and strive to drive the point that they are "correct" could help us understand better on biases and address hallucinations challenges
Trade off
- Market already has cheaper and more humn driven process. Existing legal solution/ libraries could be the market and they could be having solution that would be preffered
- market could be the low end type of legal work that might solve interesting / cutting edge solution that would help better mankind but more focussed on producivity side.