Tell me about a time when you launched or brought a product to market. What challenges did you face?

Where we are at: In September F24 the FI I work at launched it's first virtual agent based on GenAI and serving >1k front line agents delivered by a cross-functional pod of engineers, data, product, science, business, second line and innovation and lead by my technical product managers.
Where we started: In the summer of F23 my team was engaged to 'lend' a couple machine learning eng to provide support to a new POC using GenAI in the call centre.

A couple months in some general pain points emerged in the ability to execute:
1. lack of familiarity with governance procedures and expectations
2. inability to coordinate across a matrix of teams
3. lack of clear unambiguous ownership for decisions and escalations
4. poor management of specialized resources (ie. asking MLE to build slides)
5. little understanding of the infrastructure and compute implications
6. No program level oversight that could articulate business case, prioritization criteria, cost, expected benefit etc

Decisions I owned and oversaw: 
1. assigned a senior product manager and a technical product owner to lead the day to day pod and coordinate across technical teams and articulated clear separation of resposnibilities between product and technical contributors
2. stood up an internal operations playbook that clearly articulated the ins/outs - ie. roadmaps and requirements - for core pods like governance, infrastructure, delivery etc and assigned clear ownership
3. set up an internal weekly steering committee to review requirements decisions as they had high cost centre implications and asked the sr pm to create a knowledge management playbook including typical timelines, expectations, phases, resourcing, costs as well as a template of external steering cadences

The two broad lessons learned were:

1.While we had developed and honed a strong delivery muscle for BAU/predictive AI, we were exposed for a lack of coordination in stumbling to deliver much more complex and matrix-like projects. Ownership of decisions and processes became critical as we stepped on each others toes.

2. Syncing up of roadmaps, requirements gathering and third party management became a web of reactive tactics. There was no template for ensuring that technology roadmaps would meet AI requirements which were changing daily and no communication channels established for gathering dev team input in key negotiations such as that of OpenAI.

Honorable mention:
We were able to trudge through a new way of engaging our second line. Traditionally we would not be able to work in an agile format with them as independence has always prevented this. Model validation is a critical activity and in the case of a new GenAI solution two key challenges emerged: 1. no documented process for change management ie. GPT instance frequent updates 2. no clear template for monitoring which would need to include a human element

Overall the lessons learned have positioned us well to deliver on the next tranche of in scope use cases and the expectation is that our playbook will enable consistency and speed to market on the heels of some trench delivery.