What product leaders need to do for AI to succeed
Over the past 9 years, I’ve had the opportunity to work on different aspects of AI. Mostly platforms for developers and data scientists to build AI, but also AI-products for end-users. At IBM I learned that the main problem with making AI work in the real world was the data, or so I thought. AI needs clean, labeled data. Lots of it, and even for IBM, access to large amounts of labeled-data was always the bottleneck.
When I joined Figure Eight, a data labeling company later acquired by Appen, as Director of product, I received a masterclass in AI development. Not just because the Figure Eight team is extremely knowledgeable in this space, and frankly a kickass group of people, but because as data providers, we had the privilege to serve the who-is-who of the AI industry. I got to talk with AI teams at Amazon, Adobe, and more, about their projects and even their struggles in making it work. I understood that while labeled data is a huge problem that AI teams need to overcome, the main challenge is far more complicated, and less obvious than just amassing large quantities of data.
To understand this problem, I had to stop building products for AI teams and start building AI-products for end-users.
A little bit more than a year ago I joined Procore as Director of Product for data and AI. At Procore I have learned other lessons that I think would be helpful to other product leaders who want to introduce AI into their product.
Without good quality data, AI is a non-starter
Data is and will be the biggest barrier to entry for most teams, so even before talking about product leadership, we have to talk about data.
Access to all data: AI engineers need access to all data. Not just product data, but also customer data, and corporate data. If your organization has not started an effort to aggregate, clean, organize, and build pipelines for your data scientists and ML engineers, then your AI efforts are essentially handicapped. This is the first place to start, and it takes a while, so start now.
- Recommendation: product leaders need to build data platforms that aggregate all the data the company has into one system. This data platform should be in the Product and Technology (P&T) organization because the mission is to serve the product, not the company.
Garbage in — Garbage Out: A lot of data is always useful, ML models are data-hungry and this hunger is not going away anytime soon, but the pursuit of quantity hinders teams from focusing on what matters. What matters are the inputs necessary to train each ML model. Take Uber Eats as an example: to predict the ETA of any delivery, their ML model needs to be trained with very specific inputs such as type of food, distance, beginning and end of a ride, and more. If the app did not capture this data automatically, then their AI developers would not have had enough inputs to work with.
- Recommendation: product leaders need to focus on building products that capture the inputs needed for the AI to work. A data strategy focuses on capturing the data that matters.
Data is constantly changing: It would be much easier if we could just train a model. Set it, and forget about it. Fortunately, the world keeps moving, we move with it, and changes are bound to occur in the real world. This change could render a model obsolete in no time. Taking Uber Eats as an example again, if construction happens on a major road, or if a pandemic forces restaurants to change their way of preparing food, the old inputs used to train the model have to be updated, and the model needs to be re-trained.
- Recommendation: design your app for data drift. Leverage your users to create additional inputs for the model to learn from. The industry calls this “Human-in-the-loop”.
Product Leaders must create an environment for AI to succeed
AI is not just a technology: A product that uses AI can change the way users perceive and interact with it. The model itself may become the IP that positions the company at an advantage over competitors. In this way, AI and Mobile are very similar. No product team builds the same way for Mobile as they do for Desktop. Mobile changed everything. It changed the way the team organized, prioritized, developed, designed, and launched to customers. AI is not very different.
- Recommendation: organize your teams to consider AI from the beginning of a product feature. AI should be part of their opportunity assessment. Product leaders, that is, Design, Engineering, and Product Managers should work with Data Scientist to define use cases.
Don’t put AI in the corner: the technical skills to build AI are specific, just like mobile developers specialize in Android or IOS, AI developer may specialize in a type of data input, or problem space, such as NLP or Computer Vision. These developers are a new introduction to the product team, but beyond that, the skills needed to build with AI are no different than the skills needed to build great products.
- Recommendation: the team that develops AI should not be special. AI should not be the technology of “innovation labs”. AI developers should work with the product teams from inception to production, and support. The product team should own the success and failure of the model.
AI will fail: let’s face it, we are trying to build a system that learns and performs autonomous tasks. This is not how we typically design and build software. The points of failure for AI are many, but the upside is indisputably great to ignore. Product leaders must have the end goal in mind, use storytelling to share their vision, and they must work with leadership to build additional space for failure. When companies invest in AI, they usually do so with great, but sometimes unachievable expectations.
- Recommendation: AI products should not be on a critical path nor they should be high integrity commitments. Product leaders need to exercise their ability to share a vision and continuously create space for their team to experiment, fail and learn.
Put AI to work in what matters: Product teams must have a product strategy. This product strategy should have a North Star Metric (NSM). AI, just like Mobile in many cases, is a powerful tool to execute the strategy and advance the north star metric. While AI should not be on a critical path, it should not be deployed to problems that don’t matter to the product strategy. AI is just too expensive to waste and nothing motivates a product team with a tough challenge like working on something that matters to the company.
- Recommendation: Product leadership must create a product strategy and identify a north star metric. When both of these things are in place product teams can identify opportunities for AI to become an enabler of the strategy. Always start small, but with substantial vision and conviction.
