Machine Learning Project Lessons Learned
This is a practical guide to building a lessons learned for a machine learning project project — what went well and badly, captured to improve future projects, adapted to the realities of building and deploying a machine-learning model.
What a Lessons Learned is
A lessons learned is what went well and badly, captured to improve future projects. For the full concept and how it works in general, see Lessons Learned. On a machine learning project project it plays the same role, tuned to this kind of work.
Why it matters for a Machine Learning Project project
Machine Learning Project projects live or die on building and deploying a machine-learning model. A well-built lessons learned gives the team a shared, explicit reference for exactly that — reducing ambiguity, aligning stakeholders, and making problems visible early enough to act. Skipping it, or doing it generically, is how machine learning project projects drift into avoidable delay and cost.
What to include
- What happened
- Impact
- Root cause
- Recommendation
Machine Learning Project-specific considerations
Tailor the lessons learned to the risks that most often derail machine learning project projects:
- Data labelling and quality
- Model drift
- Productionisation and monitoring
Example
On a real machine learning project project, the lessons learned would be shaped by building and deploying a machine-learning model. In particular, it should explicitly account for the project’s biggest risks — data labelling and quality, model drift, productionisation and monitoring — rather than treating them as afterthoughts.