Nanny ML
Last updated: 2022-11-29 Tuesday
Nannyml
Is a python package that can estimate performance of your ml model and can be found here.
NannyML is an open-source python library that allows you to estimate post-deployment model performance (without access to targets), detect data drift, and intelligently link data drift alerts back to changes in model performance. Built for data scientists, NannyML has an easy-to-use interface, interactive visualizations, is completely model-agnostic and currently supports all tabular use cases, classification and regression.
It does something called Confidence-based Performance Estimation (CBPE) that can be used to estimate model performance in absence of labels. It can also detect data drift.
what do I think about it
I did not look to deeply into the CBPE but a package you can plug into your ML pipeline seems great! It seems you still need to build something on top of the results to get alerts.