Skip to content

monitoring.md

Latest commit

 

History

History
16 lines (11 loc) · 792 Bytes

monitoring.md

File metadata and controls

16 lines (11 loc) · 792 Bytes
Raw
description How to monitor your machine learning system?

Monitoring

{% embed url="https://youtu.be/dYqhcXOi7JM" caption="Monitoring - Testing and Deployment" %}

Summary

  • It is crucial to monitor serving systems, training pipelines, and input data. A typical monitoring system can raise alarms when things go wrong and provide the records for tuning things.
  • Cloud providers have decent monitoring solutions.
  • Anything that can be logged can be monitored: dependency changes, distribution shift in data, model instabilities, etc.
  • Data distribution monitoring is an underserved need!
  • It is important to monitor the business uses of the model, not just its statistics. Furthermore, it is important to be able to contribute failures back to the dataset.