What's changed: Created Professional Machine Learning Engineer Chapter 6 (Domain 6 "Monitoring": secure AI systems (theft/poisoning defense), Google Responsible AI (bias monitoring), fairness/readiness assessment, Vertex Explainable AI (feature attributions); Vertex AI Model Monitoring continuous evaluation, training/serving skew, feature attribution drift, performance vs baselines/simpler models/over time, common training/serving error monitoring).
6.2Monitoring and troubleshooting
Understand establishing continuous evaluation metrics (Vertex AI Model Monitoring, Explainable AI), monitoring for training-serving skew, monitoring for feature attribution drift, monitoring model performance against baselines/simpler models/over time, and monitoring for common training and serving errors.
Production models degrade quietly. Monitor continuously to detect skew and drift early and trigger retraining or correction.
6.2.1Continuous evaluation and skew/drift
Continuously evaluate production models with Vertex AI Model Monitoring. Two key targets: training/serving skew (input distributions differ between training and serving) and feature attribution/prediction drift (input distributions/contributions change over time). Detect skew by comparing "training vs serving data," and drift by comparing "past vs current serving data." On exceeding thresholds, alert and trigger retraining. Map "training vs prod input mismatch = training/serving skew" and "prod input change over time = drift."
Continue reading — free sign-up
You're reading the free preview. Sign up free to read this section in full, plus every chapter (including 4+) and all questions.