What's changed: Added per-section figures (cert-figure-retrofit). New AI-300 Chapter 2 (Domain 2 "ML model lifecycle and operations": training orchestration = MLflow/AutoML/sweep-early termination/distributed training/training pipelines/model comparison; registration and versioning = MLflow models/feature retrieval spec/responsible AI evaluation/archiving; production deployment and monitoring = online-batch endpoints/progressive rollout-traffic split/safe rollback/data drift/retraining-alert triggers)
2.3Production deployment and monitoring
Understand deploying models as real-time/batch managed endpoints, testing and troubleshooting endpoints, progressive rollout and safe rollback, and in production configuring data drift detection, performance-metric monitoring, and retraining/alert triggers when thresholds are exceeded.
Deploy models as managed endpoints to production and detect and respond to degradation while running. Safe deployment plus post-deployment monitoring and automated response are AI-300 focuses.
2.3.1Online/batch endpoints and safe deployment
Choose inference by use case: online (real-time) endpoints for low-latency synchronous inference (API calls), batch endpoints for asynchronous inference over large data. Release new versions safely via progressive rollout (traffic split to shift traffic gradually = canary-like), and use safe rollback to revert to the prior deployment instantly on problems. Test and troubleshoot endpoints (check logs and scoring) before and after release.
2.3.2Data drift and retraining triggers
In production, detect data drift (input distribution shifting from training time). Monitor drift and performance-metric degradation (e.g., accuracy drop), and configure the system to automatically trigger retraining or alerts when thresholds are exceeded. This turns "deploy and forget" into a continuous MLOps loop that maintains quality.
| Goal | Means | Key point |
|---|---|---|
| Low-latency sync inference | Online endpoint | API call, real-time |
| Inference over large data | Batch endpoint | Asynchronous, scheduled |
| Safe new-version release | Progressive rollout (traffic split) | Shift gradually; canary-like |
| Recover from issues | Safe rollback | Revert to prior deployment |
| Detect quality degradation | Data drift / performance monitoring | Threshold → retrain/alert |
Watch the mix-ups: (1) online endpoint (real-time, low-latency) vs batch endpoint (large, asynchronous). (2) progressive rollout/traffic split (safe release) vs rollback (recovery on issues). (3) Detect data drift (input distribution change) and auto-trigger retraining/alerts on threshold breach.
Map requirement → means: "low-latency synchronous inference" = online endpoint; "infer over large data overnight" = batch endpoint; "release a new model gradually" = traffic split (progressive rollout); "revert to the prior version on issues" = safe rollback; "detect input distribution change and retrain" = data drift detection + retraining trigger.
For MLOps CI/CD, Azure DevOps is also an option. Manage source in Azure Repos and automate training/evaluation/deployment with Azure Pipelines. Like GitHub Actions, pipelines can trigger Azure Machine Learning jobs and endpoint updates; choose per your organization standard.
2.3.3Section summary
- Deploy: online (real-time) / batch (large, async) managed endpoints
- Safe release = progressive rollout (traffic split) + safe rollback; test/troubleshoot before and after
- Production monitoring = watch data drift/performance; auto-trigger retraining/alerts on threshold breach
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. Which deployment target best fits low-latency synchronous inference (API calls)?
Q2. Which fits asynchronous, scheduled inference over large data?
Q3. Which safely releases by gradually shifting traffic to a new model deployment?
Q4. When a new version has problems, which reverts to the prior deployment instantly?
Q5. Which detects whether production input distribution has shifted from training time?
Q6. In production monitoring, what should be auto-triggered when model performance falls below a threshold?