What's changed: Deepened DP-100 Chapter 3 (ja figures; comparison tables/scenarios/FAQ/traps/deep paragraphs in all sections)
3.2Monitoring, Retraining, and MLOps
Understand post-deployment monitoring (data drift, performance, inference logs), automating retraining, and MLOps (pipelines, CI/CD, model versioning). Keep models healthy over time.
A model isn’t done at deployment. You monitor input changes (drift) and performance and retrain as needed. Automating this is MLOps.
3.2.1Monitoring, retraining, MLOps
For optimizing language models (in DP-100 scope), use foundation models from Azure OpenAI or the model catalog, and for RAG grounding on enterprise data use Azure AI Search as the vector store/index. Evaluate with prompt flow and fine-tune as needed.
- Data drift: when production input distribution drifts from training, performance degrades; detect and address it.
- Monitoring: track inference performance, latency, and logs via Application Insights/Log Analytics.
- Retraining: retrain → evaluate → register → redeploy with new data (as a pipeline).
- MLOps: automate train/deploy/retrain via pipelines + Git/CI-CD + versioning.
Common on DP-100: input distribution shift = data drift, inference monitoring = Application Insights/logs, retrain → evaluate → register → redeploy with new data, automation = MLOps (pipelines + CI/CD + versioning). Data drift is a typical cause of performance degradation.
Retraining can be scheduled or triggered (drift/new data). With CI/CD, only validated models are promoted to production.
After deployment, monitoring is essential: track data drift (production input distribution shifting from training—a typical cause of degradation), concept drift (the input-output relationship itself changing), and inference performance/latency/error rate via Application Insights, Log Analytics (inference logs), and Azure ML model monitoring (data quality, feature-importance shifts). On detecting degradation, trigger retraining: with new data, retrain → evaluate → (if passing) register → redeploy. Triggers are scheduled (periodic) or event (drift detected / new data). Automating this is MLOps—running the ML lifecycle via pipelines + Git/CI-CD + model versioning/registry, promoting only validated models to production (approval gates). For reproducibility, version environments, data assets, and components, with approvals/rollback. Summary: “distribution shift = data drift,” “inference monitoring = App Insights/model monitoring,” “automation = MLOps.”
| Stage | What | Tools |
|---|---|---|
| Monitor | Track drift, performance, logs | App Insights / model monitoring |
| Retrain | Retrain → evaluate → register → redeploy | Pipeline jobs |
| Automate | Promote only validated models | MLOps (Git/CI-CD, registry) |
Scenario: a model’s accuracy gradually drops after months in production. → Check data drift in model monitoring to identify the input-distribution shift. Trigger a retraining pipeline (retrain on new data → pass evaluation criteria → register a new version → redeploy via traffic split). Automate this with MLOps on a schedule/trigger, promoting only validated models.
FAQ: Q. Typical cause of accuracy drop? → A. Data drift (production input-distribution shift); detect via monitoring and retrain. Q. The essence of MLOps? → A. Automating the ML lifecycle (train/deploy/retrain) with pipelines + CI/CD + versioning, promoting only validated models to production.
Trap: “once deployed, a model keeps constant accuracy” is wrong—it can degrade in production (e.g., data drift), requiring monitoring and retraining. Also “MLOps is just manual deployment” is wrong—it’s continuous automation via pipelines + CI/CD + versioning, promoting only validated models.
3.2.2Section summary
- Monitoring = data drift, performance, logs
- Retrain + automation = pipelines + MLOps (CI/CD, versioning)
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. What is the phenomenon where production input distribution shifts from training, degrading performance?
Q2. What is automating train/deploy/retrain with pipelines, CI/CD, and versioning called?
Q3. Which Azure service helps monitor a deployed model’s inference performance and logs?