What's changed: Added per-section figures (cert-figure-retrofit). New AI-300 Chapter 2 (Domain 2 "ML model lifecycle and operations": training orchestration = MLflow/AutoML/sweep-early termination/distributed training/training pipelines/model comparison; registration and versioning = MLflow models/feature retrieval spec/responsible AI evaluation/archiving; production deployment and monitoring = online-batch endpoints/progressive rollout-traffic split/safe rollback/data drift/retraining-alert triggers)
2.1Orchestrating model training
Understand experiment tracking with MLflow, automated machine learning (AutoML), notebook exploration, hyperparameter tuning (sweeps), running training scripts, distributed training for large/deep models, training pipelines, and comparing model performance across jobs.
Train models as reproducible, comparable jobs. AI-300 covers the full operations loop: "track experiments, explore optimal models, automate and distribute training, and compare results."
2.1.1Experiment tracking and AutoML
MLflow records experiment parameters, metrics, and artifacts (models) so jobs can be compared (AML integrates it natively). To efficiently find the best model, automated machine learning (AutoML) automatically tries combinations of algorithms, preprocessing, and hyperparameters. Do exploratory work in notebooks.
2.1.2Hyperparameter tuning and distributed training
Hyperparameter tuning (sweep jobs) tries values like learning rate and batch size using a search strategy (grid/random/Bayesian), and an early-termination policy stops unpromising trials to cut cost. For large models and deep learning, use distributed training (parallelize across nodes/GPUs) to shorten training time. These run as training scripts executed as jobs.
2.1.3Training pipelines and comparing models
Define preprocessing → training → evaluation as a training pipeline (combining components) for reproducibility and automation. After running multiple jobs, compare model performance using metrics recorded in MLflow and pick the best. The orchestration flow: "track (MLflow) → explore (AutoML/sweep) → pipeline → compare."
| Goal | Means | Key point |
|---|---|---|
| Track/compare experiments | MLflow | Params/metrics/artifacts |
| Auto-find best model | AutoML | Auto-try algorithms/prep/HP |
| Optimize hyperparameters | Sweep jobs (search + early termination) | Search while cutting cost |
| Speed up large/deep training | Distributed training | Parallel across nodes/GPUs |
| Reproduce/automate steps | Training pipelines | Combine components |
Watch the mix-ups: (1) AutoML (auto-explore model/prep/HP) vs sweep jobs (search over specified hyperparameters). (2) MLflow (tracking/comparison) is not training itself. (3) distributed training (speed) vs hyperparameter tuning (exploration for accuracy) have different goals.
Map requirement → means: "track and compare experiment metrics" = MLflow; "auto-find the best model" = AutoML; "efficiently search learning rate and cut cost" = sweep + early termination; "speed up deep learning across GPUs" = distributed training; "make preprocessing→evaluation reproducible" = training pipeline.
2.1.4Section summary
- Track/compare with MLflow, explore best model with AutoML, tune HP with sweeps + early termination
- Speed up large/deep training with distributed training; reproduce/automate with training pipelines
- Flow: track → explore → pipeline → compare by metrics
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. Which standard tool, natively integrated by AML, records experiment parameters, metrics, and artifacts for cross-job comparison?
Q2. Which automatically tries combinations of algorithms, preprocessing, and hyperparameters to find the best model?
Q3. Which job searches values like learning rate with a strategy and cuts cost via early termination of unpromising trials?
Q4. Which shortens training time for large/deep models by parallelizing across nodes/GPUs?
Q5. Which combines components to make preprocessing→training→evaluation reproducible and automatable?
Q6. Which correctly distinguishes AutoML from a sweep job?