What's changed: Deepened DP-100 Chapter 2 (ja figures; comparison tables/scenarios/FAQ/traps/deep paragraphs in all sections)
2.2AutoML and Hyperparameter Tuning
Understand AutoML for automated model search, hyperparameter tuning (sweep jobs)—search space, sampling, early termination—and evaluation metrics/model selection. Find the best model efficiently.
Getting a good model requires algorithm selection and hyperparameter tuning. Azure ML offers AutoML (auto-search) and sweep jobs (tuning).
2.2.1AutoML and sweep
- AutoML: specify the task (classification/regression/forecasting); it auto-tries algorithms and preprocessing to pick the best model.
- Sweep job: define a search space and tune hyperparameters via sampling (grid/random/Bayesian).
- Early termination: stop unpromising trials early to save cost.
- Primary metric: pick the best model/config by a primary metric (accuracy, AUC, RMSE, etc.).
Common on DP-100: auto-search algorithm/preprocessing = AutoML, hyperparameter tuning = sweep job (search space + sampling), stop poor trials = early termination, best selection = primary metric. Know grid vs random vs Bayesian sampling.
Bayesian sampling chooses next candidates from past results (efficient), but may have constraints when combined with early termination.
AutoML takes a task (classification, regression, forecasting, also vision/NLP) and auto-searches featurization, algorithms, and hyperparameters, ranking on a leaderboard by a primary metric to pick the best model (configurable experiment timeout/number of trials, auto/custom featurization, allowed/blocked algorithms, ensembling). For manual sweep jobs, define the search space with choice/uniform/loguniform/normal, etc., and pick sampling: grid (all discrete combos; small spaces), random (efficient over large spaces), or Bayesian (chooses next from past results; efficient but constrained with early termination). Early-termination policies cut waste: Bandit (stop by slack from the best), median stopping, and truncation selection; max_total_trials/max_concurrent_trials control parallelism and cost. To avoid overfitting, evaluate with cross-validation or a validation set and select by the primary metric (accuracy, AUC, RMSE, normalized RMSE). Decide: “broad auto-search = AutoML,” “tune a specific model’s HPs = sweep,” “large space = random, smart = Bayesian, exhaustive = grid.”
| Sampling | How | Best for |
|---|---|---|
| Grid | All discrete combinations | Small search spaces |
| Random | Sample randomly | Large spaces efficiently |
| Bayesian | Choose next from past results | Efficient (early-term constraints) |
Scenario: quickly build a classification model on tabular data while getting the best within a cost cap. → First use AutoML with task=classification, primary metric=AUC, and caps on trials/timeout. To refine a specific algorithm, define a search space in a sweep job with random sampling + Bandit early termination, tuning max_concurrent_trials for parallelism and cost.
FAQ: Q. Grid vs random vs Bayesian? → A. Grid for small discrete spaces, random for large ones, Bayesian to narrow smartly (with early-termination constraints). Q. AutoML vs sweep? → A. AutoML auto-searches including algorithms; a sweep tunes a chosen model’s hyperparameters over a search space.
Trap: “grid sampling is best for a vast search space” is wrong—exhaustive grid suits small spaces; large spaces favor random or Bayesian. Also “early termination always loses the best model” is wrong—it only stops unpromising trials, controlled by thresholds.
2.2.2Section summary
- AutoML = auto model search, sweep = hyperparameter tuning
- Efficiency = early termination, selection = primary metric
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. Which auto-tries algorithms/preprocessing and picks the best model just from a task?
Q2. Which Azure ML job tunes hyperparameters via a search space and sampling?
Q3. What stops unpromising trials early to save cost?