What's changed: Deepened DP-100 Chapter 3 (ja figures; comparison tables/scenarios/FAQ/traps/deep paragraphs in all sections)
3.1Registering and Deploying Models (Endpoints)
Understand registering trained models (model registry) and deploying them to inference endpoints—real-time online endpoints and batch endpoints. Put models into production.
A trained, evaluated model is registered and deployed to an endpoint for inference, chosen by real-time vs batch.
3.1.1Registration and endpoints
For generative AI, also know Azure AI Foundry. It provides foundation-model deployment from the Model Catalog, prompt flow, evaluation, and content safety in one place, integrating with Azure Machine Learning to connect prototyping through operations.
- Model registration: register trained models in the registry with versioning.
- Online endpoint: real-time, low-latency predictions; managed online or Kubernetes.
- Batch endpoint: asynchronous scoring of large data on a compute cluster.
- Deployments and traffic: place multiple deployments under one endpoint and switch safely via traffic split (blue-green).
Common on DP-100: real-time/low-latency = online endpoint, async scoring of large data = batch endpoint, safe rollout = traffic split (blue-green), reuse/versioning = model registry. Choose online for immediate responses, batch for periodic bulk.
A deployment needs a scoring script (inference code) and an environment (dependencies). Managed online endpoints let Azure handle the infrastructure.
First register the trained model in the model registry with versioning (an MLflow model format can include environment/signature, sometimes avoiding custom inference code). Then choose an endpoint by use: managed online endpoint (Azure manages infra; real-time low-latency; autoscale), Kubernetes online endpoint (run on existing AKS), or batch endpoint (async scoring of large data on a compute cluster). Place multiple deployments under one endpoint and use traffic split for blue/green (old and new in parallel), gradual rollout, and instant rollback. A deployment needs a scoring script (init()/run()) and an environment, plus the right instance type/count (MLflow models may need no script). Secure it with managed identity for resource access, key/token auth, and private endpoints if needed. Summary: “immediate response = online,” “periodic bulk = batch,” “safe rollout = traffic split,” “reuse/versioning = registry.”
| Aspect | Online endpoint | Batch endpoint |
|---|---|---|
| Response | Real-time, low-latency | Asynchronous (job) |
| Workload | One/few immediate predictions | Bulk scoring of large data |
| Compute | Always-on (managed/AKS) | Cluster (only while running) |
| Rollout | Blue/green via traffic split | Run a specified version |
Scenario: release a new model to production, but a full switch is risky. → Add a new deployment to the online endpoint and route 10% to it via traffic split (blue/green) to observe. If healthy, ramp to 100%; if not, roll back instantly to the old version. For separate bulk-prediction needs, use a batch endpoint.
FAQ: Q. Online vs batch? → A. Immediate (low-latency) responses use online; periodic bulk scoring uses batch. Q. Safely switch to a new model? → A. Split traffic across deployments under one endpoint for blue/green gradual rollout and instant rollback.
Trap: “use a batch endpoint for real-time low-latency predictions” is wrong—immediate responses use an online endpoint; batch is for async bulk scoring. Also “switching to a new model must be all-at-once” is wrong—traffic split enables blue/green gradual rollout.
3.1.2Section summary
- Register = model registry (versioning)
- Deploy = online (real-time) / batch (async) + traffic split
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. You want real-time, low-latency predictions from a web app. Which endpoint?
Q2. You want to score large data asynchronously in a nightly batch. Which endpoint?
Q3. You want to roll out a new model version gradually and roll back if needed. What?