What's changed: Created Professional Machine Learning Engineer Chapter 4 (Domain 4 "Serving": batch/online inference (Agent Platform/Dataflow/BigQuery ML/Dataproc), frameworks (PyTorch/XGBoost), Model Registry on Gemini Enterprise Agent Platform, A/B testing (traffic splitting); Feature Store on Gemini Enterprise Agent Platform online serving, public/private endpoints, hardware (CPU/GPU/TPU/edge), Vertex AI Prediction autoscaling/containerized serving, latency/throughput optimization via quantization/distillation/pruning).
4.1Serving models
Understand batch and online inference (Agent Platform, Dataflow, BigQuery ML, Dataproc), serving with frameworks (PyTorch, XGBoost, etc.), organizing a model registry, and A/B testing model versions.
Serving turns a trained model into value. Choose the inference mode that fits the use, manage versions, and switch safely.
4.1.1Batch and online inference
Choose inference by use. For low-latency, one-at-a-time responses, use online inference (Agent Platform endpoints); for periodic bulk processing of large data, use batch inference (Agent Platform batch prediction; Dataflow for large transforms; BigQuery ML ML.PREDICT if data is in BigQuery; Dataproc for existing Spark). Serve with the training framework (PyTorch, XGBoost, TensorFlow) or a custom container. Map "instant single-item response = online inference," "periodic bulk = batch inference," and "predict directly on BigQuery data = BigQuery ML."
4.1.2Model registry and A/B testing
Organize model versions with the Model Registry on Gemini Enterprise Agent Platform, centralizing versions, metadata, and deployment status. To release a new version safely, use A/B testing (traffic splitting) to split production traffic between old/new, compare performance/business metrics on real traffic, then fully migrate—rolling back instantly on issues. Map "centralized model versioning = Model Registry on Gemini Enterprise Agent Platform" and "compare a new version on real traffic = A/B testing (traffic splitting)."
Common: requirement → means. E.g., "low-latency instant response to user requests" = online inference (Agent Platform endpoints); "bulk-infer large data nightly" = batch prediction/Dataflow; "predict on BigQuery data via SQL" = BigQuery ML; "centralize model versions" = Model Registry on Gemini Enterprise Agent Platform; "compare new vs old on real traffic" = A/B testing (traffic splitting).
Watch the mix-ups: (1) Choose online inference (low latency, always-on, costlier) vs batch (high throughput, cheaper) by use. (2) Model Registry is versioning—distinct from deployment itself. (3) Don't fully switch a new version at once—validate with A/B first.
4.1.3Section summary
- Instant single = online inference (Agent Platform endpoints); periodic bulk = batch prediction/Dataflow/BigQuery ML
- Centralized model versioning = Model Registry on Gemini Enterprise Agent Platform
- Validate new versions with A/B testing (traffic splitting) on real traffic before migrating
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. To return predictions one-at-a-time with low latency in response to user requests, which is best?
Q2. To generate predictions over hundreds of millions of records in bulk nightly (no real-time need), which is best?
Q3. To centrally manage multiple model versions with metadata and deployment status, which is best?
Q4. To route part of production traffic to a new model version and compare with the old before full migration, which is best?
Q5. Data is in BigQuery and you want batch inference over many records via SQL. Which is best?
Keep track of your progress
The full study guide is free to read. Sign up free to practice with the question bank, track what you have read, review your mistakes, and highlight passages.