What's changed: Created Professional Machine Learning Engineer Chapter 3 (Domain 3 "Scaling": framework/architecture choice, interpretability, training data org, file ingestion, training SDKs (Agent Platform custom training/Kubeflow on GKE/AutoML/Tabular Workflows), distributed training, hyperparameter tuning (Vizier), troubleshooting training failures, fine-tuning foundation models; hardware (CPU/GPU/TPU/edge), TPU/GPU distributed training (Reduction Server/Horovod/TPU Pod)).
3.2Choosing training hardware
Understand evaluating compute and accelerator options (CPU, GPU, TPU, edge devices) and distributed training with TPUs and GPUs (Reduction Server on Agent Platform, Horovod).
Training speed and cost hinge on hardware choice. Pick the accelerator and distribution that fit the workload.
3.2.1Choosing accelerators
Choose accelerators by workload: CPU for light/preprocessing or small scale; GPU for general deep learning (diverse frameworks, small–mid scale); cost-efficient TPU for large matrix-heavy TensorFlow/JAX training; and edge devices (Coral/Edge TPU) for on-device inference. Evaluate by requirements (throughput, latency, cost, framework support). Map "large matrix-heavy TF/JAX = TPU," "general deep learning, broad frameworks = GPU," and "on-device inference = edge."
3.2.2Distributed training
Scale beyond a single device with distributed training on TPU/GPU. In multi-GPU/node data parallelism, gradient aggregation (all-reduce) is the bottleneck; Agent Platform Reduction Server offloads aggregation to dedicated nodes to speed communication. For framework-agnostic distribution, use Horovod. Scale large TPUs with TPU Pods. Map "speed up gradient aggregation in GPU distribution = Reduction Server" and "framework-agnostic distribution = Horovod."
Common: workload → hardware. E.g., "large cost-efficient TensorFlow/JAX training" = TPU; "deep learning across diverse frameworks (e.g., PyTorch)" = GPU; "on-device/offline inference" = edge (Edge TPU/Coral); "speed up gradient aggregation across GPUs" = Reduction Server; "framework-agnostic distribution" = Horovod; "light preprocessing/small scale" = CPU.
Watch the mix-ups: (1) TPU suits large matrix-heavy TF/JAX; GPU supports broad frameworks—choose by framework. (2) Reduction Server speeds GPU data-parallel aggregation, not TPU-to-TPU. (3) Edge is for inference—not large-scale training.
3.2.3Section summary
- Large TF/JAX = TPU; general deep learning = GPU; on-device inference = edge; light/small = CPU
- Speed GPU data-parallel aggregation = Reduction Server; framework-agnostic distribution = Horovod
- Evaluate by throughput/latency/cost/framework support
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. To train a large TensorFlow model (matrix-heavy) cost-efficiently, which accelerator is best?
Q2. To train deep learning across diverse frameworks (incl. PyTorch) at small–mid scale generally, which accelerator?
Q3. In multi-GPU data-parallel training, gradient aggregation (all-reduce) communication is the bottleneck. Which is best?
Q4. To run inference on-device (offline) in the field with unstable networks, which is best?
Q5. To implement multi-node distributed training in a framework-agnostic way, which is a common option?
Keep track of your progress
The full study guide is free to read. Sign up free to practice with the question bank, track what you have read, review your mistakes, and highlight passages.