What's changed: Created Professional Machine Learning Engineer Chapter 2 (Domain 2 "Collaboration": org data sources (Cloud Storage/BigQuery/Spanner/Cloud SQL/Spark/Hadoop), Vertex AI datasets, preprocessing (Dataflow/TFX/BigQuery), Vertex AI Feature Store, PII/PHI privacy; Jupyter backends (Vertex AI Workbench/Colab Enterprise/Dataproc), security, frameworks (TensorFlow/PyTorch/sklearn/Spark/JAX), Model Garden, experiment tracking (Vertex AI Experiments/Kubeflow Pipelines/TensorBoard), gen AI evaluation).
2.2Notebooks and experiment tracking
Understand choosing the Jupyter backend on Google Cloud (Vertex AI Workbench, Colab Enterprise, notebooks on Dataproc), security best practices in Vertex AI Workbench, Spark kernels, code-repository integration, developing with frameworks (TensorFlow/PyTorch/sklearn/Spark/JAX), leveraging foundation/open-source models in Model Garden, tracking ML experiments (Vertex AI Experiments, Kubeflow Pipelines, Vertex AI TensorBoard), and evaluating generative AI solutions.
Prototype in notebooks and record experiments systematically. Trial-and-error without records cannot be reproduced.
2.2.1Notebook environments and frameworks
Choose the Jupyter backend by requirement: serious ML development = managed Vertex AI Workbench; collaboration/ease = Colab Enterprise; running on existing Spark/Hadoop = notebooks on Dataproc (Spark kernels). In Workbench, apply security best practices (least-privilege SA, private, user-managed vs managed) and version code via code-repository integration. Develop with familiar frameworks—TensorFlow, PyTorch, sklearn, Spark, JAX—and leverage foundation/open-source models in Model Garden. Map "managed ML dev environment = Vertex AI Workbench" and "use Spark = Dataproc notebooks."
2.2.2Experiment tracking and evaluating gen AI
For reproducibility, record experiments systematically. Compare params/metrics/artifacts with Vertex AI Experiments, visualize learning curves with Vertex AI TensorBoard (TensorFlow/PyTorch), and run pipeline-style experiments with Kubeflow Pipelines. For generative AI, beyond accuracy, perform generative AI evaluation (automatic metrics, human evaluation, LLM-as-judge for quality/safety/groundedness) and iterate. Map "record/compare experiments = Vertex AI Experiments," "visualize learning curves = TensorBoard," and "evaluate gen AI quality = eval metrics/human/judge."
Common: requirement → means. E.g., "managed Jupyter for serious ML dev" = Vertex AI Workbench; "easy collaboration" = Colab Enterprise; "on existing Spark" = Dataproc notebooks; "record/compare params/metrics" = Vertex AI Experiments; "visualize learning curves" = Vertex AI TensorBoard; "evaluate gen AI output quality" = gen AI evaluation (auto/human/judge).
Watch the mix-ups: (1) Vertex AI Experiments (record/compare) vs TensorBoard (visualize learning curves) play different roles—use both. (2) Do not neglect notebook security (broad SA, public IP)—least privilege/private. (3) Gen AI cannot be measured by accuracy alone—evaluate quality/safety/groundedness.
2.2.3Section summary
- Jupyter = Vertex AI Workbench (serious)/Colab Enterprise (easy)/Dataproc (Spark); secure Workbench
- Record/compare experiments = Vertex AI Experiments; learning curves = TensorBoard; pipeline experiments = Kubeflow Pipelines
- Evaluate gen AI for quality/safety/groundedness (auto metrics/human/judge)
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. For serious ML development in a managed Jupyter environment with least privilege and private networking, which is best?
Q2. To record params, metrics, and artifacts across runs and compare them later, which is best?
Q3. To visualize loss curves and accuracy over training as graphs, which is best?
Q4. To develop in notebooks with Spark kernels on an existing Spark/Hadoop cluster, which is best?
Q5. To evaluate a generative AI solution beyond accuracy—on quality, safety, and groundedness—which is best?