What's changed: Created Professional Machine Learning Engineer Chapter 2 (Domain 2 "Collaboration": org data sources (Cloud Storage/BigQuery/Spanner/Cloud SQL/Spark/Hadoop), Agent Platform datasets, preprocessing (Dataflow/TFX/BigQuery), Feature Store on Gemini Enterprise Agent Platform, PII/PHI privacy; Jupyter backends (Gemini Enterprise Agent Platform Workbench/Colab Enterprise/Dataproc), security, frameworks (TensorFlow/PyTorch/sklearn/Spark/JAX), Model Garden, experiment tracking (Experiments on Agent Platform/Kubeflow Pipelines/TensorBoard), gen AI evaluation).
2.1Exploring and preprocessing organization-wide data
Understand exploring organization-wide data (Cloud Storage, BigQuery, Spanner, Cloud SQL, Apache Spark, Apache Hadoop), organizing data types (tabular/text/speech/image/video), managing datasets in Agent Platform, preprocessing (Dataflow, TensorFlow Extended [TFX], BigQuery), creating/consolidating features in Feature Store on Gemini Enterprise Agent Platform, privacy of data usage (PII/PHI), and ingesting data into Agent Platform for inference.
ML quality is decided by data. Explore data scattered across the org, preprocess it efficiently, and shape it into reusable features—considering privacy from the start.
2.1.1Data sources and preprocessing
Data lives in diverse places: Cloud Storage (unstructured/files), BigQuery (analytic tables), Spanner/Cloud SQL (operational DBs), and Apache Spark/Apache Hadoop (existing big data). Organize types (tabular/text/speech/image/video) and manage them as Agent Platform managed datasets. For preprocessing, choose Dataflow (Apache Beam) for large-scale stream/batch transforms, TensorFlow Extended (TFX) for production ML-pipeline preprocessing, and BigQuery for SQL-native aggregation/features. Map "large-scale parallel preprocessing = Dataflow" and "preprocess in SQL = BigQuery."
2.1.2Feature Store and privacy
Consolidate computed features in Feature Store on Gemini Enterprise Agent Platform to reuse across teams/models, using the same features for training and serving to prevent duplicate computation and training/serving skew. Consider privacy from the start: when handling PII (personally identifiable information) or PHI (protected health information), protect it with Sensitive Data Protection (DLP) masking/pseudonymization, least privilege, and VPC Service Controls perimeters. Map "feature reuse and consistency = Feature Store" and "protect sensitive data = redact PII/PHI."
Common: requirement → means. E.g., "large-scale stream/batch preprocessing in parallel" = Dataflow; "aggregate to features in SQL" = BigQuery; "preprocessing components in a production ML pipeline" = TFX; "reuse features across teams, consistent in training/serving" = Feature Store on Gemini Enterprise Agent Platform; "protect data with PII/PHI" = DLP (Sensitive Data Protection) + least privilege.
Watch the mix-ups: (1) Dataflow (parallel big-data transforms) vs BigQuery (SQL aggregation) are different preprocessing means—choose by scale and shape. (2) Without a Feature Store, features drift between training/serving (a skew source). (3) Redact PII/PHI early—after-the-fact protection is risky.
2.1.3Section summary
- Preprocess = Dataflow (parallel)/BigQuery (SQL)/TFX (pipeline); data = Agent Platform managed datasets
- Feature reuse and training/serving consistency = Feature Store on Gemini Enterprise Agent Platform
- Redact PII/PHI early with DLP; protect with least privilege/perimeters
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. To preprocess large log data in parallel for both stream and batch (transform/clean), which is best?
Q2. To reuse computed features across teams/models and use the same values in training and serving to prevent skew, which is best?
Q3. To build features by SQL aggregation from tables already in BigQuery, which is best?
Q4. Data with PII/PHI will be used for training. What should you do first?
Q5. To define preprocessing components consistent across training and serving within a production ML pipeline, which is best?
Keep track of your progress
The full study guide is free to read. Sign up free to practice with the question bank, track what you have read, review your mistakes, and highlight passages.