What's changed: Created Professional Data Engineer Chapter 4 (Domain 4 "Prepare and use for analysis": visualization prep = Looker/Data Studio, materialized views/BI Engine, slow-query handling, data masking/IAM/Cloud DLP/column-row access; AI-ML prep and sharing = BigQuery ML/feature engineering, embeddings/RAG, Analytics Hub).
4.2Preparing data for AI/ML and sharing
Understand data prep for feature engineering and training/serving (BigQuery ML), preparing unstructured data for embeddings and retrieval-augmented generation (RAG), and sharing data (rules, publishing datasets, Analytics Hub).
Beyond analytics lie AI/ML and data sharing. Data engineers prepare ML-ready data and share data safely within and across organizations.
4.2.1Preparing data for AI/ML
For structured-data ML, BigQuery ML is convenient—build models with only SQL inside BigQuery (feature engineering, training, prediction in SQL). For generative AI, prepare a foundation by turning unstructured data (documents) into embeddings and using retrieval-augmented generation (RAG) to fetch relevant context for answers. Map "ML with only SQL in BigQuery = BigQuery ML" and "use unstructured data with gen AI = embeddings/RAG."
4.2.2Sharing data
Share by defining sharing rules, publishing datasets, and sharing reports/visualizations. To share BigQuery data without copying with other organizations internally/externally, use Analytics Hub (publishers expose listings; subscribers subscribe and reference). This avoids duplication and sync overhead. Map "copy-free data sharing = Analytics Hub."
Common: requirement → means. E.g., "ML with only SQL in BigQuery" = BigQuery ML; "search/answer over documents with gen AI" = embeddings + RAG; "share BigQuery data without copies across orgs" = Analytics Hub; "shape features for training" = feature engineering.
Watch the mix-ups: (1) BigQuery ML (ML in SQL) is a convenient option distinct from full custom training (Agent Platform). (2) Analytics Hub shares without copies, unlike data-duplicating transfers. (3) RAG uses embeddings + retrieval to ground answers.
4.2.3Section summary
- Structured ML = BigQuery ML (SQL); for gen AI prepare embeddings + RAG
- Feature engineering shapes data for training/serving
- Copy-free data sharing via Analytics Hub (publish/subscribe)
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. To build ML models with only SQL on BigQuery data without moving it, which is best?
Q2. To securely share BigQuery data with other organizations without copies, which is best?
Q3. To use unstructured data like documents for gen-AI search/answers, what foundation do you prepare?
Q4. Which correctly describes preparing data for training/serving?
Q5. Which best describes a key benefit of Analytics Hub?
Q6. Which correctly relates BigQuery ML and full custom training on Agent Platform?
Keep track of your progress
The full study guide is free to read. Sign up free to practice with the question bank, track what you have read, review your mistakes, and highlight passages.