What's changed: Created Professional Data Engineer Chapter 1 (Domain 1 "Design": security/compliance = IAM least privilege/org policies/GMEK-CMEK-Cloud EKM/Sensitive Data Protection/data residency/project-dataset-table design/multi-environment; reliability and migration = Dataform/Dataflow/Cloud Data Fusion, Cloud Composer, DR/fault tolerance/ACID/data validation, BigQuery DTS/Database Migration Service/Datastream/Transfer Appliance).
1.1Designing for security and compliance
Understand IAM and organization policies for data platforms, encryption at rest/in transit and key management, handling PII, data residency (sovereignty), governance via project/dataset/table architecture, and multi-environment (dev/prod) design.
A Professional Data Engineer designs robust data infrastructure to collect, transform, store, and serve data. Design starts with security and compliance—before performance or features.
1.1.1Access control and encryption
Design data access with IAM least privilege (avoid basic roles; predefined/custom), and enforce org-wide constraints (e.g., region limits) with organization policies. Data is encrypted at rest/in transit by default; choose keys by who manages them: GMEK (Google-managed)/CMEK (customer-managed via Cloud KMS)/Cloud EKM (external). Discover/mask/tokenize PII with Sensitive Data Protection (Cloud DLP). Map "least privilege = IAM," "manage keys yourself = CMEK," "PII protection = Sensitive Data Protection."
1.1.2Data governance and multi-environment
Data governance starts from project/dataset/table architecture (expressing who accesses what granularity via boundaries). Control storage geography with data residency to meet regulations (sovereignty). Separate dev and prod as multi-environments so production data is not misused in development. Map "govern data via hierarchy and boundaries," "storage-region regulation = data residency," and "prod/dev separation = multi-environment."
Common: requirement → means. E.g., "least privilege on data" = IAM predefined/custom; "org-wide constraints" = organization policies; "manage keys yourself" = CMEK (Cloud KMS); "external key backend" = Cloud EKM; "detect/mask PII" = Sensitive Data Protection; "store data in a country" = data residency; "separate prod and dev" = separate projects/environments.
Watch the mix-ups: (1) Data is encrypted by default—the point is who manages keys (GMEK/CMEK/EKM). (2) IAM (grants) and organization policies (enforced constraints) are different layers. (3) Do not copy production data to dev and leave it (a PII-leak hazard).
1.1.3Section summary
- IAM least privilege + organization policies; choose GMEK/CMEK/Cloud EKM by key management
- Discover/mask PII with Sensitive Data Protection; storage geography via data residency
- Governance via project/dataset/table design and prod/dev multi-environment separation
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. To design data access with least privilege, which is most appropriate?
Q2. To manage keys yourself with rotation/revocation control, which is best?
Q3. To detect PII and mask/tokenize it, what do you use?
Q4. For compliance, to store data in a specific country/region, which concept?
Q5. To enforce org-wide consistent constraints (e.g., disallow creation outside certain regions), what do you use?
Q6. Which data-platform design helps prevent misuse of production data?