Google Cloud Associate Data Practitioner — knowledge map
The 128 core concepts of Google Cloud Associate Data Practitioner and how they connect. Click a node in the map above to explore related terms and prerequisites; the list below indexes every concept with its definition and links to its prerequisites and related concepts.
Concepts (128)
BigQuery
Google Cloud serverless data warehouse for fast SQL analytics on petabyte-scale data.
Managed services (management boundary)
A categorization by how much AWS operates for you. The more fully managed (AWS handles patching/scaling/availability), the lower your operational burden but less control; unmanaged (e.g., EC2) is flexible but self-operated. It frames “where your responsibility ends” in the shared responsibility model.
Cloud Storage
Object storage for unstructured data such as images, video, and backups.
Performance metrics (IOPS/throughput/bandwidth/latency)
Core vocabulary for performance: IOPS = I/O operations per second (small random access, e.g., EBS io2); throughput = data volume per time (MB/s, large sequential transfer); bandwidth = link capacity; latency = delay per operation. Design and monitor by the metric that fits the workload.
Cloud Run
A service that runs containers serverless, scaling to zero when idle and billing only for usage.
Eventarc
A service that routes events from diverse sources (Pub/Sub, Cloud Storage changes) uniformly to Cloud Run/Cloud Functions.
Prerequisites: Cloud Functions、Cloud Run、Cloud Storage、Pub/Sub
Workflows
Serverless orchestration for lightweight cross-service step chaining (simple workflows); heavy dependencies use Cloud Composer.
Prerequisites: Cloud Composer
Compute Engine
Google Cloud IaaS providing virtual machines (VMs) with selectable CPU/memory and OS-level control; ideal as a lift-and-shift target.
Prerequisites: Migration strategy (lift-and-shift/improve-and-move/refactor)
Storage Transfer Service
A managed service to transfer large or continuous data into Cloud Storage; for small data, the gcloud/bq CLI is simpler.
Prerequisites: Cloud Storage、Managed services (management boundary)
Cloud Composer
Managed orchestration based on Apache Airflow; schedules and runs multi-step pipelines with dependencies.
Cloud KMS
A managed key-management service to create and manage encryption keys and control rotation/disabling; the basis of CMEK.
Related: Encryption keys (GMEK / CMEK / CSEK)
Dataproc
A managed service to run existing Spark/Hadoop workloads; differs in purpose from Dataflow (Beam).
Prerequisites: Dataflow、Managed services (management boundary)
Encryption keys (GMEK / CMEK / CSEK)
GMEK = Google-managed default keys; CMEK = customer-managed keys via Cloud KMS (control rotation/disabling); CSEK = customer-supplied keys. Data is encrypted at rest and in transit by default.
Prerequisites: Encryption at rest、Encryption in transit
Related: Cloud KMS
Scheduled queries
A simple BigQuery feature to auto-run the same query (e.g., aggregation) on a schedule; complex dependencies use Cloud Composer.
Prerequisites: Cloud Composer、BigQuery
Cloud Logging
A Google Cloud Observability capability for collecting, searching, and retaining logs, used for investigation and audit.
Related: Three observability pillars (metrics, logs, traces)
Cloud Monitoring
A Google Cloud Observability capability for metrics monitoring, dashboards, and alerts.
Related: Three observability pillars (metrics, logs, traces)
Cloud SQL
A managed relational database compatible with MySQL/PostgreSQL/SQL Server, for typical business apps.
Spanner
A distributed relational database combining global strong consistency and high availability, ideal for global core systems.
Prerequisites: Strong consistency
Encryption at rest
Encrypting data while it sits on disk or object storage. Most managed cloud storage (block/file/object/DB) offers built-in encryption integrated with the platform's key management (e.g., a KMS). A baseline defense protecting the data itself against theft or unauthorized access.
Prerequisites: Cloud Storage
Cloud Deploy
A managed continuous-delivery (CD) service for GKE, Cloud Run, and Anthos. It templates a delivery pipeline (e.g., dev → staging → prod) and provides deployment strategies like canary or blue-green, approval gates, and rollback as built-in features.
Prerequisites: Cloud Run、GKE Enterprise (Anthos)、Deployment strategies (canary/blue-green/rollback)
Related: Cloud Build
Service account
A special identity for workloads (VMs/apps) rather than humans; prefer short-lived tokens via metadata or Workload Identity Federation over downloaded keys.
Cloud Interconnect
A dedicated-connection service (Dedicated/Partner) linking on-premises to Google Cloud with high bandwidth, low latency, and an SLA; choose vs the simpler encrypted-tunnel Cloud VPN by requirements.
Prerequisites: Cloud VPN、Performance metrics (IOPS/throughput/bandwidth/latency)
Vertex AI AutoML
A Vertex AI capability that auto-builds high-quality custom ML models from tabular, text, image, or video data with minimal code. It runs neural architecture search and hyperparameter tuning under the hood to produce a model balancing accuracy against training cost. For quick SQL-only analysis, BigQuery ML can be the simpler choice.
Prerequisites: BigQuery、BigQuery ML、Vertex AI
Related: Tabular Workflows
Tabular Workflows
A feature that decomposes the tabular AutoML pipeline into components on Vertex AI Pipelines, letting you customize and reuse individual stages such as feature engineering or architecture search. Where plain AutoML is more black-box, Tabular Workflows targets advanced use cases needing fine control or scale over specific stages.
Prerequisites: Workflows、Vertex AI、Feature engineering and embeddings/RAG
Related: Vertex AI AutoML
AlloyDB
A high-performance, PostgreSQL-compatible managed database.
Ops Agent
An agent installed on Compute Engine or on-prem VMs that collects detailed system metrics (CPU/memory/disk, etc.) and logs into Cloud Monitoring / Cloud Logging. It unifies the legacy separate Monitoring agent and Logging agent—use this for new deployments.
Prerequisites: Cloud Logging、Cloud Monitoring、Compute Engine
Cloud Data Fusion
A managed service to build ETL/ELT pipelines visually with little code.
Prerequisites: Managed services (management boundary)
Lifecycle rules
Cloud Storage rules to automate "delete after N days" or "move to a cheaper class after N days," optimizing cost; in BigQuery, expiration plays a similar role.
Prerequisites: BigQuery、Cloud Storage
Transfer Appliance
An offline transfer service that ships large data on a physical device for ingestion into Cloud Storage; suits volumes where network transfer is impractical.
Prerequisites: Cloud Storage
Bigtable
A wide-column NoSQL database for huge-scale, low-latency workloads such as IoT and time series.
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Cloud Functions
A serverless functions service for event-driven small tasks, such as running when a file is uploaded.
Dataflow
A data-processing pipeline service supporting both batch and streaming.
Firestore
A scalable document NoSQL database for mobile and web app data.
Vertex AI
A unified AI platform to build, train, deploy, and operate ML models end to end, including using foundation models.
Encryption in transit
Encrypting data while it moves across the network, typically protected with TLS. Only by pairing it with encryption at rest do you cover a piece of data's whole lifecycle (stored and moving)—satisfying both is a baseline security requirement.
Prerequisites: Encryption at rest
ETL (extract, transform, load)
A data-integration pattern that extracts data from a source and transforms it (cleansing, reshaping, aggregating) before loading it into the destination. It needs separate transform compute (a server or managed service), but only clean, shaped data ever lands in the destination.
Prerequisites: Managed services (management boundary)
API Gateway (Google Cloud)
A fully managed API gateway in front of serverless backends (Cloud Run/Functions/App Engine); defined with OpenAPI to handle auth, keys, and monitoring.
Prerequisites: App Engine、Cloud Run、Managed services (management boundary)
Cloud Build
A serverless CI service that automates fetching source, building, testing, and containerizing images. Build steps are defined in cloudbuild.yaml and can be triggered by pushes to GitHub/Cloud Source Repositories; artifacts are typically stored in Artifact Registry.
Prerequisites: YAML
Related: Cloud Deploy、Artifact Registry
Filestore
A fully managed NFS file storage that mounts as a shared file system from Compute Engine VMs or GKE, providing low-latency shared storage.
Prerequisites: Compute Engine、Managed services (management boundary)、Performance metrics (IOPS/throughput/bandwidth/latency)
JSON
A lightweight text format representing structured data as braces and key–value pairs. Widely used wherever machine-readability matters most—API request/response bodies and permission definitions like IAM or bucket policies.
Deployment strategies (canary/blue-green/rollback)
Canary = release to a subset first; blue/green = instant switch between old/new environments; rollback = revert quickly on issues. Risk-limiting release techniques.
VPC Service Controls
A mechanism that prevents data exfiltration to managed services (e.g., BigQuery, Cloud Storage) via a service perimeter; a layer of control separate from IAM grants.
Prerequisites: BigQuery、Cloud Storage、Managed services (management boundary)
Cloud CLI emulators
Local mocks (Firestore, Pub/Sub, Spanner, etc.) provided by the Google Cloud CLI to speed local development/unit testing without connecting to the cloud (not a production substitute).
Three observability pillars (metrics, logs, traces)
Metrics (Cloud Monitoring, numeric trends), logs (Cloud Logging, events), and traces (Cloud Trace, request paths); correlate logs and traces by trace ID to pinpoint causes.
Related: Cloud Trace、Cloud Logging、Cloud Monitoring
Database Migration Service (Google Cloud DMS)
Google's managed service to migrate MySQL/PostgreSQL and others to Cloud SQL or AlloyDB with continuous replication, supporting minimal-downtime cutover.
Prerequisites: AlloyDB、Cloud SQL、Managed services (management boundary)
Cloud Router
A router that exchanges routes dynamically over BGP with on-prem/other networks; configure ASN, route priority/MED, and authentication, and control advertised/learned ranges with custom-advertised/learned routes.
Prerequisites: Route priority
Cross-Cloud Interconnect
A service that links another public cloud to Google Cloud over a dedicated physical link without the internet; use it for multicloud connectivity when you need more bandwidth/lower latency than Cloud VPN.
Prerequisites: Cloud VPN、Cloud Interconnect、Performance metrics (IOPS/throughput/bandwidth/latency)
BigQuery Data Transfer Service
A managed service that schedules loads from SaaS or other data warehouses into BigQuery; distinct in purpose from Database Migration Service (DB migration).
Prerequisites: BigQuery、Managed services (management boundary)
Strong consistency
A consistency model guaranteeing that a read after a completed write always returns the latest value. It typically costs more latency or throughput than eventual consistency, but is chosen where stale reads are unacceptable, such as inventory counts or balances. Consistency models are picked per use case based on this trade-off.
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Related: Eventual consistency
YAML
A human-friendly text format that expresses hierarchy through indentation. Favored for CloudFormation and other IaC tools and CI/CD pipeline config files, and interconvertible with JSON representing the same data.
Prerequisites: JSON
Cloud DNS
A Google Cloud service that manages internal/external name resolution (domain → IP).
Cloud NAT
A service that lets VMs without external IPs make outbound calls to the internet.
Cloud VPN
A service connecting on-premises or other networks to Google Cloud over an encrypted tunnel.
Managed instance group (MIG) and instance template
A Compute Engine group that creates identical VMs from an instance template (a VM blueprint), with autoscaling and self-healing.
Prerequisites: Compute Engine
Cloud Storage classes
Classes by access frequency: Standard (frequent)/Nearline (~monthly)/Coldline (~quarterly)/Archive (long-term rare). Less access = cheaper storage but pricier retrieval.
Prerequisites: Cloud Storage
Analytics Hub
A service to securely share/subscribe to BigQuery datasets with other organizations without making copies.
Prerequisites: BigQuery
Dataform
A service to develop and operate SQL-based transformations (ELT) inside BigQuery with dependencies and version control.
Prerequisites: BigQuery
Looker Studio
A visualization (BI) tool to easily create and share reports and dashboards.
Prerequisites: Looker
LookML
A modeling language in Looker to define data metrics and dimensions, keeping measures consistent.
Prerequisites: Looker
Uniform bucket-level access
A recommended Cloud Storage setting that manages permissions centrally via IAM at the bucket level instead of per-object ACLs—simpler and safer.
Prerequisites: Cloud Storage
App Engine
A fully managed web app platform (PaaS) to publish apps without managing servers, letting you focus on development.
Prerequisites: Managed services (management boundary)
BigQuery ML
A capability to create ML models and run predictions with only SQL on data in BigQuery, without moving the data.
Prerequisites: BigQuery
Gemini
Google Cloud core generative AI model for text generation, summarization, code assistance, and image understanding, embedded across services and Workspace.
Looker
A business intelligence (BI) tool that visualizes analytics in dashboards to support business use.
Pub/Sub
A messaging service that ingests events in real time and distributes them to multiple services; used for streaming ingestion.
Eventual consistency
A consistency model where a read right after a write may return a stale value, but all replicas eventually converge over time. It's often the default behavior of distributed data stores that prioritize availability and low latency, and using it for reads that don't need strong consistency buys throughput.
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Related: Strong consistency
Cloud Billing
Links a billing account to projects to track costs, set budget alerts, and export billing data to BigQuery.
Prerequisites: Billing account and budgets、BigQuery
Deployment Manager
Google Cloud's native Infrastructure as Code service (YAML/Jinja/Python templates). It is deprecated and reaches end of support on 2026-03-31; migrate to the Terraform-based Infrastructure Manager or another IaC tool.
Prerequisites: Infrastructure Manager、YAML
VPC firewall rules
Stateful rules that allow/deny traffic in/out of a VPC, evaluated by direction, priority, target (tags/service accounts), and source; hierarchical firewall policies exist at the org level.
Prerequisites: Service account、Cloud NGFW and hierarchical firewall
Gemini Cloud Assist
An AI assistant inside the Google Cloud console that analyzes logs, metrics, traces, and resource configuration to offer natural-language troubleshooting and architecture suggestions. Aimed at cutting the time operators spend investigating root causes across the console.
Prerequisites: Gemini
Related: Gemini Code Assist
Gemini Code Assist
An AI pair-programming feature that assists with code completion, generation, explanation, and review inside an IDE. It has a free individual tier and an Enterprise tier that can ground completions in a private codebase and add enterprise administration. Used as an extension for VS Code and JetBrains-family IDEs.
Prerequisites: Gemini
Related: Gemini Cloud Assist
Cloud External Key Manager (Cloud EKM)
Encrypts Google Cloud data using keys held in an external (own/third-party) key-management backend, raising key sovereignty beyond CMEK (customer-managed via Cloud KMS).
Prerequisites: Cloud KMS、Encryption keys (GMEK / CMEK / CSEK)
Cloud Trace
A distributed-tracing service that follows request paths (spans) across services to find latency bottlenecks; a different observability axis from metrics (Monitoring) or logs (Logging).
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Related: Three observability pillars (metrics, logs, traces)
Private Google Access
A setting that lets VMs without external IPs reach Google APIs (e.g., Cloud Storage) privately; distinct in purpose from general egress via Cloud NAT.
Prerequisites: Cloud NAT、Cloud Storage
Cloud Run revisions and traffic splitting
Each deploy creates an immutable revision; traffic splitting controls the percentage sent to each revision, enabling canary release and rollback.
Prerequisites: Cloud Run、Deployment strategies (canary/blue-green/rollback)
gRPC
A low-latency, typed (Protocol Buffers) RPC framework over HTTP/2, suited to internal service-to-service calls; choose REST for public web.
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Memorystore
A fully managed in-memory cache compatible with Redis/Memcached; caches frequent reads to reduce latency. Not a durable store.
Prerequisites: Managed services (management boundary)、Performance metrics (IOPS/throughput/bandwidth/latency)
Cloud CDN
A CDN enabled on an external Application LB backend that caches content at Google's edge; origins include MIG/Cloud Storage/Cloud Run/internet NEG. Drop stale content via cache invalidation on updates.
Prerequisites: Cloud Run、Cloud Storage
Cloud NGFW and hierarchical firewall
A next-generation firewall controlling in-VPC traffic; offers hierarchical policies inherited by org/folder, an effective policy evaluated by priority, and Enterprise-tier L7 inspection (IPS). Implement micro-segmentation with tags/service accounts.
Prerequisites: Service account
Network endpoint group (NEG)
A load balancer backend unit that groups container-native (GKE), serverless (e.g., Cloud Run), or internet endpoints; chosen vs managed instance groups (MIGs) by need.
Prerequisites: Managed instance group (MIG) and instance template、Cloud Run
VLAN attachment
The logical link between a Cloud Interconnect and a VPC (Cloud Router); after creating the physical Interconnect, a VLAN attachment carries traffic to the actual VPC.
Prerequisites: Cloud Interconnect、Cloud Router
BigLake
A storage engine that lets BigQuery uniformly access data (e.g., in Cloud Storage) without moving it, bridging the lake and warehouse.
Prerequisites: BigQuery、Cloud Storage
Billing account and budgets
The unit that pays costs; link projects to it (one account, many projects). Budgets and alerts only notify at thresholds and do not auto-stop spending.
GKE Autopilot
A GKE cluster mode where Google fully manages node provisioning, scaling, upgrades, and hardening. Billing is per-Pod, and you lose node-pool tuning and SSH access in exchange for minimal operational overhead.
Prerequisites: Managed services (management boundary)
Organization Policy
A mechanism that inherits and enforces constraints down the resource hierarchy for consistent org-wide rules (e.g., disallow creation outside certain regions); distinct from IAM grants.
Prerequisites: Resource hierarchy
VPC Network Peering
Directly connecting two VPC networks so they can communicate with each other.
Prerequisites: Virtual Private Cloud (VPC)
Amazon Managed Service for Prometheus (AMP)
A managed, Prometheus-compatible metrics ingestion/storage service. You query container-environment metrics (EKS/ECS, etc.) with PromQL while AWS handles scale and retention—commonly paired with Managed Grafana for visualization.
Prerequisites: Managed services (management boundary)
GKE Enterprise (Anthos)
A platform to operate and manage containers consistently across on-premises and multiple clouds, enabling hybrid and multicloud.
Resource hierarchy
The Google Cloud hierarchy of Organization → Folder → Project → resources; the project is the basic billing/permission unit and policies inherit down the hierarchy.
Sensitive Data Protection (Cloud DLP)
A service that automatically discovers, classifies, and masks sensitive data such as PII in stored data.
Spot VM
A Compute Engine VM offered at a deep discount in exchange for possible preemption; used for cost optimization of interruption-tolerant batch work.
Prerequisites: Compute Engine
Model Garden
A Vertex AI catalog to browse and choose among diverse foundation models (Google, open, third-party).
Prerequisites: Vertex AI
Classic VPN
A legacy, single-tunnel IPSec VPN gateway with no SLA guarantee. It comes in route-based (dynamic) and policy-based (static traffic-selector) variants, with the latter kept mainly for compatibility with specific peer network devices. Google deprecates it in favor of HA VPN for new builds.
Prerequisites: HA VPN
Cloud Code
An extension built into IDEs like VS Code and IntelliJ. It helps author Kubernetes manifests and Dockerfiles, deploy to GKE/Cloud Run directly from your local machine, and live-debug on a remote cluster—all from within the editor.
Prerequisites: Cloud Run
Cloud Endpoints
An API management service that manages OpenAPI/gRPC APIs via the Extensible Service Proxy, adding auth, monitoring, and quotas.
Prerequisites: gRPC
HA VPN
Google's recommended IPSec VPN, providing a 99.99% availability SLA via two redundant tunnels (each with its own external IP). Used to connect to on-prem, other clouds, or other VPCs, and assumes dynamic routing via BGP. The default choice for new deployments.
Virtual Private Cloud (VPC)
Google Cloud's software-defined network. A global resource that holds regional subnets in one network; routes, firewalls, and peering control traffic.
Workload Identity Federation (GCP)
A mechanism that trusts an external identity provider outside Google Cloud—AWS, Azure, on-prem, or CI/CD like GitHub Actions—and exchanges its issued tokens for temporary Google Cloud credentials. Used for external system integrations where you want to avoid downloading and distributing service-account keys.
Prerequisites: Service account
Dataplane V2
GKE's eBPF-based networking data plane. It enforces Kubernetes NetworkPolicy (Pod-to-Pod L3/L4 traffic control) efficiently at the kernel level and also provides visibility via flow logs. It has become the default for new clusters on recent GKE versions and runs with less overhead than the older iptables-based implementation.
IP masquerade (GKE)
A mechanism that source-NATs a GKE Pod's outbound (egress) traffic to the node's IP. By default, non-masqueraded (untranslated) destinations are RFC 1918 private ranges and link-local addresses; the ip-masq-agent configuration lets you add or change the target CIDRs. Whether this agent is auto-deployed as a DaemonSet, and its default configuration, depends on the GKE version and cluster setup (e.g., Autopilot vs. Standard, whether Dataplane V2 is in use).
Prerequisites: Dataplane V2
Horizontal scaling (scale out)
Scaling by adding more instances to spread the load. It assumes a stateless design plus a load balancer and auto scaling, and since one instance failing doesn't take down the whole system, it also improves fault tolerance. Considered the default direction in cloud-native design.
Infrastructure Manager
A Google Cloud service that runs Terraform as managed IaC, making infrastructure reproducible and reviewable/auditable via CI/CD and PRs.
Migration strategy (lift-and-shift/improve-and-move/refactor)
Lift-and-shift = move as-is (fastest, least change); improve-and-move = slight optimization; refactor = rebuild cloud-native (most effort, biggest payoff). Choose by requirements.
Application Default Credentials (ADC)
The standard way an app resolves credentials to authenticate to Google Cloud; combine with service accounts or Workload Identity Federation to avoid hardcoded keys.
Prerequisites: Service account
Artifact Registry
A registry to store and manage container images and language packages; the destination for artifacts built by Cloud Build.
Related: Cloud Build
Cloud SQL Auth Proxy
A proxy for secure Cloud SQL connections using IAM auth and encryption without exposing a public IP; the recommended way for apps to connect to the DB.
Prerequisites: Cloud SQL
Secret Manager
A managed service to securely store, retrieve, and rotate secrets (API keys, passwords, certificates); avoid embedding them in code.
Prerequisites: Managed services (management boundary)
Signed URL
A URL granting time-limited, scoped access to a Cloud Storage object without making it public; different from making a whole bucket public.
Prerequisites: Cloud Storage
Maintenance windows
A time window during which planned maintenance (patching, minor version updates) runs on a managed database. Set it to low-impact hours to align planned downtime with availability requirements.
Prerequisites: Windowing and late data (streaming)
Reverse replication
A mechanism that writes back from the new environment to the old after a migration cutover, providing rollback insurance to safely fall back if problems arise.
Prerequisites: Deployment strategies (canary/blue-green/rollback)
Synchronous/asynchronous replication
Synchronous replication gives RPO≈0 but adds write latency; asynchronous has lower latency but risks data loss on failure. Choose by RPO requirements.
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Artifact Analysis
Scanning that detects known vulnerabilities in container images/packages. Distinct from Binary Authorization (deploy-only-if-signed); combine detection with enforcement.
Prerequisites: Binary Authorization
Private Service Connect
A way to reach specific managed/published services over private IPs without traversing the internet; use it to avoid exposing public IPs.
Prerequisites: Managed services (management boundary)
Synthetic monitors
Actively and periodically probe endpoints/workflows from the user perspective (external monitoring); pair with passive monitoring to catch issues proactively.
Prerequisites: Workflows
Cloud DNS routing policies and split-horizon
Advanced Cloud DNS resolution: geolocation (vary by source region) and failover (switch to backup on primary failure) routing policies, plus split-horizon DNS resolving the same name differently via public/private zones.
Prerequisites: Cloud DNS
Dynamic routing mode (global/regional)
A VPC setting that determines how far BGP routes learned by Cloud Router propagate: regional stays within one region; global propagates to all regions. Use global to span multiple regions.
Prerequisites: Cloud Router
Hybrid DNS (forwarding zones/inbound policy/DNS peering)
Configurations for bidirectional name resolution between on-prem and Cloud DNS: forwarding zones query on-prem from cloud, inbound server policy resolves cloud from on-prem, and DNS peering references a zone in another VPC.
Prerequisites: Cloud DNS
Network Intelligence Center
A suite for network visualization, diagnosis, and optimization: Network Topology (visualize), Connectivity Tests (static route/firewall diagnosis), Performance Dashboard (packet loss/latency), Firewall Insights, Network Analyzer (automatic config diagnosis), and Flow Analyzer.
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Route priority
A value that decides which route is chosen when multiple share a destination; a lower value wins. Combine with network tags to scope where it applies.
Secure Web Proxy
A proxy that allowlists egress by URL/SNI; unlike Cloud NAT (general internet egress), it controls "where egress is allowed" at the application layer.
Prerequisites: Cloud NAT
BI Engine and materialized views
BI Engine accelerates BigQuery analytics in memory; materialized views precompute common aggregations. Distinct means to speed dashboards (combinable).
Prerequisites: BigQuery
BigQuery Editions and reservations
A way to secure compute capacity for steady use at predictable cost; on-demand suits small or irregular usage.
Prerequisites: BigQuery
Feature engineering and embeddings/RAG
Feature engineering shapes data for training/serving. For generative AI, turn unstructured data into embeddings and use retrieval-augmented generation (RAG) to fetch context for answers.
Partitioning and clustering (BigQuery)
Partitioning splits a table by e.g. date; clustering orders rows by frequently filtered columns. Both reduce scan volume to optimize BigQuery cost/performance.
Prerequisites: BigQuery
Windowing and late data (streaming)
In streaming, design windowing (time-based aggregation) and late-arriving data handling with watermarks and allowed lateness; batch processes accumulated data together.
Sensitive data protection (PII / masking)
Protecting sensitive data such as PII (personally identifiable information) and PHI: discover/classify with Macie, then protect via encryption, access control, and minimization. Masking/tokenization hides values, and handling varies by data classification; also prevent sensitive data leaking into logs.
Vertical scaling (scale up)
Scaling by upgrading a single instance's specs (CPU, memory, etc.). It requires little application change and is simple, but is capped by instance-type limits, tends to be a single point of failure, and often requires downtime to apply. The counterpart choice to horizontal scaling.
Prerequisites: Horizontal scaling (scale out)