Microsoft Fabric Data Engineer Associate — knowledge map
The 213 core concepts of Microsoft Fabric Data Engineer Associate and how they connect. Click a node in the map above to explore related terms and prerequisites; the list below indexes every concept with its definition and links to its prerequisites and related concepts.
Concepts (213)
Microsoft Fabric (capacity, workspace, experiences)
A SaaS analytics platform unifying data integration, analytics, and BI. Hierarchy: capacity → workspace → items. Capacity is the compute+billing unit (F SKUs, Power BI Premium P SKUs; consumes capacity units (CU); smoothing/bursting level usage; can pause/resume). A workspace is the collaboration/access unit. Experiences are role-specific (Data Engineering/Data Factory/Data Warehouse/Real-Time Intelligence/Power BI). Defining trait: all experiences share OneLake. It is a unified SaaS, not a bundle of PaaS.
Prerequisites: Azure Data Factory、Smoothing and bursting、Hierarchy (semantic model)
Related: Capacity (Fabric)、Experience (Fabric)、Real-Time Intelligence
Warehouse (Fabric)
A Fabric DW item supporting full T-SQL (read-write, multi-table transactions); data is stored as Delta in OneLake.
Related: Direct Lake、Lakehouse (Fabric)、OneLake
Fabric ingestion and transformation (pipelines, Dataflows Gen2, Spark, medallion)
Ingestion: data pipeline (orchestrate/schedule/retry activities like Copy/notebook), Copy activity (fast bulk movement; incremental via watermark), Dataflows Gen2 (no-code Power Query shaping), shortcuts (reference), Eventstream (streaming). Transformation: Spark notebooks (PySpark/Spark SQL; large/ML), T-SQL (Warehouse; set-based), Dataflows Gen2 (no-code). Design with the medallion architecture: bronze (raw) → silver (cleanse/join) → gold (aggregate/business-ready), storing each layer as Delta.
Prerequisites: Copy activity、Data pipeline (Fabric)、Dataflows Gen2、Microsoft Fabric (capacity, workspace, experiences)
Direct Lake
A Power BI semantic-model mode that reads OneLake Delta tables directly and fast—neither import nor DirectQuery; unsupported features fall back to DirectQuery.
Prerequisites: Delta table (Delta format)、DirectQuery mode
Related: Warehouse (Fabric)、Lakehouse (Fabric)、OneLake
OneLake
Fabric’s single unified storage per org (ADLS Gen2-compatible; tables in Delta), shared by all experiences to keep data in one place.
Prerequisites: Azure Data Lake Storage Gen2
Related: Direct Lake、Warehouse (Fabric)、Lakehouse (Fabric)、Delta table (Delta format)、Experience (Fabric)
Semantic model
Power BI’s meaning layer (formerly dataset)—tables, relationships, measures, hierarchies, and formatting that reports reference.
Data exploration and performance diagnostics (DAX/T-SQL/KQL, Performance Analyzer)
Explore with the right query language: DAX (semantic-model aggregation), T-SQL (SQL analytics endpoint / Warehouse tabular analysis), KQL (Real-Time streaming/logs), plus Power BI visual exploration (slicers/drilldown). For performance issues, measure per-visual DAX query/display time with Performance Analyzer, then dig in with DAX Studio or Tabular Editor’s Best Practice Analyzer, and inspect the storage-engine (VertiPaq, fast) vs formula-engine (single-threaded, often slow) ratio. Root fixes: star schema, measure revision, V-Order/OPTIMIZE on Delta, and avoiding Direct Lake fallback. “Measure first → find cause → improve.”
Prerequisites: DAX Studio、Direct Lake、Warehouse (Fabric)、Measure
Semantic model and DAX (measures, CALCULATE, storage modes)
Power BI’s meaning layer (formerly dataset). Composed of relationships (fact↔dimension typically one-to-many, single-direction), measures (aggregation expressions, lazy, not stored) and calculated columns (precomputed per row, stored—can bloat), plus hierarchies/formatting. DAX (Data Analysis Expressions) evaluates via row and filter context, with CALCULATE the core for modifying filter context (used with FILTER/ALL/USERELATIONSHIP; TOTALYTD/SAMEPERIODLASTYEAR for time intelligence). Storage modes: Direct Lake (load OneLake Delta directly into memory—fast and near-live; falls back to DirectQuery if conditions unmet), Import (fastest; needs refresh schedule), DirectQuery (always live; slower). The VertiPaq columnstore engine compresses/aggregates fast.
Prerequisites: CALCULATE function、Calculated column (DAX)、Filter context and row context、DAX (Data Analysis Expressions)
Report (Power BI)
Multi-page interactive analysis on one dataset—arrange visuals and explore with filters and drill-down; authored in Power BI Desktop.
Fabric monitoring and optimization (Monitoring hub, Capacity Metrics, V-Order, OPTIMIZE)
Monitoring: Monitoring hub (run history/status/duration for pipelines/notebooks/Spark), Capacity Metrics app (visualize CU consumption/throttling and find overload sources), Data Activator (auto-trigger notifications/actions on data conditions). Error handling: retry policies (transient) + failure paths. Optimization: V-Order (read-optimized layout), OPTIMIZE (compact small files—the go-to fix for the small-files problem; Z-Order for data skipping), VACUUM (remove old unreferenced files), partition design, Spark/capacity tuning.
Prerequisites: Capacity Metrics app、Data Activator、Microsoft Fabric (capacity, workspace, experiences)、Capacity (Fabric)
Dimensional modeling / star schema (fact, dimension, SCD)
Data design for analytics/BI. A star schema places dimension tables (date/product/customer perspectives) radially around a central fact table (numeric measures + foreign keys)—denormalized for fast BI. For a fact, first set the grain (what one row represents). Dimensions use a surrogate key (system-generated integer) as primary key, separate from the business key, to stabilize joins/history. SCD (Slowly Changing Dimension) handles attribute change: Type 1 (overwrite, no history), Type 2 (add rows to keep history), Type 3 (keep prior value in a column). A date dimension (contiguous dates) underpins time intelligence. A snowflake (further normalized) adds joins and often hurts BI performance.
Prerequisites: Date dimension、Denormalization、Dimension table、Fact table
Lakehouse (Fabric)
A Fabric item holding files and tables together; processed via Spark/notebooks, with an auto-generated read-only SQL analytics endpoint for T-SQL queries.
Related: Direct Lake、Warehouse (Fabric)、OneLake、SQL analytics endpoint
Performance metrics (IOPS/throughput/bandwidth/latency)
Core vocabulary for performance: IOPS = I/O operations per second (small random access, e.g., EBS io2); throughput = data volume per time (MB/s, large sequential transfer); bandwidth = link capacity; latency = delay per operation. Design and monitor by the metric that fits the workload.
Microsoft Power BI
A BI tool to visualize/share data as reports (multi-page interactive analysis) and dashboards (single-screen summary).
Related: Dashboard (Power BI)
Measure
A DAX expression that aggregates; computed at evaluation time (lazy), not stored in the model, and changes with filter context.
Prerequisites: Filter context and row context
Related: DAX (Data Analysis Expressions)
Real-Time Intelligence
The Fabric experience for ingesting, storing, analyzing, and acting on streaming data; includes Eventstream, Eventhouse, KQL, and Data Activator.
Prerequisites: Eventstream、Time intelligence
Related: Data Activator、Experience (Fabric)、Eventhouse (KQL database)、Microsoft Fabric (capacity, workspace, experiences)
Star schema
A denormalized design placing dimension tables (analysis perspectives) radially around a central fact table; few joins, fast for BI.
Prerequisites: Denormalization、Dimension table、Fact table、Normalization
Azure Synapse Analytics
A platform unifying large-scale data warehousing and integrated analytics (SQL, Spark, pipelines). Its capabilities are being integrated into the next-generation Microsoft Fabric, which is recommended for new workloads.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)
Azure ML jobs and AutoML/sweep (MLflow, hyperparameter tuning)
Training runs as jobs: command (single script), sweep (hyperparameter search), and pipeline (a DAG of components). Define in YAML (CLI v2)/SDK v2. Tracking uses natively integrated MLflow (log_metric/log_param/log_artifact, autolog) to compare experiments. AutoML auto-searches algorithms/preprocessing from a task and picks via a leaderboard. A sweep tunes hyperparameters with a search space + sampling (grid/random/Bayesian) + early termination (Bandit/median stopping), controlling parallelism/cost via max_concurrent_trials.
Prerequisites: Azure Machine Learning、Command job、Component (reusable step)、Early termination policy (Bandit/median)
SQL analytics endpoint
An auto-generated, read-only T-SQL query endpoint over a lakehouse’s tables (no writes); distinct from a Warehouse’s full T-SQL.
Related: Lakehouse (Fabric)、T-SQL (Fabric)
T-SQL (Fabric)
The SQL dialect used in the warehouse and SQL analytics endpoint; suited to set-based transforms/aggregation, with read-write and transactions in the Warehouse.
Related: SQL analytics endpoint
Azure Data Lake Storage Gen2
A data lake for large-scale analytics that adds a hierarchical namespace on top of Blob storage. With directory structure and POSIX-like ACLs, it serves as the storage layer for Synapse/Databricks analytics.
Prerequisites: Blob storage、Azure Databricks、Azure Synapse Analytics、Data lake
Azure Machine Learning
Supports the ML workflow (prepare→train→evaluate→deploy→predict); includes Automated ML (auto-best) and Designer (no-code).
Data lake
A platform storing large amounts of raw, varied data (structured–unstructured) cheaply; schema on read.
Prerequisites: Unstructured data
Data warehouse
A platform storing shaped, structured data for analytics/aggregation; schema defined on write.
Prerequisites: Warehouse (Fabric)、Structured data
Delta table (Delta format)
Fabric’s default table format: Parquet plus a transaction log enabling ACID, time travel, and schema management; OneLake tables are stored this way.
Prerequisites: ACID、Columnar formats (Parquet/ORC)、Microsoft Fabric (capacity, workspace, experiences)
Related: OneLake
Azure ML compute and data (workspace, cluster, data assets)
The Azure ML workspace centrally manages ML artifacts (data, models, experiments, endpoints) with associated resources (storage/Key Vault/Application Insights/Container Registry). Compute: a compute instance for dev (personal; stop to save cost) and a compute cluster to scale training (autoscale, 0 nodes when idle, low-priority/Spot VMs). Data: a datastore (connection info to Blob/ADLS) and data assets (reusable, versioned references: uri_file/uri_folder/mltable). Trained models are versioned in the model registry. Environments (conda/Docker) pin dependencies for reproducibility. Interact via Studio/Python SDK v2/CLI v2.
Prerequisites: Blob storage、Azure Machine Learning、Compute cluster、Compute instance
Fabric security and governance (workspace roles, sensitivity labels, deployment pipelines)
Workspace roles: Admin > Member > Contributor > Viewer (least privilege). Data-level control via item permissions, row-level security (RLS), and column/object permissions. Governance: sensitivity labels (Microsoft Purview Information Protection for classification/encryption/watermarks) and Purview (auditing, lineage, data catalog). Releases: deployment pipelines (promote artifacts dev→test→prod with diff comparison) and Git integration (CI/CD). Tenant-wide feature availability is governed by admin-portal tenant settings.
Prerequisites: Deployment pipeline (Fabric)、Microsoft Fabric (capacity, workspace, experiences)、Git integration (Fabric)、Item permissions (Fabric)
Eventstream
A no-code streaming pipeline that ingests events (IoT, logs, queues), transforms them, and routes to Eventhouse or lakehouse.
Prerequisites: Eventhouse (KQL database)、Lakehouse (Fabric)
Experience (Fabric)
Role-specific feature sets (Data Engineering / Data Factory / Data Warehouse / Real-Time Intelligence / Power BI); all share the same OneLake.
Prerequisites: Azure Data Factory、Data warehouse
Related: Microsoft Fabric (capacity, workspace, experiences)、OneLake、Real-Time Intelligence
Fact table
The central table holding numeric measures (sales, quantity) and foreign keys to dimensions; first decide its grain (what one row represents).
Prerequisites: Foreign key
Orchestration (Fabric)
Automatically running multiple steps (pipelines/notebooks) combined by order, dependency, and conditions; parameters and dynamic expressions improve reuse.
Prerequisites: Pipeline parameters and dynamic expressions
V-Order
A Fabric write-optimization that reorders Parquet/Delta for reads, speeding Power BI (especially Direct Lake) and SQL reads.
Prerequisites: Columnar formats (Parquet/ORC)、Direct Lake、Microsoft Fabric (capacity, workspace, experiences)、Microsoft Power BI
Stored procedure (Warehouse)
A reusable object bundling a sequence of T-SQL operations (multiple statements, control flow, parameters) to run in the warehouse. Commonly invoked from a pipeline activity to orchestrate data-transformation logic.
Prerequisites: Warehouse (Fabric)、Orchestration (Fabric)、T-SQL (Fabric)
Azure Data Factory
An ETL/ELT pipeline service for ingesting, transforming, and orchestrating data.
Prerequisites: Orchestration (Fabric)
Data pipeline (Fabric)
Orchestrates activities like Copy and notebooks (order, schedule, retry, branching) for ingestion/processing; built in the Data Factory experience.
Prerequisites: Azure Data Factory、Microsoft Fabric (capacity, workspace, experiences)、Experience (Fabric)、Orchestration (Fabric)
DirectQuery mode
Queries the source on each request; always live but depends on source performance and is generally slower than Import.
Azure ML deployment (online/batch endpoints, traffic split)
Inference is served via endpoints. A managed online endpoint = real-time low-latency (Azure manages infra, autoscale); Kubernetes online = existing AKS; batch endpoint = async scoring of large data on a compute cluster. Place multiple deployments under one endpoint and use traffic split for blue/green (gradual rollout, instant rollback). A deployment needs a scoring script (init/run), an environment, and the right instance type/count (MLflow models may need no inference code). Access via managed identity + key/token auth, with private endpoints if needed.
Prerequisites: Azure Machine Learning、Batch endpoint、Compute cluster、Managed online endpoint
Eventhouse (KQL database)
A Real-Time Intelligence store for large event/time-series data; contains KQL databases queried fast with KQL.
Related: Real-Time Intelligence
Capacity (Fabric)
Fabric’s compute and billing unit, bought as F SKUs (or Power BI Premium P SKUs) and consuming capacity units (CU); can pause/resume and is bound to workspaces.
Prerequisites: Microsoft Power BI
Related: Microsoft Fabric (capacity, workspace, experiences)
Item permissions (Fabric)
Shares/grants access to individual items (lakehouse/report) to specific users, separate from workspace roles.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)、Lakehouse (Fabric)、Report (Power BI)、Workspace roles (Fabric)
OneLake shortcut
References internal/external data (ADLS Gen2, Amazon S3, GCS) from OneLake without copying; avoids duplication and virtually unifies data.
Prerequisites: Azure Data Lake Storage Gen2、OneLake
OPTIMIZE (table compaction)
A Delta maintenance command that compacts small files into larger ones to speed reads; the go-to fix for the small-files problem.
Related: The small-files problem
Azure Cosmos DB
A globally distributed, low-latency, auto-scaling managed NoSQL DB supporting multiple APIs (NoSQL/MongoDB/Cassandra/Gremlin/Table).
Prerequisites: Non-relational (NoSQL)、Performance metrics (IOPS/throughput/bandwidth/latency)
Data Activator
A Fabric feature that auto-triggers notifications/actions when data meets conditions (thresholds); part of Real-Time Intelligence.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)
Related: Real-Time Intelligence
Dataflows Gen2
A no-code transform tool using Power Query to get/shape data and write to a destination (lakehouse/warehouse); suited to small-to-medium shaping.
Prerequisites: Warehouse (Fabric)、Lakehouse (Fabric)
DAX (Data Analysis Expressions)
The expression language for measures and calculated columns in a semantic model; evaluated via row context and filter context.
Prerequisites: Calculated column (DAX)、Filter context and row context、Semantic model
Related: Measure
Columnar formats (Parquet/ORC)
Analytics-oriented formats storing data by column. Reading only needed columns cuts scan and I/O and compresses well, greatly lowering Athena/Spark cost.
Identifying and resolving errors (Fabric)
Operationally isolating and fixing errors in pipelines/Dataflows Gen2/notebooks/Eventhouse/Eventstream/T-SQL/shortcuts from run logs and messages.
Prerequisites: Dataflows Gen2、Microsoft Fabric (capacity, workspace, experiences)、Eventhouse (KQL database)、Eventstream
Medallion architecture
A design layering data by quality: bronze (raw) → silver (cleanse/join) → gold (aggregate/business-ready), each stored as Delta.
Related: Bronze layer、Silver layer、Gold layer
Gold layer
The final medallion layer; aggregation and business logic make report/analytics-ready data (often a dimensional model).
Prerequisites: Report (Power BI)
Related: Medallion architecture
Row-level security (Fabric)
Data-level security that limits which rows a user sees; in the warehouse/SQL endpoint/semantic model, predicates restrict rows.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)、Warehouse (Fabric)、Semantic model、SQL analytics endpoint
The small-files problem
Many tiny files slow reads; addressed by compaction via OPTIMIZE and proper partition design.
Structured data
Data with a fixed schema of rows and columns; typed, fast to search/aggregate. E.g., relational DB tables.
VertiPaq (columnstore engine)
The columnar in-memory engine behind Import/Direct Lake models; compresses per column and accelerates aggregation.
Prerequisites: Columnar formats (Parquet/ORC)、Direct Lake
User-defined function (Warehouse)
Functions in Fabric Warehouse that encapsulate reusable logic in T-SQL. In addition to inline table-valued functions (inline TVFs), scalar UDFs (returning a single value) have been supported since 2025 (in preview, not GA). A scalar UDF must be inlineable to be used in a SELECT statement (constructs like WHILE loops or nondeterministic calls prevent inlining). It lets a common calculation be maintained in one place; note the feature gap versus on-premises SQL Server or Synapse for multi-statement TVFs and other constructs.
Prerequisites: Azure Synapse Analytics、Microsoft Fabric (capacity, workspace, experiences)、Warehouse (Fabric)、T-SQL (Fabric)
Workspace roles (Fabric)
The workspace permission hierarchy Admin > Member > Contributor > Viewer; assign by least privilege and combine with data-level permissions.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)、Hierarchy (semantic model)
ACID
Properties of reliable transactions: Atomicity, Consistency, Isolation, Durability.
Blob storage
Storage for unstructured data (objects) such as images, video, and backups.
Prerequisites: Structured data、Unstructured data
Capacity Metrics app
An admin app visualizing capacity-unit (CU) consumption and throttling to pinpoint the items causing overload.
Prerequisites: Capacity (Fabric)
Related: Throttling (capacity)
Choosing a data store
Pick by use: lakehouse for Spark/unstructured, warehouse for full T-SQL/transactions, Eventhouse for streaming/time-series.
Prerequisites: Eventhouse (KQL database)、Warehouse (Fabric)、Lakehouse (Fabric)、T-SQL (Fabric)
Table formats (Iceberg/Hudi/Delta)
Open table formats that bring ACID transactions, upsert/delete, schema evolution, and time travel to S3 data lakes.
Direct Lake on OneLake
The Direct Lake execution path where column data from OneLake Delta tables is read without going through the SQL analytics endpoint, transcoded into memory for the VertiPaq engine on-demand at query time. Contrasted with Direct Lake on SQL. Whether it's used by default and how many feature restrictions apply depend on situational factors at model-creation time (fallback conditions, supported feature scope)—it isn't uniformly true that it always has fewer restrictions and is always the higher-performance default.
Prerequisites: Delta table (Delta format)、Direct Lake、OneLake、SQL analytics endpoint
Sweep job (hyperparameter tuning)
A job that runs many trials with a search space, sampling, primary metric, and early termination to find the best hyperparameter configuration.
Prerequisites: Search space
Related: Primary metric
Modern data warehouse architecture
The analytics flow: ingest → store (data lake) → prepare/transform → model/serve (DWH) → visualize. It combines batch and stream, with Synapse/Fabric covering each stage.
Prerequisites: Azure Synapse Analytics、Data lake、Data warehouse、Microsoft Fabric (capacity, workspace, experiences)
ELT (extract, load, transform)
A data-integration pattern that loads raw extracted data into the destination (warehouse/lake) first, then transforms it there using the destination's own compute. It grew popular as cloud warehouses gained parallel processing power, and keeping the raw data around adds flexibility.
Prerequisites: Data warehouse、Warehouse (Fabric)
Related: ETL (extract, transform, load)
Hierarchy (semantic model)
A drill-down structure defined in the model (year→quarter→month, country→city); used for drilldown in reports.
Prerequisites: Report (Power BI)、Semantic model
PySpark
The Python API for Apache Spark; transforms large data at scale via DataFrames—one of Fabric’s main transform languages.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)
Spark notebook
A notebook for distributed processing of large data with PySpark or Spark SQL; suited to complex transforms, ML, and iteration.
Throttling (capacity)
When sustained overuse of capacity causes delays/rejections; identify the cause in Capacity Metrics and resolve via optimization or scaling.
Related: Capacity Metrics app
Calculated column (DAX)
A column precomputed per row and stored in the model; unlike a measure it is stored, so it can bloat the model—prefer source-side or measures.
Prerequisites: Measure
Column-level security (Fabric)
Data-level security that hides specific columns from users. In the warehouse or SQL analytics endpoint, GRANT/DENY restricts access per column—a different axis from row-level security (RLS, rows vs. columns)—and the two can be combined.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)、Warehouse (Fabric)、SQL analytics endpoint
Related: Object-level security (Fabric)
Copy activity
A pipeline activity that moves large data fast; incremental loads use a watermark column to pull only new rows.
Prerequisites: Watermark (incremental marker)
Filter context and row context
The two DAX evaluation contexts: row context (the current row) and filter context (active filters); CALCULATE manipulates the filter context.
Related: CALCULATE function
Data lake and format optimization
An S3-centric design improving quality through raw (bronze) → cleaned (silver) → curated (gold) layers. Columnar (Parquet/ORC) + partitioning + compaction of small files cut downstream scan and cost. Use Iceberg/Hudi/Delta table formats for updates/deletes/time-travel.
Prerequisites: Data lake、Columnar formats (Parquet/ORC)、Table formats (Iceberg/Hudi/Delta)、The small-files problem
Deployment pipeline (Fabric)
A release feature that promotes artifacts across dev→test→prod stages with diff comparison and deployment rules; prevents manual-deploy mistakes.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)
Dimension table
A table of analysis perspectives/attributes (date, product, customer); joined one-to-many to the fact, serving as filter/group axes.
Prerequisites: Fact table、Grouping and aggregation (transform)
Component (reusable step)
A reusable unit of processing bundling inputs, outputs, code, and environment. Composed as pipeline steps and versioned/shared.
Synapse Spark / serverless Spark
Spark compute to interactively wrangle large data from notebooks, using an attached Synapse Spark pool or Azure ML serverless Spark.
Prerequisites: Azure Machine Learning、Azure Synapse Analytics、Spark notebook、Spark performance optimization
ETL (extract, transform, load)
A data-integration pattern that extracts data from a source and transforms it (cleansing, reshaping, aggregating) before loading it into the destination. It needs separate transform compute (a server or managed service), but only clean, shaped data ever lands in the destination.
Prerequisites: Managed services (management boundary)
Related: ELT (extract, load, transform)
Smoothing and bursting
Mechanisms that level capacity use: bursting temporarily uses extra compute, and smoothing spreads that consumption over time; overload triggers throttling.
Prerequisites: Throttling (capacity)
Git integration (Fabric)
Connects a workspace to an Azure DevOps/GitHub repo to source-control items; the basis for version control and CI/CD.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)
Bronze layer
The first medallion layer; holds raw ingested data nearly as-is (the basis for history/reprocessing).
Related: Medallion architecture
Silver layer
The middle medallion layer; cleansing, dedup, joins, and typing produce consistent, integrated data.
Related: Medallion architecture
Microsoft Purview
Microsoft's unified governance and compliance solutions: information protection, data lifecycle, risk management, eDiscovery, and auditing, plus cross-estate data governance via the data map/catalog (integrating the former Azure Purview).
Mirroring
Replicates changes from external databases (Azure SQL, Snowflake, Cosmos DB) to OneLake in near real time using CDC, enabling analytics without building ETL.
Prerequisites: Azure Cosmos DB、OneLake
Monitoring hub
A Fabric monitoring view listing run history, status, and duration for pipelines/notebooks/Spark.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)
Normalization
Splitting/designing tables to reduce duplication and keep integrity; prevents update anomalies.
Storage mode
How a semantic model stores/retrieves data—Import, DirectQuery, Direct Lake—chosen by the speed-vs-freshness trade-off.
Prerequisites: Direct Lake、DirectQuery mode、Semantic model
Tabular Editor
An external tool to edit semantic models efficiently; used to create calculation groups and run the Best Practice Analyzer.
Prerequisites: Semantic model
Related: Calculation group
Tenant settings (admin portal)
Settings where a Fabric admin governs tenant-wide feature availability (export, sharing, enabling specific features).
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)
Time intelligence
Time-series DAX calculations (YoY, YTD, moving average) that rely on a date dimension (TOTALYTD, SAMEPERIODLASTYEAR).
Related: Date dimension
Unstructured data
Data with no fixed schema, stored as files/objects. E.g., images, video, audio, documents.
Prerequisites: Structured data
View (Warehouse)
An object that lets a saved SELECT statement in the warehouse be reused as if it were a table. Complex joins/aggregations are hidden behind the view, simplifying downstream reports and queries; a view stores no data itself and re-evaluates the underlying tables on each reference.
Prerequisites: Warehouse (Fabric)、Report (Power BI)
Related: Stored procedure (Warehouse)、User-defined function (Warehouse)
Azure SQL Database
A fully managed relational PaaS; Azure handles patching, backups, availability. Best for new cloud apps.
Prerequisites: Managed services (management boundary)
CALCULATE function
The core DAX function that modifies filter context to re-aggregate; combined with FILTER/ALL/USERELATIONSHIP for conditional aggregation.
Related: USERELATIONSHIP (DAX)、Filter context and row context
Calculation group
Applies the same logic (YoY, YTD) across many measures at once; reduces measure duplication and eases maintenance; built in Tabular Editor.
Prerequisites: Measure
Related: Tabular Editor
Denormalization
Deliberately joining/redundifying normalized tables to speed reads/analytics; common in analytics platforms to cut join cost.
Prerequisites: Normalization
Compute cluster
Autoscaling compute that scales training or batch inference. It can shrink to 0 nodes when idle and use low-priority/Spot VMs to cut cost.
Search space
The definition of which hyperparameters to tune and their candidate ranges (discrete choice, continuous uniform/loguniform, etc.).
Azure ML workspace
The top-level resource that centrally manages ML artifacts (data, models, experiments, endpoints), with associated storage, Key Vault, Application Insights, and Container Registry.
Prerequisites: Azure Machine Learning、Azure Key Vault
Data ingestion and processing (pipelines)
Azure Data Factory / Synapse pipelines copy and orchestrate data, with mapping data flows transforming it without code; an integration runtime performs execution.
Prerequisites: Azure Data Factory、Azure Synapse Analytics、Orchestration (Fabric)
Power BI components (Power Query/model/visuals)
In Power BI Desktop, Power Query ingests and shapes (cleanses) data → build a data model (relationships, measures) → construct reports with visuals, then share via the Power BI service.
Prerequisites: Measure、Report (Power BI)、Microsoft Power BI
Synapse components (SQL/Spark pools)
Azure Synapse Analytics unifies a dedicated SQL pool (provisioned DWH), serverless SQL pool (query the data lake), Apache Spark pool (big-data processing), and pipelines in one studio.
Prerequisites: Azure Synapse Analytics、Data lake、Spark performance optimization
Folder/file-level access control
OneLake security (data access roles): RBAC that grants Read or ReadWrite at the table or folder scope. Deny-by-default, with permissions inheriting to child folders.
Prerequisites: Role-based access control (RBAC)、OneLake、OneLake security
Foreign key
A column referencing another table’s primary key to relate tables; preserves referential integrity.
Prerequisites: Primary key
Grouping and aggregation (transform)
Transforms that group by keys to compute sum/count/average; used to build gold-layer and report-ready data.
Prerequisites: Gold layer、Report (Power BI)
Impact analysis (downstream dependencies)
Analysis to foresee how changes to lakehouses/warehouses/dataflows/semantic models ripple to downstream reports.
Prerequisites: Lakehouse (Fabric)、Report (Power BI)、Semantic model
JSON
A lightweight text format representing structured data as braces and key–value pairs. Widely used wherever machine-readability matters most—API request/response bodies and permission definitions like IAM or bucket policies.
Prerequisites: Structured data
Non-relational (NoSQL)
Databases storing data flexibly without a fixed schema; four kinds: key-value/document/column-family/graph.
Object-level security (Fabric)
Security controlling access at the object level (tables, views, etc.)—a coarser control that hides an entire table, acting as the first gate before finer column- or row-level rules apply. Configured via GRANT/REVOKE in the warehouse.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)、Warehouse (Fabric)
Related: Column-level security (Fabric)
OLAP (analytical)
An analytical workload of large reads/aggregations over historical data, run against a data warehouse.
Prerequisites: Data warehouse、Warehouse (Fabric)
Dashboard (Power BI)
A single-screen summary collecting key metrics (KPIs) from multiple reports; created/shared in the Power BI service (reports are multi-page, dashboards single-screen).
Prerequisites: Report (Power BI)
Related: Microsoft Power BI
Primary key
A column that uniquely identifies each row; no duplicates or nulls.
Query acceleration (OneLake shortcuts)
In Real-Time Intelligence, caches/indexes data over OneLake shortcuts to speed KQL queries; tested vs standard shortcuts.
Prerequisites: OneLake、OneLake shortcut、Real-Time Intelligence
Semantic model refresh (monitoring)
Reloading data into a Power BI semantic model; monitor failures/duration—refresh behaves differently with Direct Lake.
Prerequisites: Direct Lake、Microsoft Power BI、Semantic model
Sensitivity labels
Classify data and apply protection (encryption, usage limits); labels travel with the file, persisting outside the org.
Spark SQL
Using SQL on Spark to transform/aggregate data; lets SQL-savvy users write large-scale processing from notebooks.
Storage engine vs formula engine
The two DAX engines: storage engine (VertiPaq, parallel, fast) and formula engine (single-threaded, often slow); over-reliance on the latter causes slowness.
Prerequisites: VertiPaq (columnstore engine)
Choosing a streaming engine
Pick by use: Eventstream for no-code broad connectivity, Spark structured streaming for complex/large processing, KQL for low-latency query aggregation.
Prerequisites: Eventstream、Performance metrics (IOPS/throughput/bandwidth/latency)、Spark structured streaming
Surrogate key
A system-generated integer key used as a dimension’s primary key, separate from the business key, to stabilize joins and ease history (SCD).
Prerequisites: Primary key
USERELATIONSHIP (DAX)
A DAX function that temporarily activates an inactive relationship in the model, scoped to a single calculation. Used as a CALCULATE modifier, it lets you pick which of several relationships between two tables (e.g., order date vs. ship date) applies for a given expression.
Related: RELATED (DAX)、CALCULATE function
VACUUM
A Delta maintenance command that deletes old unreferenced files no longer needed for time travel, reclaiming storage; mind the retention period.
Statistics (Warehouse)
Column distribution info the optimizer uses to pick efficient plans; stale/missing statistics cause slow queries.
Prerequisites: Warehouse (Fabric)
Watermark (incremental marker)
A marker value (last-modified timestamp or sequence) tracking how far data was loaded; the next run pulls only rows newer than it.
Related: Full and incremental loads
YAML
A human-friendly text format that expresses hierarchy through indentation. Favored for CloudFormation and other IaC tools and CI/CD pipeline config files, and interconvertible with JSON representing the same data.
Prerequisites: JSON、Hierarchy (semantic model)
Azure storage account
A container bundling Blob, File, Table, and Queue storage services.
Prerequisites: Blob storage、Queue storage
Composite model
A model mixing Import and DirectQuery (or Direct Lake); choose a mode per table, with dual mode allowing both.
Prerequisites: Direct Lake、DirectQuery mode
Date dimension
A dedicated dimension of contiguous dates and calendar attributes (year/quarter/month/weekday); the prerequisite for time intelligence (e.g., YoY).
Related: Time intelligence
DAX performance improvement
Making slow DAX faster: use variables to avoid re-evaluation, reduce iterators, optimize filters, and adopt a star schema.
Prerequisites: OPTIMIZE (table compaction)、Star schema
DAX Studio
An external tool to run, time, and trace DAX queries; server timings reveal the storage-engine vs formula-engine breakdown.
Prerequisites: Storage engine vs formula engine
Dynamic data masking (Fabric)
Shows data masked (e.g., partial email) only to unauthorized users; the underlying data is unchanged, masked at display.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)、Sensitive data protection (PII / masking)
Preparing for a dimensional model
Shaping data into facts (measures) and dimensions (attributes), loading with surrogate keys and support for slowly changing dimensions (SCD).
Prerequisites: Fact table、Surrogate key
Direct Lake fallback
When Direct Lake hits unsupported situations (certain features/limits), it auto-switches to DirectQuery; avoiding this fallback is tested.
Prerequisites: Direct Lake、DirectQuery mode
Batch endpoint
An endpoint that asynchronously scores large data on a compute cluster. Invoking it starts a batch scoring job.
Prerequisites: Compute cluster
Early termination policy (Bandit/median)
A sweep setting that cancels unpromising trials to save cost. Options include Bandit (slack factor), median stopping, and truncation selection.
Prerequisites: Sweep job (hyperparameter tuning)
MLflow tracking
Experiment tracking natively integrated in Azure ML. Logs training via log_metric/log_param/log_artifact and autolog to compare runs.
Prerequisites: Azure Machine Learning
Managed online endpoint
An endpoint serving real-time low-latency inference where Azure manages infrastructure and autoscale. Kubernetes online uses an existing AKS.
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Pipeline job
A sequence connecting multiple components as a DAG, passing data between steps. It can be scheduled and monitored.
Prerequisites: Component (reusable step)
Responsible AI dashboard
A set of Azure ML tools to assess models for responsible AI, offering error analysis, interpretability, fairness, and counterfactuals in one dashboard.
Prerequisites: Azure Machine Learning、Dashboard (Power BI)
Sampling method (grid/random/Bayesian)
How a sweep picks configurations from the search space: grid (exhaustive), random (broad and fast), and Bayesian (learns promising regions from prior results).
Prerequisites: Search space、Sweep job (hyperparameter tuning)
Cosmos DB APIs
Cosmos DB offers multiple APIs by workload: NoSQL (document, default), MongoDB, Cassandra, Gremlin (graph), and Table—chosen for migration or use case.
Prerequisites: Azure Cosmos DB、Non-relational (NoSQL)
Query tools (Azure Data Studio / SSMS)
Tools to connect, run SQL, and manage databases: Windows SQL Server Management Studio (SSMS), Visual Studio Code with the MSSQL extension, and web-based SQL query editors in the Azure portal or Microsoft Fabric (Azure Data Studio was retired on 2026-02-28).
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)
Transactional vs analytical workloads
OLTP handles many small reads/writes fast with consistency for operational systems; OLAP aggregates/analyzes large data for decisions. Data flows OLTP → (ETL) → OLAP.
Prerequisites: OLAP (analytical)、OLTP (transactional)
Dynamic format strings
Changes a measure’s display format (currency, percent) dynamically via DAX by condition; often used with calculation groups.
Prerequisites: Calculation group、Measure
Eventual consistency
A consistency model where a read right after a write may return a stale value, but all replicas eventually converge over time. It's often the default behavior of distributed data stores that prioritize availability and low latency, and using it for reads that don't need strong consistency buys throughput.
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Related: Strong consistency
Configuring alerts (Fabric)
Settings that detect refresh failures or met conditions and notify; combined with Data Activator/monitoring to automate operations.
Prerequisites: Data Activator、Microsoft Fabric (capacity, workspace, experiences)
Audit logs (Fabric)
Logs of who did what and when (access, changes, sharing); reviewed via Purview/admin portal for compliance and investigation.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)、Microsoft Purview
Domain (Fabric)
A logical group bundling related workspaces by business area (sales, finance), enabling data-mesh-style decentralized governance with per-domain defaults and delegated admins.
Prerequisites: RELATED (DAX)、Microsoft Fabric (capacity, workspace, experiences)
Workspace (Fabric)
The unit that groups items (lakehouse, notebook, pipeline) for collaboration and access control; assigned to a capacity.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)、Lakehouse (Fabric)
Field parameters
Lets report users dynamically switch which measures or axes are shown; one visual offers multiple views.
Prerequisites: Measure、Report (Power BI)
Grain
The level of detail one fact row represents (e.g., one transaction, one product per day); decided first and key to correct aggregation.
Prerequisites: Fact table
Azure Key Vault
Securely stores and manages keys, secrets, and certificates so they need not be embedded in apps.
Managed services (management boundary)
A categorization by how much AWS operates for you. The more fully managed (AWS handles patching/scaling/availability), the lower your operational burden but less control; unmanaged (e.g., EC2) is flexible but self-operated. It frames “where your responsibility ends” in the shared responsibility model.
OLTP (transactional)
A workload of many small, fast reads/writes for operations; ACID guarantees transaction correctness.
Prerequisites: ACID
OneLake catalog
A data-discovery hub to search and understand OneLake items, checking trust (endorsement) and lineage.
Prerequisites: Endorsement (promoted/certified)、OneLake
OneLake security
Access control over OneLake data; OneLake data access roles define folder/table-level permissions applied consistently across engines.
Prerequisites: OneLake
Partition design
Splitting data by frequently filtered columns (e.g., date) so queries read only needed partitions; over-partitioning causes small files, so mind granularity.
Related: The small-files problem
Power BI Desktop project (.pbip)
A format that saves report and model as text-based files; easy to source-control in Git for diff review and CI/CD.
Prerequisites: Report (Power BI)、Microsoft Power BI
Performance Analyzer
A Power BI Desktop feature that measures each visual’s DAX query and display time to find slow visuals; the starting point for tuning.
Prerequisites: Microsoft Power BI
Query performance optimization
Speeding queries via execution plans, statistics, good joins/filters, and partition pruning; important in the warehouse/SQL endpoint.
Prerequisites: Warehouse (Fabric)、SQL analytics endpoint
Queue storage
A message queue to decouple components and smooth processing spikes.
Prerequisites: Component (reusable step)
Retry policy
A setting that auto-reruns an activity on transient failures (e.g., brief network drops); combine with failure paths for robustness.
Semi-structured data
Data structured by tags/keys but not bound to a fixed schema; flexible. E.g., JSON, XML.
Prerequisites: JSON、Structured data
Snowflake schema
A design that further normalizes dimensions into multiple tables; more joins and usually lower BI performance than a star schema.
Prerequisites: Normalization、Star schema
Spark performance optimization
Tuning Spark: starter/custom pools, partition count, skew mitigation, caching, and broadcast joins.
Strong consistency
A consistency model guaranteeing that a read after a completed write always returns the latest value. It typically costs more latency or throughput than eventual consistency, but is chosen where stale reads are unacceptable, such as inventory counts or balances. Consistency models are picked per use case based on this trade-off.
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Related: Eventual consistency
Table storage
Simple, inexpensive key-value (NoSQL) tables for large volumes of lightweight structured data.
Prerequisites: Non-relational (NoSQL)、Structured data
Visual Query Editor
A no-code, drag-based Fabric query builder for selecting, filtering, and aggregating; generates SQL behind the scenes.
Prerequisites: Microsoft Fabric (capacity, workspace, experiences)、Query tools (Azure Data Studio / SSMS)
XMLA endpoint
A connection point to connect, deploy, and manage semantic models from external tools (Tabular Editor); used to operate large models.
Prerequisites: Semantic model、Tabular Editor
Z-Order (data skipping)
Co-locates data by frequently filtered columns so irrelevant files can be skipped (data skipping).
Role-based access control (RBAC)
Authorization that grants permissions by assigning roles to identities at a scope; least privilege, inherited downward.
Azure Database for MySQL
A managed relational service for the open-source MySQL engine. The current offering is the Flexible Server, supporting same-zone placement for latency optimization and flexible compute/storage scaling; Azure handles patching, backups, and failover.
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Azure Databricks
An Apache Spark-optimized platform for big-data processing and ML; strong for data science.
Batch processing
Processing data in groups on a schedule; tolerates latency, handles large volumes efficiently. E.g., nightly rollups.
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Calculated table
A table generated by a DAX expression; used to create helper tables not in the source, like date or parameter tables.
Prerequisites: DAX (Data Analysis Expressions)
Capacity planning / demand forecasting
Estimating future load to plan needed capacity, cost, and scaling policy. In the cloud, Auto Scaling and predictive scaling follow demand dynamically, avoiding fixed over-provisioning; Compute Optimizer and CloudWatch trend analysis feed it.
Prerequisites: Capacity (Fabric)
Data temperature and tiering (hot/warm/cold)
Classifying data by access frequency to optimize cost: hot = frequent (fast, costly), warm = occasional, cold = rare (cheap, slow retrieval—Glacier). S3 lifecycle and Intelligent-Tiering move data between tiers automatically.
Prerequisites: OPTIMIZE (table compaction)
Database project
Manages warehouse schema as code for source control and diff-based deployment (a SQL project).
Prerequisites: Warehouse (Fabric)
Model benchmarks comparison
Evaluation that compares language models on metrics like accuracy, coherence, cost, and throughput to select candidates fit for the use case.
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Command job
The basic job that runs a single script with a specified environment, compute, and inputs. Pass parameters and data inputs, and diagnose failures via logs.
Compute instance
A managed VM for individual development and notebook execution. It can be stopped to save cost and includes a terminal and Jupyter.
Data assets (uri_file/uri_folder/mltable)
A reusable, versioned reference to data. Types: uri_file (a single file), uri_folder (a folder), and mltable (a schematized tabular form).
Tracing to evaluate a flow
Recording (tracing) each flow step’s inputs/outputs, latency, and token usage to evaluate and improve quality and performance bottlenecks.
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Model catalog
A catalog in Azure AI Foundry / Azure ML to discover, compare, and deploy many foundation models (OpenAI, Hugging Face, Meta, etc.).
Prerequisites: Azure Machine Learning
Model registry
A place to version, register, and reference trained models with stages and lineage, used to select what to deploy.
Primary metric
The metric a sweep or AutoML optimizes against (e.g., accuracy, AUC), with a direction to maximize or minimize.
Related: Sweep job (hyperparameter tuning)
Registries (cross-workspace sharing)
An org-level catalog to share models, components, environments, and data across multiple workspaces, used to promote from dev to production.
Prerequisites: Component (reusable step)
Scoring script (init/run)
The script implementing inference in a deployment: init() loads the model and run() handles each request. It may be unnecessary for MLflow models.
Traffic split and blue/green
Place multiple deployments under one endpoint and split traffic by percentage. Roll out a new version gradually and roll back instantly on issues.
Data visualization and report types
Choose visuals by purpose (bar, line, pie, matrix, map, KPI). There are interactive reports and paginated reports formatted for printing.
Prerequisites: Report (Power BI)
Relational objects (views/stored procedures)
Beyond tables, database objects like views (saved queries), stored procedures, and functions structure data access.
Prerequisites: Stored procedure (Warehouse)
Encryption at rest
Encrypting data while it sits on disk or object storage. Most managed cloud storage (block/file/object/DB) offers built-in encryption integrated with the platform's key management (e.g., a KMS). A baseline defense protecting the data itself against theft or unauthorized access.
Encryption in transit
Encrypting data while it moves across the network, typically protected with TLS. Only by pairing it with encryption at rest do you cover a piece of data's whole lifecycle (stored and moving)—satisfying both is a baseline security requirement.
Prerequisites: Encryption at rest
Endorsement (promoted/certified)
A governance feature that marks trustworthy items as Promoted or Certified so users can identify high-quality data.
Full and incremental loads
Full load replaces everything each time; incremental load ingests only changes since last run (via watermark or CDC)—incremental is efficient at scale.
Related: Watermark (incremental marker)
Horizontal scaling (scale out)
Scaling by adding more instances to spread the load. It assumes a stateless design plus a load balancer and auto scaling, and since one instance failing doesn't take down the whole system, it also improves fault tolerance. Considered the default direction in cloud-native design.
Incremental refresh (semantic model)
On model refresh, loads only recently changed partitions and skips historical data; greatly cuts refresh time and resources.
Prerequisites: Semantic model
KQL (Kusto Query Language)
A read-only query language for fast search/aggregation over log/time-series data; used with Eventhouse/KQL databases.
Prerequisites: Eventhouse (KQL database)
Large semantic model storage format
A storage format for large models; raises size limits and enables XMLA write and some advanced features.
Prerequisites: Semantic model
Managed identity
An identity letting apps access Azure resources securely without managing secrets; a service principal auto-managed by Azure.
Model relationships
Links between tables; fact↔dimension is typically one-to-many, single-direction. Use bidirectional cross-filter only when needed (watch ambiguity/performance).
Prerequisites: Fact table
Model row-level security (roles)
Defines roles in a semantic model and restricts visible rows via DAX predicates; dynamic RLS uses USERPRINCIPALNAME to scope to the user’s rows.
Prerequisites: Semantic model
Sensitive data protection (PII / masking)
Protecting sensitive data such as PII (personally identifiable information) and PHI: discover/classify with Macie, then protect via encryption, access control, and minimization. Masking/tokenization hides values, and handling varies by data classification; also prevent sensitive data leaking into logs.
Pipeline parameters and dynamic expressions
Parameters pass values at runtime, and dynamic expressions build those values; lets one pipeline be reused across environments/targets.
Real-Time hub
A central hub to list, discover, and connect to streaming/event data across the org; the entry point to Real-Time Intelligence.
Prerequisites: Real-Time Intelligence
Relational database
A database storing data in tables related by keys, operated with SQL.
Prerequisites: RELATED (DAX)
Reusable assets (.pbit / .pbids)
.pbit is a Power BI template (model/layout blueprint); .pbids is a data-source connection definition; shared to reuse the same setup.
Prerequisites: Microsoft Power BI
Spark structured streaming
Continuously processes streaming data in Spark with batch-like code; lets you write complex stream transforms in notebooks.
Stream processing
Processing events as they arrive in near real time; for low-latency needs. E.g., sensor alerts.
Prerequisites: Performance metrics (IOPS/throughput/bandwidth/latency)
Vertical scaling (scale up)
Scaling by upgrading a single instance's specs (CPU, memory, etc.). It requires little application change and is simple, but is capped by instance-type limits, tends to be a single point of failure, and often requires downtime to apply. The counterpart choice to horizontal scaling.
Prerequisites: Horizontal scaling (scale out)