Operationalizing Machine Learning and Generative AI SolutionsStudy guide

Understand creating and managing an Azure Machine Learning (AML) workspace, datastores, compute targets, and identity and access management for the workspace (managed identities / RBAC)—the foundation of MLOps infrastructure.

Understand creating and managing AML assets (data assets, environments, components) and sharing assets across workspaces using registries, for reproducibility and reuse.

Understand secure GitHub–Machine Learning integration, deploying workspaces/resources with Bicep and Azure CLI, automating provisioning with GitHub Actions workflows, restricting network access to workspaces, and Git source control—as the automation foundation of MLOps.

Understand experiment tracking with MLflow, automated machine learning (AutoML), notebook exploration, hyperparameter tuning (sweeps), running training scripts, distributed training for large/deep models, training pipelines, and comparing model performance across jobs.

Understand registering an MLflow model, packaging a feature retrieval specification with the model artifact, evaluating a model using responsible AI principles, and managing the model lifecycle including archiving.

Understand deploying models as real-time/batch managed endpoints, testing and troubleshooting endpoints, progressive rollout and safe rollback, and in production configuring data drift detection, performance-metric monitoring, and retraining/alert triggers when thresholds are exceeded.

Understand creating and configuring Microsoft Foundry resources and project environments, identity and access management with managed identities and RBAC, network security and private networking, and deploying infrastructure with Bicep templates and the Azure CLI.

Understand deploying foundation models via serverless API endpoints and managed compute, selecting models for use cases, model versioning and production deployment strategies, and configuring provisioned throughput units (PTU) for high-volume workloads.

Understand designing and developing prompts, creating prompt variants and comparing their performance, and version-controlling prompts with Git repositories—from a source-control and experimentation perspective.

Understand creating test datasets and data mapping, AI quality metrics (groundedness/relevance/coherence/fluency), risk and safety evaluations for harmful content detection, and configuring automated evaluation workflows using built-in/custom metrics.

Understand continuous monitoring in Foundry, monitoring performance metrics (latency, throughput, response times), tracking and optimizing cost metrics (token consumption, resource usage), and configuring detailed logging, tracing, and debugging for production troubleshooting.

Understand optimizing retrieval performance by tuning similarity thresholds, chunk sizes, and retrieval strategies; selecting and fine-tuning embedding models for domain-specific accuracy; hybrid search combining semantic and keyword retrieval; and evaluating and improving RAG with relevance metrics and A/B testing.

Understand designing and implementing advanced fine-tuning methods, creating and managing synthetic data for fine-tuning, monitoring and optimizing fine-tuned model performance, and managing a fine-tuned model from development through production deployment.