What's changed: Added per-section figures (cert-figure-retrofit). New AI-300 Chapter 5 (Domain 5 "Optimize generative AI": RAG optimization = similarity threshold/chunk size/retrieval strategy/embedding model selection-fine-tuning/hybrid search/relevance metrics-A-B testing; advanced fine-tuning = methods/synthetic data/monitoring-optimization/dev-to-production lifecycle management)
5.2Advanced fine-tuning and model customization
Understand designing and implementing advanced fine-tuning methods, creating and managing synthetic data for fine-tuning, monitoring and optimizing fine-tuned model performance, and managing a fine-tuned model from development through production deployment.
When prompting or RAG is insufficient, fine-tuning adapts the model itself to a specific task or domain. AI-300 treats fine-tuning as an operational lifecycle.
5.2.1Fine-tuning methods and synthetic data
Fine-tuning hinges on high-quality training data (input–desired-output pairs). When enough data is unavailable, prepare synthetic data (generated with AI), managing quality, diversity, and bias. Design the method—when and how much to adapt the model. Understand the order (balancing cost and benefit): "try prompting/RAG first → fine-tune only if still insufficient."
5.2.2Monitoring and managing to production
A fine-tuned model also needs monitoring and optimization: watch for loss of general performance (overfitting) or degradation on new data, and re-fine-tune if needed. Manage the model developed in dev through a lifecycle: evaluate → register/version → deploy to production → monitor. This applies the same operational mindset as Chapter 2 MLOps to fine-tuned generative AI models.
Continue reading — free sign-up
You're reading the free preview. Sign up free to read this section in full, plus every chapter (including 4+) and all questions.