What's changed: Added per-section figures (cert-figure-retrofit). New AI-300 Chapter 5 (Domain 5 "Optimize generative AI": RAG optimization = similarity threshold/chunk size/retrieval strategy/embedding model selection-fine-tuning/hybrid search/relevance metrics-A-B testing; advanced fine-tuning = methods/synthetic data/monitoring-optimization/dev-to-production lifecycle management)
5.2Advanced fine-tuning and model customization
Understand designing and implementing advanced fine-tuning methods, creating and managing synthetic data for fine-tuning, monitoring and optimizing fine-tuned model performance, and managing a fine-tuned model from development through production deployment.
When prompting or RAG is insufficient, fine-tuning adapts the model itself to a specific task or domain. AI-300 treats fine-tuning as an operational lifecycle.
5.2.1Fine-tuning methods and synthetic data
Fine-tuning hinges on high-quality training data (input–desired-output pairs). When enough data is unavailable, prepare synthetic data (generated with AI), managing quality, diversity, and bias. Design the method—when and how much to adapt the model. Understand the order (balancing cost and benefit): "try prompting/RAG first → fine-tune only if still insufficient."
5.2.2Monitoring and managing to production
A fine-tuned model also needs monitoring and optimization: watch for loss of general performance (overfitting) or degradation on new data, and re-fine-tune if needed. Manage the model developed in dev through a lifecycle: evaluate → register/version → deploy to production → monitor. This applies the same operational mindset as Chapter 2 MLOps to fine-tuned generative AI models.
| Stage | What | Key point |
|---|---|---|
| Design adaptation | Advanced fine-tuning methods | How much to adapt |
| Secure data | Create/manage synthetic data | Quality/diversity/bias |
| Maintain performance | Monitor and optimize | Detect overfitting/degradation |
| Manage to production | Evaluate→register→deploy→monitor | Same lifecycle as MLOps |
Watch the mix-ups: (1) Optimization order: prompting → RAG → fine-tuning (cost/benefit). (2) Synthetic data needs quality/diversity/bias management (poor data degrades performance). (3) Fine-tuned models still need monitoring and a production lifecycle (not build-and-forget).
Map requirement → means: "prompting/RAG still insufficient" = fine-tuning; "not enough training data" = create synthetic data (with quality control); "prevent loss of general performance after adaptation" = monitoring and re-fine-tuning; "manage dev→production consistently" = the evaluate→register→deploy→monitor lifecycle.
5.2.3Section summary
- Optimization order: prompting → RAG → fine-tuning (cost/benefit)
- Manage synthetic data for quality/diversity/bias; fine-tuning hinges on high-quality data
- Fine-tuned models also need monitoring + the evaluate→register→deploy→monitor lifecycle
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. When prompting or RAG is insufficient, which adapts the model itself to a specific task/domain?
Q2. When training data for fine-tuning is insufficient, what do you create while managing quality, diversity, and bias?
Q3. Which is the generally recommended order for optimizing generative AI?
Q4. Which is an appropriate risk to monitor for a fine-tuned model?
Q5. Which is the correct lifecycle to manage a fine-tuned model from development to production?
Q6. What happens if you use poor-quality synthetic data?
Keep track of your progress
The full study guide is free to read. Sign up free to practice with the question bank, track what you have read, review your mistakes, and highlight passages.