What's changed: Added per-section figures (cert-figure-retrofit). New AI-300 Chapter 5 (Domain 5 "Optimize generative AI": RAG optimization = similarity threshold/chunk size/retrieval strategy/embedding model selection-fine-tuning/hybrid search/relevance metrics-A-B testing; advanced fine-tuning = methods/synthetic data/monitoring-optimization/dev-to-production lifecycle management)
5.1Optimizing RAG performance and accuracy
Understand optimizing retrieval performance by tuning similarity thresholds, chunk sizes, and retrieval strategies; selecting and fine-tuning embedding models for domain-specific accuracy; hybrid search combining semantic and keyword retrieval; and evaluating and improving RAG with relevance metrics and A/B testing.
RAG (retrieval-augmented generation) retrieves external context to ground answers and reduce hallucination. RAG quality depends strongly on retrieval quality, so optimizing retrieval is an AI-300 focus.
5.1.1Optimizing retrieval
Optimize retrieval by tuning the similarity threshold (how close context must be to be used), chunk size (how documents are split—too large adds noise, too small loses context), and retrieval strategy (top-k, re-ranking, etc.). The choice of embedding model also drives accuracy, and for domain-heavy terminology, fine-tuning the embedding model improves relevance. Tune iteratively while evaluating.
5.1.2Hybrid search and A/B testing
Hybrid search combines semantic (vector) search (by meaning) with keyword search (by term match) to cover each other’s weaknesses and improve recall and precision. Validate improvements quantitatively with relevance metrics (are the retrieved results appropriate) and A/B testing (compare configurations under near-production conditions). The basis of RAG optimization: "measure → tune → measure again."
Continue reading — free sign-up
You're reading the free preview. Sign up free to read this section in full, plus every chapter (including 4+) and all questions.