5Testing, Validation, and Troubleshooting

• 5.1Evaluation Systems for GenAI

  Learn to systematically evaluate FM output quality: Bedrock Model Evaluations, RAG evaluation, LLM-as-a-Judge, Bedrock Agent evaluations, A/B and canary testing, relevance/factual/consistency/fluency metrics, and regression testing.

• 5.2Troubleshooting GenAI Applications

  Learn to isolate GenAI-specific issues: context window overflow, FM API integration problems, prompt issues, retrieval issues (embedding quality/drift), and prompt maintenance with schema validation, CloudWatch Logs/X-Ray.