What's changed: Added per-section figures (cert-figure-retrofit). New AI-300 Chapter 4 (Domain 4 "GenAI quality assurance and observability": evaluation = test datasets/data mapping/AI quality metrics (groundedness/relevance/coherence/fluency)/risk-safety evaluation/automated evaluation workflows; observability = Foundry continuous monitoring/performance (latency/throughput/response time)/cost (token consumption/resource usage)/logging-tracing-debugging)
4.2Observability for generative AI apps and agents
Understand continuous monitoring in Foundry, monitoring performance metrics (latency, throughput, response times), tracking and optimizing cost metrics (token consumption, resource usage), and configuring detailed logging, tracing, and debugging for production troubleshooting.
Generative AI apps are continuously monitored in production to understand performance, cost, and failures and to improve. This is observability, which supports the GenAIOps operations loop.
4.2.1Monitoring performance and cost
With continuous monitoring in Foundry, track performance metrics: latency (delay to respond), throughput (work per unit time), and response time. A key generative-AI metric is cost, especially token consumption (input + output tokens) and resource usage. Because token consumption maps directly to cost, reduce it via concise prompts, caching, and appropriate model selection.
4.2.2Logging, tracing, and debugging
Production troubleshooting needs detailed logging, tracing, and debugging. Tracing visualizes how a request flows through app → retrieval (RAG) → model → tools, pinpointing causes of latency or failure (distributed tracing). For multi-step flows like agents, tracing is especially valuable. With logs, you quickly find the root cause of quality degradation or errors.
Continue reading — free sign-up
You're reading the free preview. Sign up free to read this section in full, plus every chapter (including 4+) and all questions.