What's changed: Added per-section figures (cert-figure-retrofit). New AI-200 Chapter 4 (secret management = Key Vault/App Configuration/managed identity keyless/RBAC, observability = OpenTelemetry instrumentation/distributed tracing/Application Insights·Azure Monitor/KQL/AI-specific monitoring = tokens·cost·quota·generation quality)
4.2Observability (Monitoring, Tracing, Logs)
Understand observability for a distributed AI back-end—instrumentation with OpenTelemetry, distributed tracing, Application Insights / Azure Monitor, log analysis with KQL, and monitoring tokens/cost/quality.
In an event-driven, distributed AI pipeline, a single request is processed across multiple services. Design observability so that when something goes wrong you can trace "where and why it is slow/failing." The standard is instrumentation with OpenTelemetry (embedding traces, metrics, and logs in code), collected into Application Insights / Azure Monitor.
4.2.1Distributed tracing and log analysis
Distributed tracing stitches the per-service work (spans) of one request into a single trace, so you can see bottlenecks like "embedding is fast but the AI call is stalling." Query the collected logs/metrics with KQL (Kusto Query Language) to analyze error rates, latency distributions, and specific errors. Set threshold alerts to catch anomalies early.
4.2.2AI-specific monitoring
Beyond general metrics (errors, latency), monitor AI-specific signals: token usage and cost (input + output), hitting rate limits (quota), and generation quality (evaluation = groundedness/relevance, as in AI-103). Monitoring and evaluating these also operationally upholds Responsible AI (reliability & safety, accountability).
Common: (1) "trace one request across distributed services" = distributed tracing (instrumented with OpenTelemetry). (2) telemetry sink = Application Insights / Azure Monitor. (3) query language for logs/metrics = KQL. (4) AI-specific monitoring = tokens/cost, hitting quota, generation quality (evaluation).
Watch out: (1) metrics (aggregated numbers) / logs (event records) / traces (request tracking) are different lenses—observability combines all three. (2) OpenTelemetry is the instrumentation standard, separate from the sink (Application Insights/Azure Monitor). (3) KQL is a query language for analysis, not a store. (4) For AI, also monitor cost/quality—general metrics alone are insufficient.
4.2.3Section summary
- Instrument with OpenTelemetry, collect into Application Insights / Azure Monitor; trace requests via distributed tracing
- Analyze logs/metrics with KQL; threshold alerts for early detection
- AI-specific: tokens/cost, quota, generation quality (evaluation) = operational Responsible AI
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. Which stitches one request’s processing across multiple distributed services into a single trace?
Q2. Which is the industry standard for instrumenting code with traces, metrics, and logs?
Q3. Which query language analyzes logs/metrics collected in Azure?
Q4. Beyond general metrics (errors/latency), which should you especially monitor in an AI back-end?
Q5. Which is correct about the three pillars of observability (metrics, logs, traces)?
Keep track of your progress
The full study guide is free to read. Sign up free to practice with the question bank, track what you have read, review your mistakes, and highlight passages.