What's changed: Added per-section figures (cert-figure-retrofit). New AI-103 Chapter 2 (generative AI app = Foundry SDK chat completion/system message/parameters/streaming/structured output & function calling/safety filters; RAG = embeddings/vector search/Azure AI Search index/chunking/hybrid search/retrieve→augment→generate→cite/evaluation = groundedness, relevance)
2.1Implementing a Generative AI App
Understand how to implement a generative AI app by calling a deployed model with the Foundry SDK—chat completion, system message, generation parameters, streaming, and structured output (function calling)—and how to wire in safety filters (guardrails).
The heart of a generative AI app is a chat completion call to a deployed model. With the Foundry SDK (mainly Python) the basic shape is: create a client (endpoint + managed-identity auth) → send a messages array → receive the response. The messages array consists of a system message fixing the role, the per-turn user message, and optionally prior turns (history). AI-103 tests combining this base flow with generation parameters, streaming, structured output, and safety filters.
2.1.1Generation parameters and streaming
Tune responses with temperature (low = stable/accurate, high = diverse/creative) and max tokens (output-length cap). For long responses, streaming returns tokens as they are generated, greatly improving perceived latency—the standard choice for chat UIs. As in planning (Chapter 1), input + output token counts drive billing and context length.
2.1.2Structured output and function calling
To consume model output programmatically, request structured output (a fixed schema like JSON) instead of free text. Going further, function (tool) calling has the model return a structured instruction—"call this function with these arguments"—and your app executes the real function (inventory lookup, booking API, etc.). This is the foundation of agents (Chapter 3). Developers register each function’s name, argument schema, and description, then validate the returned call request before executing.
2.1.3Safety filters (guardrails)
Production apps must include safety filters (content filters/guardrails). Foundry deployments can be configured with Azure AI Content Safety-based filters that detect/block categories—violence, hate, sexual, self-harm—on both input and output by threshold. Additionally, defend against prompt injection (input that hijacks instructions) by protecting the system prompt and validating input. These are production must-haves, not nice-to-haves.
Common: (1) "call the model from an app" = chat completion via Foundry SDK (endpoint + managed identity). (2) "make long responses feel fast" = streaming. (3) "consume output programmatically / call external functions" = structured output / function calling. (4) "block harmful in/out" = Content Safety-based safety filters. (5) "fix the role" = system message; "stable/accurate" = low temperature.
Watch out: (1) system message (role foundation) vs user message (per request) differ. (2) Streaming improves perceived latency but does not reduce total tokens or billing. (3) Function calling only returns "which function/arguments"—actual execution and validation are the app’s responsibility. (4) Apply safety filters to both input and output.
When you need custom model training, fine-tuning, or MLOps, use Azure Machine Learning. While Azure AI Foundry centers building/evaluating/operating generative AI apps, Azure Machine Learning handles training on your own data, pipelines, and model management.
2.1.4Section summary
- Base shape: create client (endpoint + managed identity) → send messages → receive response with the Foundry SDK
- Tuning: temperature/max tokens; use streaming for long responses
- Structured output / function calling for programmatic integration (foundation of agents)
- Apply Content Safety-based filters to in/out + prompt-injection defenses (production must-have)
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. What do you use to call a deployed model from app code to implement generative AI features?
Q2. What improves perceived latency for long responses?
Q3. Which mechanism has the model return a structured "function + arguments" so the app can run external features?
Q4. What underpins safety filters that detect/block harmful input/output in a production generative AI app?
Q5. To reliably consume model output programmatically in an app, what should you request?
Q6. Which is the developer’s responsibility when using function calling?