What's changed: Added per-section figures (cert-figure-retrofit). New AI-901 Chapter 3 (Domain 2: LLM next-token generation, prompt elements = system message/instruction/few-shot/context, temperature, grounding = RAG order retrieve→augment→generate, RAG ≠ fine-tuning, calling via the Foundry SDK)
3.2Grounding and RAG
Understand grounding—making a model answer based on up-to-date or organization-specific information it was not trained on—and its common implementation RAG (retrieval-augmented generation), plus how to wire it into an app with the Foundry SDK.
An LLM only has general knowledge up to its training cutoff and does not know up-to-date or organization-specific information; it can also hallucinate. Grounding solves this: provide trustworthy information as context in the prompt and have the model answer based on it. Because answers rest on the documents you supplied rather than the model’s memory, they become current and accurate, and you can cite sources.
3.2.1How RAG works
The common implementation of grounding is RAG (Retrieval-Augmented Generation). Three steps: (1) retrieve documents relevant to the question (from internal knowledge or Azure AI Search, etc.) → (2) add the retrieved excerpts to the prompt context → (3) the model generates an answer informed by that context. This yields answers based on your latest documents without retraining the model. In Foundry you connect data to a project and assemble this retrieve → augment → generate flow.
| Problem | Without grounding | With grounding (RAG) |
|---|---|---|
| Up-to-date info | Knows only up to training cutoff | Can rely on supplied current docs |
| Org-specific info | Unknown | Retrieves internal knowledge to answer |
| Factuality | Hallucination risk | Reduced, with source citation |
3.2.2Wiring it into an app with the Foundry SDK
From an app you call the deployed model via the Foundry SDK (mainly Python). The basics: create a client (endpoint + auth) → pass messages (system + user) → receive the response. To add RAG, when user input arrives you first run a search and add the results to the message context before sending to the model. AI-901 does not test detailed code, but know the roles and order: "the app calls the model via the SDK" and "grounding adds context before sending."
Scenario: an assistant for internal policies. Asked "what is the expense-report deadline?" → (1) search the policy docs for the relevant passage → (2) add that excerpt to the context → (3) the model answers based on it and cites the source (policy name, section). It answers the latest policy accurately without retraining.
Watch out: (1) RAG (add context via search) differs from fine-tuning (update the model itself by extra training)—RAG handles current info without retraining. (2) Grounding is only as good as the supplied material—old/wrong docs cause wrong answers. (3) You can cite sources, but human review is not eliminated. (4) RAG is "retrieve → generate," not generate then retrieve.
Common: (1) "answer based on current/internal info; reduce hallucination" = grounding / RAG. (2) "handle latest docs without retraining" = RAG (not fine-tuning). (3) RAG order = retrieve → add to context → generate. (4) "how an app calls the model" = the Foundry SDK.
3.2.3Section summary
- Grounding = provide trustworthy info as context and answer based on it (current/internal info, better factuality)
- RAG order: retrieve → add relevant docs to context → generate; no model retraining
- RAG ≠ fine-tuning (the latter updates the model itself via extra training)
- Apps call the model via the Foundry SDK; RAG adds context before sending; cite sources but keep human review
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. What is the technique of making an LLM answer based on current or organization-specific info, reducing hallucination?
Q2. Which is the correct order of RAG (retrieval-augmented generation)?
Q3. To answer based on your latest documents without retraining the model, which fits best?
Q4. What do you use to call a deployed model from an app and embed it?
Q5. Which is a correct caution about grounding (RAG)?