What's changed: Added per-section figures (cert-figure-retrofit). New AI-901 Chapter 3 (Domain 2: LLM next-token generation, prompt elements = system message/instruction/few-shot/context, temperature, grounding = RAG order retrieve→augment→generate, RAG ≠ fine-tuning, calling via the Foundry SDK)
3.1Generative AI and Prompt Engineering
Understand how large language models (LLMs) generate text by predicting likely next tokens, and the basics of controlling output via prompt elements—system message, instructions, examples, and parameters.
The core of generative AI is the large language model (LLM). Trained on vast text, an LLM generates by selecting, one at a time, the token most likely to come next after the input. So it "weaves a plausible continuation probabilistically" rather than "understanding and answering." Because of this, the same question can yield varying output or include untrue content (hallucination). That is exactly why how you craft the prompt strongly affects quality.
3.1.1Parts of a prompt
- System message: the foundation that sets the AI’s role, tone, and constraints up front (e.g., "You are a polite support agent").
- User instruction (prompt): the specific request each time (e.g., "Summarize this in three lines").
- Examples (few-shot): show a few input/output samples in the desired format to steer output.
- Context: include extra information to reference (e.g., excerpts of internal docs) in the prompt.
3.1.2Parameters that control generation
Output is tuned not only by the prompt but also by parameters. A key one is temperature: lower = more stable/deterministic (nearly the same, conservative answers), higher = more diverse/creative (more variation and ideas). Use low for accuracy-first tasks like FAQ, high when you want variety like brainstorming. You can also set max tokens to cap output length. Treat these as task-dependent tuning, not right/wrong.
| Temperature | Tendency | Best for |
|---|---|---|
| Low (e.g., 0–0.3) | Stable, deterministic, conservative | FAQ, summary, extraction, classification |
| High (e.g., 0.7–1.0) | Diverse, creative, variable | Brainstorming, marketing copy |
Scenario: tuning a support chat. Fix the role via the system message ("be polite and concise; do not assert guesses") → give the user instruction "answer this inquiry" → provide good past answers as few-shot examples → set low temperature for accuracy. These alone greatly improve tone, format, and stability.
Watch out: (1) the system message (role/constraints foundation) differs from the user instruction (per-request). (2) Raising temperature ≠ smarter—it only adds variety and may reduce accuracy. (3) Prompting helps but cannot fully remove hallucination (you also need grounding/human review = next section). (4) Few-shot "teaches format by example"; it does not retrain the model.
Common: (1) "fix the AI’s role/tone up front" = system message. (2) "show a few examples of the desired format" = few-shot. (3) "make answers stable/accurate" = low temperature; "want diverse ideas" = high. (4) "LLMs fully understand meaning" is false—they generate by probabilistic next-token prediction.
3.1.3Section summary
- LLMs generate by probabilistic next-token prediction → output varies and can hallucinate
- Prompt parts: system message / user instruction / examples (few-shot) / context
- Temperature: low = stable/accurate, high = diverse/creative; tune per task
- Prompting is powerful but not a cure-all → combine with grounding/human review (next section)
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. What is the basic mechanism by which an LLM generates text?
Q2. Which element specifies the AI’s role, tone, and constraints at the start of a conversation?
Q3. To make answers stable and deterministic (nearly the same each time), how do you set temperature?
Q4. What technique steers the desired output format by showing a few input/output samples?
Q5. Which statement about raising temperature is correct?