What's changed: In-scope coverage: added an AI/ML support in-scope services block to Ch3 §3 (Amazon SageMaker AI/Amazon Personalize/Amazon Augmented AI = A2I)
3.1Prompt Engineering and Inference Parameters
Understand prompt techniques (zero-shot, few-shot, chain-of-thought) and inference parameters like temperature and context window.
Getting the most from a foundation model hinges on good prompts and right inference parameters. Even without changing the model, these tuning choices change results significantly.
3.1.1Prompt techniques
- Zero-shot: instruct with no examples.
- Few-shot: give a few examples to show the desired format/behavior.
- Chain-of-thought: ask for step-by-step reasoning to improve hard problems.
Prompt engineering is the lightest way to improve output by changing only how you instruct, not the model. Zero-shot (no examples) often suffices, but few-shot (a few examples) helps align format/behavior. For complex problems needing calculation or logic, chain-of-thought ("think step by step") raises accuracy. A system prompt giving the model a role and constraints also matters.
3.1.2Inference parameters
With the same prompt, inference parameters change the output’s character. Temperature and top-p tune the "spread" of next-token selection—lower is deterministic/focused, higher is diverse/creative. Max tokens caps output length; the context window caps the total input+output tokens handled at once. Tune by use: lower temperature for fact-focused tasks, higher for ideation.
| Item | Role | Low/small | High/large |
|---|---|---|---|
| Temperature/top-p | Output randomness | Focused/deterministic | Diverse/creative |
| Max tokens | Output length cap | Shorter output | Longer output |
| Context window | Total tokens handled | Short inputs | Long inputs/many docs |
Scenario: tune by use. For accuracy (e.g., extracting key points from contracts), use lower temperature and few-shot to fix the format. For creativity (e.g., ad-copy ideation), raise the temperature. For inquiries with complex calculation, use chain-of-thought for step-by-step reasoning. To pass a whole long internal document, pick a model with a large context window.
Watch the mix-ups: (1) zero-shot (no examples) / few-shot (examples) / chain-of-thought (step reasoning). (2) Higher temperature = diverse/creative, lower = focused (don’t reverse). (3) Context window (total tokens handled) vs max tokens (output length cap). (4) Prompt tuning doesn’t change the model (unlike fine-tuning).
Q. Few-shot vs chain-of-thought? Few-shot "shows examples"; chain-of-thought "elicits intermediate reasoning"—they can combine. Q. Temperature or top-p? Both tune randomness; typically you use one of them. Q. When prompting is not enough? Consider RAG (next section) or fine-tuning.
Common: differences among zero-shot/few-shot/chain-of-thought, higher temperature = diverse/creative, lower = focused, and context window = max tokens handled. Also: prompt tuning does not change the model.
3.1.3Section summary
- Prompts: zero-shot / few-shot / chain-of-thought (plus system prompt)
- Parameters: temperature/top-p (randomness) / max tokens (output length) / context window (token limit)
- Lower temperature for facts, higher for creativity; prompt tuning leaves the model unchanged
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. Which prompt technique gives no examples, only an instruction?
Q2. To make output more focused (deterministic), how should you set the temperature?
Q3. What term is the max tokens of input + output a model handles at once?