What's changed: Initial: 4 sections for Domain 3 (AI safety, security, and governance)
3.1Input and Output Safety Controls
Learn defense-in-depth against harmful inputs/outputs: input/output filtering with Bedrock Guardrails, toxicity/moderation, hallucination reduction (Knowledge Base grounding, JSON Schema), and prompt injection/jailbreak detection.
GenAI must defend against both harmful inputs (attack prompts) and harmful outputs (toxicity, hallucinations, PII leaks). Use defense-in-depth across pre-processing, model layer, and post-processing—not a single control.
3.1.1Layered safety
- Bedrock Guardrails: filter inputs/outputs (denied topics, toxicity, PII, words)—the core to enforce Responsible AI rules declaratively.
- Hallucination reduction: ground answers with a Knowledge Base, verify with confidence scores/semantic similarity, and enforce structured output via JSON Schema.
- Defense-in-depth: pre-processing (Comprehend filters) → model layer (Guardrails) → post-processing (Lambda validation) → API layer (API Gateway response filtering).
- Threat detection: detect prompt injection/jailbreak, sanitize input, use safety classifiers, and automate adversarial testing.
Common: safety filter for I/O = Guardrails, reduce hallucination = KB grounding + JSON Schema, attack prompts = prompt injection/jailbreak detection, defense-in-depth, not a single control. Guardrails apply to both input and output sides.
Safe design achieves “keep attacks out, harmful output out, hallucinations out” in layers. On input, pre-process with Comprehend or classifiers and detect/sanitize prompt injection (instruction override) and jailbreak (constraint bypass). At the model layer, block denied topics, toxicity, PII, and words with Bedrock Guardrails (both input and output). On output, add Lambda post-processing validation and API Gateway response filtering. For hallucinations, ground answers with a Knowledge Base (cite sources), fact-check with confidence scores/semantic similarity, fix format with JSON Schema, and prefer determinism (e.g., text-to-SQL) where exact results are needed. For adversarial inputs, bake automated adversarial testing into CI to continuously validate Guardrails. Crucially, do not rely on “Guardrails only” or “prompt only”—layer controls redundantly from pre-processing through the API.
| Threat | Control | Layer |
|---|---|---|
| Harmful input/attack prompt | Injection detection + sanitize | Pre-processing |
| Harmful output (toxicity/PII) | Bedrock Guardrails | Model layer (I/O) |
| Hallucination | KB grounding + JSON Schema | Retrieval + post-processing |
| Ongoing bypass testing | Automated adversarial testing | CI/CD |
Trap: “writing ‘do not ignore’ in the system prompt prevents prompt injection” is wrong—do not rely on the prompt alone; layer detection/sanitization + Guardrails + post-processing. Also “Guardrails only apply to output” is wrong (they apply on input too).
3.1.2Section summary
- Safety = defense-in-depth (pre → Guardrails → post → API)
- Hallucination = KB grounding + JSON Schema / attacks = injection/jailbreak detection
Sign in to track progress — Log in.
Quick check
(just a quick review)Q1. You must filter both input and output so a customer-facing GenAI avoids denied topics and PII output. What is the core control?
Q2. You want to reduce ungrounded hallucinations in RAG-based answers. Which combination is most effective?
Q3. You must defend against “instruction override” attacks where users try to bypass system constraints. Appropriate design?